When the AI Wasn't the Problem: How Incomplete Data Broke a Recommendation System

The travel webapp was producing recommendations that looked plausible, but some of them were clearly wrong for the user's preferences. A Beach + Nature traveller, for example, could receive Bangkok among the top recommendations while destinations such as Da Nang were missing entirely.

My first instinct was to investigate the AI ranking system. I had already made two consecutive prompt-level ranking changes. Neither produced a meaningful improvement.

That was the signal to stop tuning the prompt and ask a more fundamental question:

Was the ranking system actually receiving enough information to make the right decision?

The hypothesis

The recommendation pipeline uses several stages:

1. Hard filtering removes destinations that do not satisfy deterministic constraints.
2. AI ranking evaluates the remaining candidates and selects the best-fit destinations.
3. Enrichment adds pricing, flights, weather, visa and other destination information.
4. Validation checks the generated plan against the user's constraints.

The ranking stage can only make a good recommendation from the information available to it. I therefore investigated the destination seed data rather than immediately modifying the ranking logic again. The investigation uncovered a problem: several destinations were effectively hollow records.

For some destinations, the interest data looked like this:

interest_tags: ["travel"]

That meant a destination could be a genuine beach-and-nature destination in the real world, while the system had almost no structured information telling it that. The system was not necessarily ranking the destination incorrectly. It was ranking an incomplete representation of the destination.

The experiment

I selected four Southeast Asian destinations that had been represented as stubs:

• Kota Kinabalu
• Da Nang
• Krabi
• Luang Prabang

I replaced the hollow records with real destination data, including:

• airport codes
• timezones
• flight times
• airlines
• hotel chains
• cost breakdowns
• interest tags
• climate data
• visa requirements

The key constraint was deliberate:

Change the data, not the ranking system.

No ranking weights were changed.

No scoring logic was changed.

No reduction logic was changed.

No new prompt was introduced.

This created a clean experiment: if recommendations improved, the result would provide evidence that the underlying destination representation was part of the problem.

The result

The most useful test was a Beach + Nature persona. Before the data update, Da Nang effectively had no meaningful signal for the ranking system indicating that it was a strong Beach + Nature destination. After adding its real beach and nature attributes, Da Nang moved into the top position. At the same time, Bangkok moved from #3 to outside the top three.

That was significant because nothing in the ranking algorithm had changed. The recommendation changed because the system finally had better information about one of its candidates.

Before → after

Persona	Before	After	Interpretation
Beach + Nature	Bangkok #3; Da Nang absent	Da Nang #1; Bangkok outside top 3	Targeted improvement after destination data enrichment
Food + Culture	Beijing, Bali	Beijing, Bali	No unintended change

The second test was particularly important. If enriching Da Nang had caused recommendations to change indiscriminately, that could have indicated a broader ranking instability. Instead, the Food + Culture scenario remained unchanged.

That gave us stronger evidence that the intervention was targeted rather than globally disruptive.

Tracing the data through the pipeline

I also verified that the newly added destination data was not simply appearing in the seed file but being lost somewhere downstream.

For Da Nang, the cost, hotel and airline information propagated through the pipeline correctly. The pricing metadata also identified the source as a seed estimate, allowing the downstream calculations to be traced back to the intended source. This mattered because fixing an upstream data problem is only useful if the information actually survives the rest of the system.

The investigation therefore became less about:

"Did the destination data look correct?"

and more about:

"Can I trace the corrected information all the way to the final product output?"

What this changed in my mental model

The biggest lesson was that AI output quality is a system property, not just a model property. When an AI recommendation looks wrong, there are several possible failure points:

User constraints
      ↓
Hard filtering
      ↓
Candidate universe
      ↓
Destination data
      ↓
AI ranking
      ↓
Enrichment
      ↓
Validation
      ↓
Final recommendation

It is tempting to look at the AI ranking stage because that is the most visible "AI" component, but the failure may exist upstream. A model can only reason over the candidate representation it receives.

That led me to a broader debugging principle:

Before changing the model, verify that the model has the information required to make the decision you are asking it to make.

Why prompt tuning had failed

The earlier prompt experiments were not useless. They were simply operating at the wrong layer. I was attempting to improve the ranking behaviour while the system's representation of the candidates was incomplete.

This creates a classic product-development trap:

Observed problem: The recommendation is wrong.

Initial interpretation: The AI needs a better ranking instruction.

Intervention: Modify the prompt.

Result: Little or no improvement.

Deeper diagnosis: The AI does not have sufficient structured information about the candidate.

Better intervention: Improve the candidate representation.

The important PM skill here was not writing a better prompt. It was recognising when not to keep prompting.

The reliability lesson

I used to think about reliability primarily in terms of things like:

• latency
• API failures
• timeouts
• retries
• infrastructure stability

Those are important, but this experiment expanded the definition.

For an AI product, reliability also includes:

Does the system have sufficiently accurate and complete inputs to consistently produce trustworthy outputs?

A system can have:

• successful API calls,
• low error rates,
• valid JSON,
• fast execution,

and still produce poor recommendations because the underlying knowledge is incomplete.

In other words: Technical reliability ≠ product reliability. The latter includes the quality of the information flowing through the system.

The product decision

The experiment did not justify a wholesale rewrite of the recommendation system. Instead, it gave me a more targeted path.

There are approximately 28 other destinations across regions including South Asia, South America, Australia, the UK/Ireland, Eastern Europe and the Pacific Islands that still contain the same hollow ["travel"] stub pattern.

I deliberately did not backfill all of them immediately. The next expansion is scoped to an explicit product need rather than becoming an open-ended data-cleaning exercise. That distinction matters.

The goal is not:

"Make every destination record perfect."

The goal is:

"Improve the candidate universe where insufficient destination knowledge is demonstrably affecting user outcomes."

What I learned

1. Don't confuse an AI problem with a model problem

When an AI output is poor, inspect the entire pipeline before changing the model or prompt.

2. Data quality is part of AI product quality

Structured destination attributes can materially affect ranking behaviour even when the ranking logic remains unchanged.

3. Isolate variables when debugging AI systems

Changing the data while keeping the ranking system constant created a much cleaner experiment than changing both simultaneously.

4. Validate targeted effects, not just positive effects

Da Nang improving was useful. Food + Culture remaining unchanged was even more useful. It showed that the intervention was not simply causing arbitrary recommendation movement.

5. Good AI PM work includes knowing what not to change

The most valuable decision in this investigation was ultimately a non-decision: I did not rewrite the ranking system. The evidence did not justify it.

The broader principle

AI systems are often described as: Model + Prompt + Tools

In practice, a production AI product is much closer to: User input + Product data + Retrieval + Deterministic logic + Model + Validation + UX

Failures can emerge at any layer. That means AI product debugging requires a systems mindset. Sometimes the best way to improve an AI system is not to make the AI smarter. It is to make sure the AI has something better to work with. The model wasn't the problem. The system's representation of the world was.