The question

One afternoon, I faced the same architectural question twice:

Should I introduce a second LLM into the recommendation pipeline?

The proposed model was DeepSeek. The first proposal was rejected. The second was implemented. At first glance, that might look inconsistent. It wasn't.

The two proposals were trying to solve fundamentally different problems. That distinction became the basis for the decision.

Proposal A: Use DeepSeek to improve recommendations

The first proposal suggested routing several recommendation sub-tasks through DeepSeek:

1. Identify the airlines commonly used by travellers from the user's home city and use that to determine an airline/cabin recommendation.
2. Suggest a specific hotel and its price.
3. Have Claude select the top three candidate cities, then have DeepSeek identify compatible multi-city pairings and construct the itinerary.

The proposed architecture also aimed to run these tasks concurrently to stay within a 60-second generation budget.On the surface, this seemed attractive.

A second model could potentially:

• distribute workload,
• reduce latency,
• give different reasoning behaviour,
• or improve recommendation quality.

But I stopped at the more fundamental question:

What problem would adding DeepSeek actually solve?

The first diagnosis: the missing capability wasn't another model

The travel webapp did not have live hotel or flight inventory behind its enrichment layer. Flight and accommodation pricing was still calculated using formulas and multipliers rather than retrieved from a live inventory source.

That meant a request such as:

"What are the top airlines people from Singapore fly?"

or:

"Suggest a hotel and its price."

did not have a reliable source of truth behind it.

An LLM could produce a plausible answer, but plausibility isn't the same as grounding. Switching from Claude to DeepSeek would therefore change which model generated the guess. It would not change the fact that the application lacked the underlying data.

That led to the first decision:

Don't introduce a second model to compensate for missing ground-truth data.

If live hotel information becomes important enough to justify building, the appropriate solution is a real data source, such as a hotel inventory API, not another model's parametric knowledge.

Decision: Reject Proposal A

No code was written.

The proposal was documented as rejected so that the same idea would not quietly reappear later without its original reasoning being reconsidered.

There was another problem: the proposed parallelism wasn't actually parallel

The multi-city proposal contained another architectural assumption.

The idea was to have Claude select the top three candidate cities and have DeepSeek simultaneously determine which cities could form viable pairs, but the second task depends on the output of the first. You cannot meaningfully pair cities before you know which cities have been selected.

The actual dependency is:

Claude
Select candidate cities
        ↓
      Top 3
        ↓
DeepSeek
Evaluate compatible pairings
        ↓
Multi-city itinerary

So even if the second model were useful, the correct architecture would be:

Select first → then parallelise independent downstream work.

Not:

Run everything simultaneously.

This was a useful reminder that adding models does not automatically create parallelism. The dependency graph still determines what can actually run concurrently.

Proposal B: Use DeepSeek for live pricing search

Later that same afternoon, a different proposal surfaced. This time, the problem was not recommendation quality. It was a persistent reliability problem in the existing live pricing search.

The application was already using web search to retrieve grounded flight and hotel pricing. The problem was that these searches were timing out.

Earlier fixes had already been attempted:

• a shared concurrency pool had been introduced
• simultaneous live searches had been capped
• the timeout budget had been increased from 9 seconds to 15 seconds

Those changes reduced some contention, but a live test still showed a single pricing search inside a real request exceeding its budget.

The question became:

What is the search actually competing with?

The root cause was concurrency contention

The investigation showed that pricing searches and the recommendation/prose calls were competing for the same host:

                 api.anthropic.com
                       │
          ┌────────────┴────────────┐
          ↓                         ↓
   Recommendation              Live pricing
       calls                       search
          │                         │
          └──── shared capacity ────┘

The application had introduced a concurrency limit, but that limit was still shared across workloads. Pricing search was competing with the very AI calls responsible for generating the recommendation. At this point, introducing a second model had a completely different rationale.

It wasn't being introduced because:

"DeepSeek might be better at recommendations."

It was being introduced because:

A second provider could give the pricing workload a separate execution path and therefore a separate concurrency budget.

That is an architectural decision, not a model-quality decision.

The important distinction

The two proposals used the same technology but solved different problems.

Proposal	Underlying problem	Would DeepSeek solve it?	Decision
Recommendation sub-tasks	Missing reliable hotel/airline data	No	Rejected
Live pricing search	Contention on shared Anthropic capacity	Potentially	Shipped for testing

This became my decision rule:

Don't add a model because you have a problem. Add a model only when the model or its infrastructure directly addresses the problem.

What actually changed in the architecture

The second proposal was implemented behind an environment variable:

PRICING_SEARCH_PROVIDER=claude

remains the default.

Setting:

PRICING_SEARCH_PROVIDER=deepseek

routes the existing pricing-search client through DeepSeek's Anthropic-compatible endpoint.

Critically, I did not redesign the pricing system.

The application continued using:

• the same live web-search tool
• the same search purpose
• the same prompt contract
• the same JSON response structure

The primary change was the execution host. That distinction matters.

This wasn't:

"Let's ask DeepSeek to know the hotel price."

It was:

"Let's run the same grounded search workload through a different provider so it isn't competing for the same execution capacity."

The model provider changed. The source of truth did not.

An unexpected integration issue

The first live integration surfaced another difference between providers. DeepSeek's endpoint was spending tokens on thinking blocks around the search request. At the existing 512-token ceiling, requests could reach: stop_reason: max_tokenswithout returning useful text.

The fix was to explicitly disable thinking for this workload and raise the token ceiling to provide sufficient headroom.

This reinforced another reason to avoid casually adding providers:

A second model is never just another API endpoint.

It introduces another set of:

• API semantics
• latency characteristics
• token behaviour
• failure modes
• configuration requirements
• observability requirements

The integration itself has a cost.

The cost of a second model

Before introducing DeepSeek, I considered the additional system complexity:

Integration cost

• new API key
• new client configuration
• new endpoint
• provider-specific behaviour

Reliability cost

• new failure modes
• unknown concurrency characteristics
• another external dependency

Observability cost

The existing Langfuse tracing primarily covered Anthropic-hosted calls. Introducing another provider meant making sure the new path could be observed and debugged.

Product cost

Every additional model makes the system harder to reason about.

If something changes after deployment, the team needs to know:

Was it the prompt?

The model?

The provider?

The endpoint?

The concurrency limit?

The tool?

The response format?

Model diversity can be valuable.

But it should earn its complexity.

What I am not claiming yet

The DeepSeek integration is shipped into the development environment, not yet proven successful. That distinction is important.

The architectural hypothesis is:

Separating pricing search from the Anthropic host will reduce contention and improve live-search reliability.

But I do not yet have the before-and-after evidence to claim that it worked.

The next experiment is therefore straightforward:

Compare

Before

Claude pricing search under the existing architecture.

After

DeepSeek pricing search through the separate host.

Measure:

• live-search success rate
• timeout rate
• P50 latency
• P95 latency
• end-to-end generation latency

If the timeout simply moves from one provider to another, the hypothesis was wrong.

If timeout rates fall materially while recommendation quality remains unchanged, the architectural change is justified.

The decision framework

This experience gave me a simple framework for deciding whether to introduce another model.

1. What is the actual problem?

Is it:

• model quality?
• missing data?
• latency?
• concurrency?
• cost?
• reliability?
• capability?

2. Does another model directly address that problem?

If the problem is missing ground-truth data, another LLM is probably not the answer.

If the problem is execution contention, another provider might be.

3. What new complexity does it introduce?

Consider:

• integration
• observability
• latency
• reliability
• cost
• debugging
• provider-specific behaviour

4. Can I isolate the change?

A provider switch behind an environment variable is easier to experiment with than rewriting the entire pipeline around a second model.

5. What evidence would make me keep or remove it?

Every architectural experiment should have a kill condition.

In this case:

If DeepSeek does not materially improve live pricing reliability, revert to Claude.

The broader AI product lesson

The easy answer to an AI product problem is often:

"Use another model."

Sometimes that is exactly right, but models are not interchangeable magic boxes.

A product problem might actually be caused by:

Data

→ the model doesn't have the information it needs.

Architecture

→ the model is being called in the wrong place.

Orchestration

→ dependent tasks are being treated as parallel.

Infrastructure

→ workloads are competing for the same capacity.

Evaluation

→ the team hasn't established whether the output is actually better.

UX

→ the model output is technically correct but doesn't help the user.

Adding another model without identifying the actual failure mode simply adds another moving part.

The better question is:

What capability is missing from the system, and what is the simplest mechanism that provides it?

Sometimes the answer is another model.

Sometimes it is an API.

Sometimes it is deterministic code.

Sometimes it is better data.

And sometimes the best product decision is to add nothing at all.

The decision

Two DeepSeek proposals arrived on the same day. One was rejected because it attempted to use a second LLM to compensate for missing ground-truth data. The other was implemented because the application already had grounded search and the problem was architectural contention. The technology was the same but the product reasoning was different.

That's the principle I want to carry forward:

Don't ask whether you should use another model. Ask what problem another model uniquely solves - and whether that benefit is worth the complexity it introduces.