Choosing models like a portfolio, not a religion

Start with the failure, because it is a quiet one. A team runs an evaluation over three or four candidate models, one of them wins, and the winner gets written into the system. Not into configuration, into the system: its name inside a client constructor, its habits absorbed into the prompts, its refusal behaviour compensated for in the retry logic, its token accounting baked into the cost model. None of that is unreasonable on the day it happens. Six months later the model has a retirement date, a cheaper model has become good enough for two thirds of the traffic, and the team cannot act on either fact without opening a project, because the choice was never built as a choice.

The word usually attached to this is standardising, which sounds like discipline. What it generally is is a single position taken on a fast-moving asset with no rebalancing mechanism attached to it.

This piece is about the alternative, and the alternative is not "use more models". It is treating model selection as a portfolio problem: a small number of positions, each held for a stated reason, sized to the work it actually does, with an exit path defined before anyone needs one. The reasoning below is deliberately durable rather than current. Model names, prices and benchmark positions move faster than any page describing them, so nothing here depends on which model happens to be ahead this quarter, and no model, version or generation is named anywhere in it.

One model is a position, not a decision

The single-model estate is not wrong because it is inefficient. It is wrong because it hides a decision that is still being made, just not by anyone.

It hides it three ways. On cost, because you pay a uniform rate for work of wildly non-uniform difficulty: sorting an inbound message into one of a dozen categories and drafting a regulated letter to a customer about their claim are not the same task, and paying the same rate for both is a decision nobody took deliberately. On latency, because a request inside a live conversation and an overnight batch have different budgets, and one model has one latency profile. On risk, because every workload you have now depends on one provider's availability, one quota, one pricing decision and one retirement calendar.

The counter-argument is a good one and it deserves stating properly, because it is the reason this piece is not a case for variety. Every model you add is a surface you have to evaluate, monitor, cost, secure, document and eventually migrate. Portfolios carry costs. A three-model estate with no routing tests and no per-model evaluation is genuinely worse than a one-model estate that at least knows what it has. Dependency count is not diversification.

So the argument is about structure rather than number. A model portfolio is a small set of models, each carrying a named class of work, with a routing rule that decides between them, a fallback path for when one is unavailable, and an evaluation gate that has to pass before any of it changes. A collection of models without those four things is not a portfolio. It is an unmanaged dependency count with better marketing.

A position you cannot exit is not a position, it is an exposure. And a portfolio you cannot rebalance without opening a project is one you will not rebalance at all.

If you are making this decision now

• Write down the classes of work before you look at a single model. Cost of being wrong, person on the turn, latency budget. The classes outlive every model that will ever serve them.
• Start with static routing. Route on the call site, then on attributes you already hold. Build a model-based router only when those genuinely run out, and never without a labelled routing test set.
• Separate fallback from tiering. Fall back across availability first, degrade the behaviour before you degrade the model, and make every fallback event loud enough that nobody can have a quiet week on the secondary.
• Get the harness before the second model. Adding models without a substitution test adds risk faster than it adds optionality.
• Keep a register. Model, version, workload, region, contract, data class, owner. It answers the deprecation question, the lock-in question and the audit question at once.
• Query the lifecycle rather than waiting to be told and pin the judge apart from the system it judges.
• Read the licence and the provider documentation before the benchmark table.

The reason "not a religion" is in the title is not that anyone argues theologically about models. It is that a single choice, made once, defended afterwards and never re-tested, behaves like a belief rather than a decision, whatever the evaluation that originally produced it. The alternative is not cleverness about which model is best this quarter. It is holding a small number of positions you can each give a reason for, and being able to change any one of them inside a week, because the machinery that tells you whether the change is safe was built before anybody needed it.