What “leading” actually means in a buying context
Public leaderboards reward reasoning puzzles and multi-lingual trivia. Procurement care about a narrower set of outcomes: grounded answers over internal knowledge, reliable tool calling, predictable latency at p95, and contractual terms that survive a legal review. A model that tops a public eval can still fail your invoice-classification queue or leak proprietary phrasing into user-facing copy.
Treat model selection as a portfolio decision. Most mid-market and enterprise stacks now run at least two tiers: a high-capability model for complex synthesis and planning, and a faster, cheaper model for classification, extraction, and routine drafting. Routing logic between them is often where the real savings appear.
Capability clusters you will actually shortlist
General frontier APIs
Closed frontier models from the major labs remain the default for multi-step reasoning, code generation, long-context analysis, and high-stakes writing. Strengths include aggressive tool-use support, multimodal inputs (documents, images, sometimes audio), and frequent capability updates. Weaknesses include price volatility, rate-limit choreography, data-residency constraints, and limited ability to customise weights.
Efficient mid-tier and “workhorse” models
Distilled and mid-size commercial models often deliver 70–90% of frontier quality on structured tasks at a fraction of the token cost. They shine in customer-support deflection, CRM field fills, meeting-note cleanup, and first-pass research packs. Many teams lock a workhorse model into pipelines while reserving frontier calls for escalation paths when confidence scores or human review flags demand it.
Open-weight deployments
Open-weight models have matured enough for serious private deployment—especially when data cannot leave a VPC, air-gapped environment, or regulated cloud tenancy. Expect heavier MLOps load: GPU capacity planning, quantisation choices, guardrail layers, and continuous evaluation. The payoff is cost control at volume, deeper fine-tuning, and clearer data custody. The risk is silent quality drift if you lack an eval harness.
Comparison axes that matter more than brand
| Axis | Why it matters | Operator tip |
|---|---|---|
| Context window | Determines how much policy, contract, or ticket history you can hold without brittle chunking. | Measure usable context after tool schemas and system prompts, not headline token counts. |
| Tool-calling reliability | Agent loops collapse when arguments are malformed or tools are invented. | Log schema-validation failures as a first-class KPI. |
| Latency & throughput | Interactive UX budgets differ sharply from overnight batch jobs. | Track p95, not averages; test under concurrent load. |
| Pricing shape | Input/output asymmetry, cached tokens, and batch discounts change unit economics. | Model monthly cost on real prompt/response ratios from pilot logs. |
| Governance surface | Retention controls, regional endpoints, and audit logs gate enterprise adoption. | Align security questionnaire answers with architecture diagrams early. |
Pricing and specs without the hype cycle
As of this edition, production teams commonly see frontier text generation priced in the low-to-mid dollars per million input tokens and higher for output tokens, with steep discounts for batch, cached, or smaller sibling models. Open-weight self-hosting shifts cost into GPU hours, platform engineering, and observability—often cheaper above steady high volume, more expensive below it.
Context windows ranging from roughly 128K tokens to multi-million-token extremes exist in market, but “can accept” does not equal “should dump.” Retrieval quality, citation discipline, and memory design usually beat naive long-context stuffing. Multimodal pricing may meter images, pages, or audio minutes separately; model these lines explicitly in TCO sheets.
Pros and cons by deployment pattern
Single-vendor frontier stack
Pros: fastest time-to-value, integrated tooling, strong default safety layers. Cons: concentration risk, negotiating leverage limited, sudden model deprecations force retesting.
Multi-model router
Pros: cost/performance balance, resilience if one provider degrades. Cons: prompt portability is imperfect; eval surface multiplies; security review broadens.
Private open-weight core
Pros: data control, customisation, predictable marginal cost at scale. Cons: requires specialised staff, slower access to frontier leaps, harder safety evaluation without a mature lab apparatus.
A practical selection sequence
- Write five golden tasks with accept/reject criteria and sample artefacts—not vague “improve productivity” goals.
- Run blinded comparisons on those tasks with fixed prompts and tool sets; score with humans and automated checks.
- Instrument cost per successful task, not cost per thousand tokens alone.
- Test failure behaviour: refusals, hallucinations on missing knowledge, and tool errors.
- Only then negotiate contracts, residency, and rate limits around the shortlist that already works.
Models are the substrate. The subsequent chapters cover agents that plan over them, services that productise them, platforms that schedule them, and tools that measure them. Skip the substrate decision and every layer above inherits your mistake.