AgenticGENERALLIFETIME-SPECIFICCONTESTEDSCALE-SPECIFIC

The Model Is a Dependency

Non-deterministic, fallible, mutable under you, priced per call and slow. Four of those five you already know how to design around; the fifth is the only genuinely new thing.

Constraints & invariants Who owns what What the next change costs

The requirement, the obvious build, and why it breaks

Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind survives until the requirement changes.

The question

What kind of dependency is a language model, and which of my existing techniques still apply to it?

The requirement

A product manager wants "AI summaries" on the ticket list. Engineering estimates two days, because it is one API call.

The obvious build

Import the SDK where you need it. It is one call — const summary = await openaiish.complete(prompt) — and wrapping a one-line call in an interface is exactly the speculative ceremony this domain warns about. That reasoning is correct in general, which is why the mistake is so easy to make.

Why it breaks

The provider deprecates the model. The call site is in eleven files, each with a slightly different prompt and its own retry, and the migration is a codebase-wide grep rather than an edit (Shotgun Surgery).

How it breaks as requirements change

The provider deprecates the model. The call site is in eleven files, each with a slightly different prompt and its own retry, and the migration is a codebase-wide grep rather than an edit (Shotgun Surgery).
CI gets slow, flaky and expensive, because the tests call the real thing. The team's response is to skip those tests, which removes the only signal they had (Volatile Dependencies).
A prompt tweak improves ticket summaries and quietly degrades the release-notes feature, because both call sites share a helper that someone made generic (Duplicate Knowledge).
The first time the provider is slow, every request thread is parked for thirty seconds and the page falls over — a textbook missing timeout, invisible because the call did not look like a network call (What Changes at the Network Boundary).
A hundred-fold traffic increase turns a rounding-error line item into the largest single cost in the service, and nobody can attribute it per feature because there is no single place to measure (Cost-Aware Interfaces).

Requirement→Constraints→Invariants→Responsibilities→Boundaries→Interfaces→State→Dependencies→Failure→Implementation→Tests→Feedback→Evolution

What limits the solution, and what must never stop being true

This domain leads with these two. A design that ignores its constraints is not a design, and an invariant nobody named is one nothing is protecting.

Constraints

The provider is external, versioned on their schedule, and can deprecate a model with weeks of notice.
Cost is per token and scales with traffic, which means a feature can become unaffordable by succeeding (Designing for Cost).
Latency is seconds, not milliseconds, and the tail is much worse than the median (Tail Latency: Why p50 Being Fine Does Not Help in Observability & Performance).
The team already has a working vocabulary for external dependencies — timeouts, adapters, injection, fakes — and no vocabulary at all for sampled output.

Invariants

A model outage degrades the feature; it never takes down the page that shows it (Error Boundaries).
No test in the main suite calls the provider. A build must be reproducible and must not cost money (A Deterministic Core).
The vendor's request and response shapes never appear in domain code (Anti-Corruption Layer).

Who owns what, and where the seams fall

Responsibilities decide boundaries; boundaries decide what an interface has to say.

Responsibilities

A port owns the *capability*, stated in your language: summarize(ticket): Summary. Not complete(prompt): string, which is the vendor's language and leaks their model of the world into yours (Designing a Module Interface).
An adapter owns everything vendor-shaped: prompt construction, the wire format, the retry policy, the token accounting, the model identifier (Boundary Adapters).
The domain owns the meaning of the result and what happens when there is not one. It must not know a model exists.
Configuration owns which model, at which version, with which prompt revision — as data, not as a literal in the middle of the adapter (Prompts and Models Are Deployables in DevOps owns the release side).

Boundaries

One seam, at the capability. Everything vendor-specific is on the far side of it, and the test for whether the seam is in the right place is whether swapping providers is an adapter change (Stable Boundaries).
Not one seam per call site. Eleven thin wrappers around the same SDK is the same coupling with extra files (Over-Decomposition).
The seam is also where cost, latency and failure are measured, because it is the only place that sees every call (Logging at Boundaries).

Five properties, and where you have met each before

Laying the properties out side by side is the point of this lesson, because four of the five rows have an answer you already own. The value of the table is that it makes the fifth row stand alone, so it can be given the attention it actually needs instead of being lost in generalised anxiety about AI.

The first four rows are why the seam exists at all — they would justify it for a geocoding API.
The fifth row is why the seam has to be *narrow*: every decision you leave on the model's side of it inherits the property, and the cost of inheriting it is that you can no longer write an assertion, only an estimate.
Randomness as an injected dependency is the closest existing analogue, and it is a genuinely useful lens — with the difference that a seeded RNG can be made reproducible and a model, in practice, cannot (Randomness as a Dependency).

Property	Where you have met it	What you already do about it
Can fail	Any network dependency	Timeout, bounded retry, circuit breaker, defined degraded behaviour (Designing for Failure)
Is slow	A remote call in a request path	Move it off the request path, cache, or budget the latency explicitly (What Changes at the Network Boundary)
Costs money	A metered third-party API	Measure at one boundary, budget per tenant, alert on the derivative (Designing for Cost)
Changes under you	A vendor SDK with a deprecation policy	Own the interface, adapt at the edge, pin and version deliberately (Anti-Corruption Layer)
Is non-deterministic	Nowhere. This is the new one.	Confine it. One component whose tests are statistical, and a deterministic system around it (A Deterministic Core)

The seam, in about twenty lines

The port speaks your language, not the vendor's. That is the whole test: if a reader of the interface can tell which provider is behind it, the abstraction has not been drawn, only renamed.

Note that the failure is in the return type rather than thrown. A summary that could not be produced is an ordinary outcome for this feature — it happens several times a day — and expressing an ordinary outcome as an exception pushes it into a handler far away from the code that knows what to do about it (Result Types).

A port in domain language, and what the adapter is allowed to know

1// domain side — knows nothing about models, tokens or vendors
2export interface Summarizer {
3  summarize(ticket: Ticket): Promise<Result<Summary, SummaryUnavailable>>
4}
5 
6export type Summary = {
7  text: string
8  // provenance: which model + prompt produced this, recorded on the value
9  producedBy: { modelId: string; promptRevision: string }
10}
11 
12export type SummaryUnavailable =
13  | { kind: 'timeout' }
14  | { kind: 'unparseable' }
15  | { kind: 'budget_exhausted' }
16  | { kind: 'provider_error'; status: number }
17 
18// adapter side — the only file in the codebase that imports the SDK
19class VendorSummarizer implements Summarizer { /* prompt, retry, parse, meter */ }
20 
21// test side — deterministic, free, and never off by a token
22class FakeSummarizer implements Summarizer { /* canned Summary, or a chosen failure */ }

producedBy is the field teams add after their first unexplained behaviour change and wish they had had from the start. Without it, "the summaries got worse last Tuesday" is unanswerable, because the model and the prompt both move and neither is recorded on the output (Debuggability by Design).

What the seam costs, honestly

The argument for a port is routinely made in terms of switching providers, and that argument is weak: most teams never switch, and the ones that do find the port underfitted the new vendor anyway. The strong arguments are duller and much more reliable — a test suite that does not cost money, one place that knows the model id, and one place that can tell you what the feature spends.

The matrix is a comparison of three real positions, not a recommendation with two strawmen either side. Direct SDK calls are the right answer for some systems, and saying so is part of teaching the trade-off honestly.

Three ways to hold a model dependency

Option	Simplicity	Flexibility	Testability	Operational	Migration cost	Note
SDK called directly at each site						Fastest to write and genuinely correct for a prototype or a single call site with a short life. Its cost is that there is no single place to apply a timeout, change a model, or count what you spend.
One domain port + one adapter						The recommendation for anything expected to live a year. Testability is the axis that carries it: the domain becomes deterministic, which is the property everything else in this domain assumes.
A general multi-provider abstraction layer						Flexibility scores lower than teams expect, because a lowest-common-denominator interface cannot express the capabilities that differ between providers — which are exactly the ones worth having.

caveat These numbers compare structures, not outcomes, and they cannot express the thing that decides the choice in practice: how many call sites there are and how long the code has to live. At one call site for six weeks the first row wins outright and the scores are misleading. Nor can a matrix price the risk it is buying insurance against — a deprecation notice with a six-week deadline is either an afternoon or a quarter, and which one it is was decided long before the notice arrived.

How to build it

Most important first.

Define the port in domain terms and let the vendor SDK exist only behind it. This is unremarkable dependency inversion and it is unremarkable on purpose (Dependency Inversion).
Treat every call as a network call, because it is: timeout, bounded retry with jitter, circuit breaker, and a defined behaviour when it does not answer (Designing for Failure, Retries Are a Property of the Operation).
Decide what the feature does with no answer, and make that path ordinary rather than exceptional. For summaries the answer is easy — show the ticket without one — and the design that makes it easy is the one where the summary was always optional (Optional Values and Absence).
Make the model identifier and the prompt revision explicit inputs to the adapter, recorded on every output. When behaviour changes you need to know which pair produced it, and reconstructing that later is impossible (Stable Identifiers).
Give the domain a deterministic fake, not a mock of the SDK. A fake that returns a canned Summary keeps tests fast, free and honest about the boundary; a mock of the vendor's response shape pins your tests to the vendor (Test Doubles, Precisely).
Budget the cost at the boundary the same way you budget latency: per request, per tenant, per day, with a defined behaviour on exceeding it (Budgets, Deadlines and Step Limits in Backend Engineering owns the implementation).

What the next change costs

The field this whole domain exists for. A structure is only better if it makes the change after this one cheaper — and it is worth saying which changes it does not help.

Cost of the next change

A model upgrade: one line of configuration, plus re-running the eval set for the one capability behind the port. It is a config change with a measurement attached. Without the seam it is a regression hunt across every call site, with no list of what to check, because nobody ever wrote down what each prompt was relied upon to do.
A provider switch: one new adapter, one wiring change, and the eval set is reused unchanged because it tests the capability, not the vendor. The domain and its tests do not move at all.
A prompt change: contained to the adapter and versioned, so a regression can be attributed to a revision and rolled back. Without the seam, prompts live next to whatever code needed them and there is no revision to roll back to (Rollback: Only Useful If It Is Actually Safe in DevOps).
What stays expensive: changing the *shape* of the capability — going from summarize(ticket) to summarize(ticket, audience) — crosses the port and every caller. The seam bought vendor and model independence, not interface independence, and pretending otherwise is how ports acquire optional parameters until they are wire formats again.

What the recommended approach costs

The port is indirection, and for a single call site in a product that will never change providers it is cost with a weak return. The defence is testability and cost attribution rather than portability, and if you do not care about either, do not build it.
A domain-shaped port hides capabilities the vendor offers — structured output modes, caching, streaming — and you will occasionally have to widen it in ways that feel like leaks. Some of them are.
Deterministic fakes make tests fast and make them agree with a world that does not exist. The eval job is what stops that becoming dangerous, and it is extra machinery someone has to own.

What can go wrong

Failure modes

The port leaks. A temperature parameter or a messages array appears on the interface, and now the domain knows about token sampling and the seam has stopped being one (Leaky Abstractions).
The abstraction is built for provider portability that never happens, and it costs indirection every day for a switch nobody makes. This is a real risk and the honest defence is that the seam pays for itself in testability and cost attribution even if you never switch (Speculative Generality).
Retries are added at the adapter and also by the SDK, so a timeout produces four calls, four charges, and — if the call had an effect — four effects (One Retry per Tier Is Not One Retry — It Multiplies in Distributed Systems).
The fake diverges from reality. It always returns well-formed output, so nothing in the test suite has ever seen the empty, truncated or refusing response that production sees weekly.

Dependencies, and their direction

Domain -> port (an interface you own). Adapter -> port and -> vendor SDK. Nothing in the domain points at the vendor, which is the whole arrangement (Dependency Direction).
The adapter depends on a prompt, which is now a versioned artifact with a lifecycle rather than a string literal (API Stability).
The system acquires a dependency on the provider's availability, pricing and deprecation policy — three things you do not control and cannot test (Do We Need a Package for This?).

Misreads

"It is just an HTTP call." Everything except one property, yes — and that property invalidates the assumption every other integration test rests on, which is that the same input gives the same output (Determinism: Same Input, Same Output? in Concurrency & Parallelism).
"So wrap every third-party call in an interface." No. This one earns it because it is volatile on four axes at once — version, price, latency and behaviour. A stable, cheap, deterministic library does not, and wrapping it is the ceremony this domain criticises elsewhere (Premature Abstraction).
"Non-determinism means untestable." It means the assertion changes shape: distributions and thresholds instead of equality. Confining that to one component is precisely why the boundary is worth drawing (Deterministic Evaluators in the agentic domain).
"Pinning the model version makes it deterministic." It does not — sampling remains, and providers can change serving behaviour behind a stable name. Pinning reduces one source of drift and is worth doing; it is not a determinism guarantee.

Smells this explains

leaky-abstractions
shotgun-surgery

Testing it, and how it ages

What to test, and at which boundary

Domain tests use the fake and are fully deterministic. If a domain test can fail because of a sampling outcome, the boundary is in the wrong place (Purity and Testing).
Adapter tests use recorded responses, including the ugly ones: truncated output, a refusal, valid JSON with a missing field, and a response that arrives after your timeout.
A small eval set runs against the real provider on a schedule and on model upgrades — not in the main suite. It is a monitoring job that happens to be written as tests (Regression Gates and Online Evaluation in the agentic domain).
Property tests on the parse step: whatever the model returns, the adapter either produces a valid Summary or a typed failure, and never a half-populated object (Property-Based Testing).

How this design ages

Capability, price and latency are all moving fast. A design that assumes today's cost per call is a design with an expiry date, and the cheapest response is to keep the cost visible at one point rather than to optimise around a number that will change.
The port narrows over time, not widens. Every parameter that gets added to make one call site work is a small leak, and the healthy version of this design has an interface that looks the same after two provider changes.
Eventually some capabilities move in-process or become cheap enough to call speculatively. That changes the latency and cost arguments completely while leaving the non-determinism argument exactly where it was.

Where this applies

This domain's advice is contested more than most. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view rather than a caricature.

GENERALThat a volatile external dependency wants a seam you own holds for payment providers, geocoders and search engines alike. What is specific here is the reason the seam pays: not portability, which rarely happens, but the ability to test deterministically on one side of it.
LIFETIME-SPECIFICFor a three-week internal tool, the SDK call inline is right and the port is waste. The argument sharpens with expected lifetime because the deprecation and price changes it defends against arrive on the provider's schedule, not yours — typically within a year.
CONTESTEDThe strongest counter: provider abstractions consistently underfit, because each vendor's best capabilities are the non-portable ones, and teams that build a lowest-common-denominator port end up with a worse product than teams that used one SDK well and accepted a migration cost if it ever came. That case is real; note that it argues against a *portability* port, not against a seam for testing and cost attribution, and the two are often conflated.
SCALE-SPECIFICAt one call site the seam is arguable. At eleven it is not, because the cost is no longer indirection but the absence of any single place to change the model, measure the spend, or apply a timeout.

Where the depth lives

This domain teaches the codebase-level structure and hands the rest off.

Architecturehexagonal-architecture

Backendagent-budgets external-dependencies circuit-breaker timeouts

Performancellm-latency agent-cost tail-latency

Concurrencydeterminism nondeterminism

Distributed Systemsretry-amplification timeout-ambiguity

DevOpsprompt-and-model-versioning rollback agent-production-cost

Agenticstructured-outputs deterministic-evaluators regression-and-online-evaluation fallbacks-caching-routing evals-overview

Domains that do not exist yet

— Testing & Reliability Engineering — deciding what "good enough" means for a statistically-tested component, and how a threshold on a distribution becomes a release gate, is their question rather than ours.
— Programming Languages & Runtime Internals — how far a type system can carry the proposal/decision distinction differs sharply by language, and the mechanics of sum types and exhaustiveness belong there.