The application cleared its release checks on a Friday afternoon. By Monday morning, the downstream reconciliation queue showed a spike in manual corrections. No engineering team had deployed a new build. The health checks remained green, and the API continued to return successful responses. The model behind the floating alias had been updated overnight, and it had started wrapping its JSON output in markdown code fences. The strict downstream parser choked on the first character.
Pointing production code to an alias like latest or a generic model name delegates your change control to a vendor’s release schedule. The API contract did not break. The behavior did. Those are different things. A standard vendor service level agreement guarantees the endpoint will be available and latency will stay under a certain threshold. It does not guarantee the model will interpret English the same way it did on Tuesday.
Pinning the model version in the configuration file looks like the conservative choice. It prevents the silent overnight shift. But a pinned version is a lease, not a permanent dependency. Eight months later, the API returns a 404 error. The deprecation email went to a distribution list a contractor set up two years ago that nobody monitors.
Now the application team has to take a model replacement to the Change Advisory Board. The board expects deterministic answers. They want a changelog, a regression test report, and a rollback plan. The application owner has to explain that the changelog is a vendor blog post claiming improved reasoning, the regression test is a probabilistic evaluation against a golden dataset, and the rollback plan has a vendor-mandated expiration date of thirty days. A rollback plan that names a retired model is just a record of what used to be possible. The board rejects the change for insufficient testing. The vendor deprecation date remains thirty days away.
The fundamental issue is that a prompt is not portable code. It is an artifact tuned to the specific weights and biases of a specific model version. What worked as a few-shot example in March might trigger a refusal or a hallucination in November. Upgrading a model requires refactoring the prompt. You are not updating a dependency. You are hiring a replacement.
The first line of defense is exact identifier logging. Record the actual resolved model identifier alongside the application build, prompt version, and configuration for every single request. Stating that the application called the latest alias is not enough to reconstruct an incident six months later. Route all inference requests through an internal model gateway. The gateway manages the mapping to specific vendor versions, enforces rate limits, and can mirror traffic to candidate versions.
Because prompt behavior is non-deterministic, unit tests that check for exact string matches fail. You need an evaluation pipeline. Build a golden dataset of real-world inputs and human-approved ideal outputs. When a deprecation notice arrives, run the new model against this dataset. Use shadow routing to duplicate production traffic to the new model for a week, logging the responses without serving them, to identify edge cases before cutover. An evaluation suite you cannot run against a new version in a day is an evaluation suite you do not have.
Engineering cannot solve this alone. Procurement needs to negotiate deprecation notice periods of 90 to 180 days, version stability commitments, and migration support clauses before signing a multi-year contract for an API where the core compute engine will be forcibly replaced multiple times before renewal. Inside the organization, establish a pre-approved fast path in the change process for switching between evaluated versions in the same model family. Require full board review only for switching families or providers.
Treat the prompt and the model as a single versioned artifact. When the model version increments, the prompt version must increment, because the assumptions it was built on have changed. Store both in version control and link them in your internal model registry.
Enterprise architecture spent decades building abstraction layers to isolate applications from the underlying infrastructure, ensuring that when the database or operating system changed, the application did not notice. With generative models, the abstraction layer cannot hide the change, because the dependency is the behavior itself.