The Lock-In You Don't See: Prompts, Evals, Embeddings, and Fine-Tunes

Swapping a large language model endpoint takes five minutes. You change the URL, paste in a new API key, and restart the service. The build passes. The application deploys. Then the customer service bot starts offering unauthorized refunds and the contract analysis tool stops recognizing indemnification clauses. Architecture review boards treat model providers like internet service providers because the API is just JSON. The lock-in does not happen at the database layer. It happens in the latent space.

An API is a contract. A prompt is a negotiation. You cannot port a negotiation. Engineers build prompts by tweaking adjectives and formatting until the model behaves. They add phrases to suppress hallucinations or force structured output. These are not universal commands. They are levers that manipulate the specific weights of the model being used. Stack fifty of these iterations, and the prompt is no longer a portable instruction. It is a model-specific artifact. A system prompt might contain two thousand tokens of accumulated patches for one vendor’s quirks. Half of those fixes address problems the new model does not have. The other half miss the new model’s actual failure modes.

Then there is the vector store. Re-embedding ten million documents is not a configuration change. It is a capital expense. Embedding models map semantic meaning to high-dimensional vectors. Even when the dimensions match, the vector spaces are entirely different. The mathematical distance between a checking account and an overdraft fee changes completely when you switch models. Cosine similarity between embeddings from different providers is meaningless. You cannot mix them, and you cannot incrementally migrate them. When you change the embedding model, the entire geometry of your database shifts. Every hardcoded similarity threshold and hybrid search weight in your retrieval pipeline becomes invalid simultaneously.

Most teams build their ingestion pipeline for incremental, near-real-time updates. It has never re-embedded the full corpus. When the time comes to switch, they discover the pipeline has no batch mode and no backfill capability. The source text behind the current index is in a separate pipeline nobody has tested recently.

This is where the abstraction layer fails. Frameworks that promise provider-agnosticism normalize the network calls and payload formatting. They do not normalize behavior. The abstraction makes the switch look easy right up until you run your first evaluation. Most enterprise AI applications run on vibes. The product team tests fifty queries, agrees the answers look good, and ships. When procurement asks engineering to switch to a cheaper model, engineering refuses. They cannot mathematically prove the new model will not break production because they never built a deterministic evaluation pipeline for the old one.

Your eval suite tells you what your old model got wrong. It says nothing about what the new one will get wrong. Evals are calibrated to the current model’s specific failure modes. A new model fails differently. Without a versioned set of production-derived cases, including known failures and expected behavior, migration is a guessing game. The exit test is not whether the new model can answer the question. It is whether the organization can recognize the answers it should not give.

Fine-tuning creates the deepest form of lock-in. Weights do not transfer across providers. You cannot take a fine-tuned model and run it on a different architecture. The training data might be portable, but the training pipeline, hyperparameters, and evaluation criteria are usually provider-specific. A fine-tune becomes institutional knowledge with no documentation and no exit. The person who built it left eighteen months ago. The training examples are in a storage bucket, but the team has not kept the labeling rules or the disagreement decisions that produced them. The fine-tune works, so nobody wants to touch it, and nobody knows how to rebuild it.

The vendor does not hold your data hostage. You have built the cage yourself, through accumulated tuning that only makes sense in the context of one model. Procurement asks about data residency and service level agreements. They need to ask about model deprecation timelines, pricing change notice periods, and embedding export. The provider’s deprecation timeline is your migration timeline, and you just do not know it yet.

Multi-model strategy is easy to say in a steering committee and hard to do in a continuous integration pipeline. You do not need to run two models in production. You need the ability to move to a different one without a six-month project. The time to build the exit architecture is when everything is working. You version your prompts with model metadata, you build the batch re-indexing pipeline, and you run a quarterly portability test against a second provider. The provider will eventually deprecate the model version you are pinned to. The only question is whether you know how to leave before they send the notice.