The model had passed architecture review. The copy in production had not. The lead engineer had presented a new internal RAG application, noting they avoided vendor lock-in and API usage limits by selecting a popular open-weight model. The security architect asked what the patch service level agreement would be if a data-poisoning vulnerability was discovered in the training set. The engineer forwarded a model card and a link to a GitHub issue tracker. Security has no field in the risk register for “community responds when someone gets around to it.”
We have spent two decades and billions of dollars building muscle for software supply chains. When a zero-day drops in a standard logging library, the scanner flags the CVE, the ticket routes to the owning team, the version number bumps in the package manager, and the pipeline runs. We have none of that muscle for open-weight AI models.
Downloading a 70B parameter model takes an afternoon. Figuring out who owns the liability for what it generates takes six months. Enterprise engineering teams are pulling weights from public repositories at the same speed they pull JavaScript libraries, assuming the operational risk is identical. It is not.
“Open weights” is a distribution decision, not a governance model. It implies a download link. “Open source” implies a community with commit rights, an issue tracker, and a predictable patching cycle. Open-source software provides the human-readable code required to understand, modify, and recompile the program. Open-weight models provide the compiled output. They are massive matrices of floating-point numbers released by a research lab. You cannot submit a pull request to a matrix.
When a model leaks PII or outputs copyrighted code, there is no hotfix. You cannot patch a specific floating-point number in a neural network. Safety fixes arrive as entirely new weights. A new checkpoint is a proposed fix, not a drop-in patch. It requires a full re-deployment and re-qualification. The remediation paths are to retrain, fine-tune, build prompt guards, or swap the model entirely. None of them involve applying a patch and redeploying the same artifact.
This shifts the maintenance burden entirely. If there is no upstream maintainer, your engineering team is the upstream maintainer. The problem is that nobody told the teams downloading the models that they just signed up for that job.
The governance gap is not technical. It is that the artifact is cheap to acquire and expensive to own, and the acquisition bypasses the processes that would normally surface the ownership cost.
Procurement and legal teams are accustomed to standardizing consumption around MIT, Apache 2.0, and GPL licenses. Open-weight models have discarded this standardization. Every major release comes with a bespoke license and a highly specific Acceptable Use Policy. Legal teams are trying to map behavioral restrictions onto traditional software procurement frameworks that only understand copyright and patent indemnification.
In one common scenario, a team merges a fine-tune into a customer-facing summarizer. The base model’s Acceptable Use Policy explicitly prohibits that specific use case. Nobody read past the license name, and there is no vendor account manager to call for clarification. The policy is a constraint on the enterprise, not a commitment to it.
The disconnect extends to the artifact itself. A procurement reviewer might have the model publisher’s license in the contract file. Meanwhile, the deployment team downloaded a quantized version from a third-party repository because it fit the available GPUs. The legal review covered the publisher’s release, not the artifact running in production. A cryptographic hash can prove which file you received. It cannot tell you how that file was trained, or if the quantization process stripped the original metadata.
Model names are not artifact identifiers. Updates from the upstream lab are often silent. The repository keeps the same name but serves new weights with different behavior. There is no semantic versioning, no changelog, and no diff. If a team does not pin the deployment by hash, they do not actually know what they are running. When the base model shifts under them, they do not get a deprecation notice. They just get a different model with the same name.
Software supply chain tooling relies on Software Bills of Materials to track nested dependencies. Models require a Model Bill of Materials detailing the base model, the fine-tuning datasets, the alignment techniques, and the hardware environment. This metadata is routinely stripped, lost, or intentionally withheld by the releasing labs. Enterprise scanning tools ingest the weight file and see an opaque blob. You cannot attest to what you cannot list.
A fine-tune is a derivative work with its own provenance chain. It is effectively a fork. Most teams do not treat them that way. If an enterprise fine-tunes a permissively licensed model using a dataset from a public repository that carries a non-commercial, share-alike license, the resulting weights are legally contaminated. There is no tooling in the standard pipeline to track data provenance through the fine-tuning process.
The research labs publishing these models are incentivized by leaderboard cycles. They release weights to claim the top spot on a public benchmark. Once they achieve state-of-the-art status and publish the accompanying paper, their incentive to maintain the previous artifact drops to zero. The enterprise is building production systems on top of academic exhaust.
An issue tracker is useful. It is not a maintenance commitment. A publisher can respond promptly to a vulnerability report without being able to guarantee a corrected checkpoint. They may not have the training data, the pipeline, or the authority to retrain it. An internal team that fine-tuned the model may be the only group able to fix the deployed variant, and they may still depend on a new base release from a lab that has already moved on.
To close the gap, enterprises need to treat the model artifact like a vendor. That requires a named owner, documented provenance, a review of both the license and the Acceptable Use Policy, strict version pinning, an evaluation baseline, a regular review cadence, and an exit plan.
If an enterprise chooses to host open-weight models to avoid vendor lock-in and API costs, the budget saved must be reallocated to internal engineering. The operational burden of model alignment, output filtering, and legal compliance shifts entirely from the vendor to the internal platform team. At some point, the model in production stops being something you adopted and starts being something you maintain.