The vendor demo always follows the same script. A sales engineer puts a block of COBOL on the left side of the screen, clicks a button, and a block of Java appears on the right. The syntax is correct. The brackets match. The steering committee applauds. What the screen does not show is that the generated Java code manages memory using simulated flat files, relies on a proprietary runtime library that costs half a million dollars a year to license, and implements a tax calculation rule that was legally deprecated in 2004.
The modernization team asked for a Java version of the billing program. The batch operator asked a different question: which files have to be rolled back if the job fails after midnight? Translating the calculation paragraphs does not answer whether rerunning the job will bill customers twice.
Every legacy modernization program hits the same wall. The longest phase is not writing the new system. It is the forensic archaeology required to figure out what the old system actually does. You cannot decide what to do with a system you do not understand. The documentation desert is the actual blocker. The requirements document was wrong in 2009 and nobody updated it. The executable is the spec.
This is where the AI hype cycle gets disconnected from the spreadsheet. Vendors sell automated translation because it makes for a larger statement of work. Practitioners need comprehension because that is what is actually blocking the migration. The deciding factor is the asymmetry of verification. You can check an AI-generated summary in an afternoon. You cannot check a translated 40,000-line COBOL program without essentially rebuilding the test suite you do not have. The cost of being wrong is the whole story.
Automated translation tools have existed for decades. AI is just the latest engine, and it falls into the same trap. It translates COBOL into Java with a heavy COBOL accent, preserving the logic but ignoring the architecture. It cannot refactor a batch-processing paradigm into an event-driven microservices architecture because it lacks the enterprise context to know why the batch window existed in the first place.
A model can hold a single program in its context window, but legacy systems are not self-contained. A single routine relies on copybooks, JCL scripts, mainframe database schemas, and upstream batch outputs. If the model cannot see the entire dependency tree, it will hallucinate the missing pieces when trying to write replacement code. Reading requires bounded context; replacing requires enterprise context. To replace a module, the AI needs to know your target cloud architecture, your new security model, and how you plan to handle state management across distributed services.
When teams ignore this and push for automated translation, they fall into the figure-it-out-in-testing trap. A translation project delivers 200,000 lines of generated Java. It compiles. It passes unit tests. Then finance’s control total differs by a small amount at month-end. Investigation lands on packed-decimal handling and the order in which adjustments are applied. Or the team spends six months discovering that the original COBOL had undocumented behaviors that the generated code simply drops. The project becomes a manual rewrite. Compilation proves the replacement can run. It does not prove the business can close its books.
The highest-return use of an LLM in a migration is not asking it to write a Python script. It is feeding it a bounded unit of work—the program, the copybooks, the call references, the record layouts—and asking it to explain what happens.
A senior engineer loads a massive, undocumented RPG program into a secure, locally hosted model. The original author retired a decade ago. The program handles inventory allocation, but nobody knows how it handles backorders. The engineer prompts the model to explain the conditional logic for when stock falls below zero. The model produces a sequence diagram that reveals a hardcoded workaround from 1998 tying backorders to a defunct warehouse code. That is useful. The output of reading is a decision, not a deliverable.
When you bound the context and require the model to point back to the source for every claimed rule, you get a reviewable draft. A summary with source references can be challenged. A summary without them becomes one more piece of undocumented system lore. You separate source-grounded observations from inferences. “This paragraph sets field X when condition Y holds” is a fact an engineer can verify. “This is the policy for dormant accounts” is an inference that needs confirmation from a business owner.
This approach also solves the retirement problem. The people who know the legacy systems are leaving. AI-assisted documentation is a way to capture what is in their heads before they walk out the door. A senior developer with AI-assisted summarization can review more programs, answer more questions, and unblock more decisions in the same week. Instead of budgeting 8,000 hours for manual business rule extraction, a procurement lead can feed the source into a tuned model and turn it into 1,000 hours of human verification.
The unglamorous win is test generation. Before you can replace a legacy module, you have to know its exact outputs for any given input, including the errors that downstream systems now rely on. You feed the legacy code to the model and ask it to write a test suite that feeds edge-case inputs into the running system. You capture the actual outputs. You now have a definitive baseline. You hand that test suite to the human developers building the new system. If the new system passes the AI-generated tests, the migration is safe. Nobody wants to write characterization tests for a 40-year-old batch job. The model will draft them.
Translation is a one-way door. Once you cut over to generated code, you own it, along with every hidden defect it carried over. Explanation is a thousand small doors. It is reversible, incremental, and keeps the original system running while you figure out the target state. We use AI to extract the intent and map the dependencies. We use humans to design the architecture and reconcile the totals.
The old executable is evidence of current behavior, and reading it is the only way to know what you are actually replacing.