OpenAI: GPT-6.1 Sol completes 86.6 percent of COBOL migrations in its own test

OpenAI says its latest models have become much better at writing and migrating code in older and less common programming languages such as COBOL, Fortran, OCaml and Visual Basic. The company published the results on 19 October and released one of its two tests as open source. The announcement was posted on X at 00:30 Norwegian time on Tuesday.

Every figure in this story comes from OpenAI's own tests. None has been independently verified.

Two new tests from OpenAI

OpenAI built two evaluations. The first, MultiLangBench, asks a model to build a SQLite-compatible database engine from scratch in 20 different programming languages. Its tests are adapted from SQLite's own test suite. The second measures whether a model can move existing COBOL programs to C++, Java, Kotlin, Python or Rust without changing what the programs do. A migration only counts as successful if hidden tests produce byte-for-byte identical outputs.

The company says the work follows feedback from developers and enterprise customers that coding models often struggle with these languages. OpenAI justifies the effort by citing sources saying COBOL supports more than 40 percent of online banking systems and 95 percent of ATM transactions, and that Fortran is used in weather forecasting and nuclear reactor safety analysis.

The numbers OpenAI reports

On MultiLangBench, GPT-6 Astra passes an average of 87.4 percent of tests across the 20 languages, according to OpenAI. The older GPT-5.6 Sol managed 26.8 percent. For COBOL, the pass rate rose from 10.9 to 72.1 percent, and COBOL remains the hardest language. The cheaper GPT-6 Luna reaches 40.3 percent. OpenAI estimates Luna's API cost at 0.77 dollars per sample, against 3.48 dollars for GPT-5.6 Sol, roughly 78 percent lower.

In the COBOL migration test, OpenAI says GPT-6.1 Sol reaches 86.6 percent at low reasoning effort, against 5.4 percent for GPT-5.6 Sol at the same setting. According to the company, it does so at 61 percent of the API cost. At higher effort the model reaches 97.9 percent, which OpenAI describes as close to saturating the test.

OpenAI is publishing the MultiLangBench setup and test suite on GitHub, with results per language. The company does not say whether the COBOL test will be published. It also says nothing about new prices, new licence terms or which customers have used the models for such migrations.

The caveat OpenAI makes itself

OpenAI writes that production use still requires expert review and testing against the system's business rules and dependencies. The company gives an example: COBOL discards excess decimal digits unless rounding is explicitly requested. A rewrite that rounds instead can run without errors and still produce different financial results.

Assessment: The example shows where the risk sits. The costly failure is not code that crashes, but code that runs and calculates slightly differently.

What it means for Norwegian leaders

Assessment: Nothing changes in prices or contracts today. The results may still shift the business case for modernising legacy systems, which has often been postponed because it is expensive and few people know the languages.

• Leaders running COBOL or Fortran systems should ask their IT department or supplier to test the models on their own, limited programs before relying on the figures in budgets.
• Approval of an AI-driven migration should require tests confirming that calculations, rounding and handling of missing records are unchanged, as OpenAI itself recommends.
• Source code and business rules are sent to an external supplier through the API. That should be settled in contracts and risk assessments before work starts.

Sources and media

• OpenAI: "Advancing coding in low-resource programming languages", dated 19 Oct 2026 (time not stated on the page)
• OpenAI on X, posted 20 Oct 2026 at 00:30 Norwegian time
• Sources were checked on 20 Oct 2026.
• Illustration: AI-generated with Grok Bot / hogby.ai