Microsoft’s MAI-DxO Diagnoses 83% of Hard Cases in a Science Study, but Mostly in Simulation Against Generalists

Microsoft's AI diagnostic system, MAI-DxO, correctly diagnosed 83% of complex New England Journal of Medicine case records in a simulation, against 36% for generalist physicians working alone with reference materials, according to a study published in *Science* on 22 October 2026, as TechCrunch reported. The result is the strongest public claim yet from Microsoft AI, led by chief executive Mustafa Suleyman, that its software can reason through hard medical cases. It also arrives with caveats that matter as much as the headline number.

What did Microsoft test?

MAI-DxO is an orchestration layer, not a single model. It coordinates several AI agents that behave like a panel of doctors: they question a simulated patient, order tests and debate a diagnosis, according to The AI Insider. The system is built to be model-agnostic. Microsoft ran it on models from OpenAI, Google, Anthropic, Meta and DeepSeek, and the strongest results came from OpenAI's GPT-5 and GPT-5.5.

The test material comes from the Sequential Diagnosis Benchmark, which Microsoft researchers first described in a 2025 paper. That benchmark turns 304 diagnostically difficult New England Journal of Medicine clinicopathological case records into step-by-step encounters. The model or physician starts with a short case summary and must ask a "gatekeeper" model for each further finding, so cost and the order of questions count as well as the final answer. The earlier version of that work, which Microsoft Research lists as a July 2025 paper, paired MAI-DxO with OpenAI's o3 and reported 80% accuracy against about 20% for generalist physicians.

The *Science* version updates those numbers. AI Weekly, summarising the study, reports that MAI-DxO scored 83.2% on the 304 cases while generalist physicians with reference resources scored 36.1%, a gap of about 2.3 times, on text-only historical records. Its estimated cost per case ran from $2,241 to $5,431 for the AI system, against $3,049 to $4,725 for physician workups.

How much of the gain comes from the orchestrator?

The study's own data suggests the answer is: less than the headline implies. TechCrunch noted that much of the improvement appears to come from more powerful underlying models rather than from the orchestration layer.

The figures reported by TechCrunch and The AI Insider show the pattern:

- GPT-5 on its own scored 78.9% on the complex cases at roughly $8,400 per case. Adding MAI-DxO cut about $3,000 from that cost but raised accuracy by only about four points. - GPT-5.5, released in April 2026, rose from 72% to nearly 87% accuracy with MAI-DxO at a slightly lower cost. - Weaker models gained most. Gemini 2.5 Flash climbed from 46.4% to nearly 69%, though the gap between weak and strong models remained larger than the boost from the orchestrator. - On emergency department cases, neither GPT-5 nor GPT-5.5 improved meaningfully with MAI-DxO. GPT-5.5 scored about 85% with and without it, according to TechCrunch.

Microsoft's researchers noted that gains shrink as models approach the theoretical maximum. For buyers of clinical software, that points to a question about where the value sits. If frontier models improve quickly, the orchestration layer's advantage over a plain model narrows, and the commercial case may rest on workflow, integration and liability rather than on diagnostic accuracy alone.

What are the limits of the comparison?

The comparison pits the system against generalists, not the people who normally handle such cases. Specialists and multidisciplinary physician teams, the standard of care in hospitals for hard diagnoses, were not part of it. The cases were text-only historical records, not live patients, and the physicians worked alone. AI Weekly's summary also relays criticism from outside commentators Cornford and Cross, who noted that the study reports no false-alarm data on common conditions and that all authors are Microsoft-affiliated. They also said the cost estimates cannot establish real hospital savings without workflow data.

Those points are consistent with the 2025 paper, which describes a benchmark of unusually hard cases. Cases of that kind are valuable for measuring reasoning, but they are not what most patients present with. MAI-DxO remains a research system, not an approved clinical product.

What happens next?

Microsoft plans a real-world test. According to TechCrunch, it will evaluate MAI-DxO with Beth Israel Deaconess Medical Center in Boston. Patients there receive standard care, and consenting patients also interact with the system, so researchers can compare its recommendations with those of treating doctors. Dominic King, Microsoft AI's vice president of health, told the outlet the central challenge is a widening gap between demand for medical expertise and its supply, and The AI Insider reports that he expects the Beth Israel work to be the largest study of its kind.

The competitive context is crowded. TechCrunch notes that Google has published a separate study with Beth Israel in *The Lancet* testing its AMIE diagnostic AI in urgent care visits, and that Microsoft joins Google and OpenAI in claiming AI can match or beat doctors on some tasks. Microsoft already sells dictation and radiology tools to clinicians and presents MAI-DxO as an extension of that business, aiming to lead in clinical decision support.

Suleyman framed the result as a step toward what he calls "medical superintelligence". TechCrunch reports that he said Microsoft is working with the Mayo Clinic on a healthcare foundation model planned for release in 2027, that he expects a breakthrough in predicting from medical records that year, and that he expects healthcare models to improve by an order of magnitude within 24 to 36 months. Those are forecasts by an executive of the company selling the technology, not findings.

What we don't know yet

Several material points are unconfirmed. This article draws on press coverage of the *Science* paper and on Microsoft's earlier 2025 paper; it does not rest on a direct reading of the journal article. Outlets also differ on details such as which model versions were used, and the comparison with emergency department physicians is reported differently across summaries. Whether results on archived case records carry over to live, messy clinical encounters is the open question the Beth Israel study is meant to answer. Until then, the defensible reading is narrow: in a simulation built from hard published cases, a coordinated panel of AI agents running on the strongest available models beat unaided generalists, and most of the improvement tracks the quality of the underlying model.