Medical superintelligence: Microsoft AI system diagnoses complex cases at high rate

Microsoft is pushing further into the healthcare arena with the release of a study detailing the performance of its latest diagnostic tool. Published on Thursday in the journal Science, the research indicates that Microsoft's AI system identified complex medical conditions in a simulated setting with significantly higher accuracy than human general practitioners. The system, known as MAI-DxO, is designed to coordinate multiple AI agents to simulate a collaborative medical panel, questioning patients and ordering tests to arrive at a diagnosis.

The study compared the AI system against generalist physicians, including primary care doctors and general internists working individually with reference materials. While the AI system achieved an 83% accuracy rate on complex cases derived from the New England Journal of Medicine, the human physicians answered correctly 36% of the time. The research notes that these specific cases are typically managed by specialists or physician teams rather than generalist doctors.

Mustafa Suleyman, the CEO of Microsoft AI, characterized the findings as a significant step toward what he calls 'medical superintelligence.' Dominic King, vice president of Health at Microsoft AI, emphasized the urgency of such tools, stating, 'The biggest challenge in healthcare delivery at the moment is the growing mismatch between demand and supply of medical expertise.' The company’s publication follows similar claims from Google and OpenAI regarding the capabilities of their own artificial intelligence technologies.

The data suggests that while Microsoft's orchestration system contributes to higher scores, the core capability relies on the underlying language models. Microsoft tested the diagnostic tool using models from various developers, including Anthropic, Meta, and DeepSeek, but the highest performance figures were achieved using OpenAI's GPT-5 and GPT-5.5. The study also evaluated the financial cost associated with running these complex cases, noting that using GPT-5 with a basic prompt cost approximately $8,400 per case with an accuracy of 78.9%.

With the assistance of the MAI-DxO system, costs were reduced by roughly $3,000 while accuracy saw a modest increase. However, the system's effectiveness varied when applied to different models. For instance, the orchestration layer boosted Gemini 2.5 Flash's score from 46.4% to nearly 69%, though Microsoft researchers stated that accuracy gains naturally diminish as models approach their theoretical maximum. Microsoft maintains that the tool helps balance the optimization of both accuracy and cost.

Looking ahead, Suleyman revealed that Microsoft is collaborating with the Mayo Clinic to train a foundation model for healthcare, with a release scheduled for next year. He expressed confidence that this initiative would enable significant advances, stating, 'I really believe that in 2027, we will be on the verge of a coding style breakthrough for medical records.' He envisions a future where systems can stream highly accurate predictions about how a medical record might unfold, providing valuable foresight for clinicians.

To test these capabilities in a live setting, Microsoft has entered a partnership with Beth Israel Deaconess Medical Center in Boston. The program will evaluate the clinical reasoning capabilities of the AI across different hospital environments. Patients will receive standard care, while consenting participants will also engage with the AI system, allowing Microsoft and its partners to compare the machine's recommendations against those of human doctors.

This initiative is part of a broader manifesto regarding the construction of 'medical superintelligence,' which Microsoft defines as the capacity to surpass the strongest contemporary human-led health systems in medical reasoning. Suleyman indicated that this focus on accelerating the introduction of such technology could serve as a differentiator for Microsoft compared to other frontier labs. 'We have to have the absolute number one state-of-the-art medical clinical decision support products over the next few years,' Suleyman said.