#What the study found
Kemal Turker Ulutaş and Adnan Pekmezci evaluated three large language models using 100 published educational cases in clinical pathology and laboratory medicine. Two blinded experts assessed open-ended responses for diagnosis, interpretation, management and safety. The investigators also developed 500 multiple-choice questions.
ChatGPT-5.2, Gemini 3 Pro and DeepSeek-V3.2 answered 96%, 95% and 94% of those questions correctly, respectively. Yet their open-ended responses contained at least one clearly unsafe or clearly guideline-discordant recommendation in 2, 4 and 5 of the 100 cases, respectively.
Those safety counts require careful interpretation. The binary flag was designed to capture overt problems. The authors explicitly caution that an unflagged response could still contain omissions, delayed escalation, unnecessary testing or inappropriate prioritization. With few flagged events, the results do not establish a reliable safety ranking among the models.
This was a text-only educational benchmark, not a study of patient outcomes. Each case was tested once per model, so repeatability was not measured. Published textbook cases may have appeared in training data; the multiple-choice questions were internally developed rather than independently validated; and consumer-interface testing limited control over model settings.
#ClearVeillance’s interpretation
The useful contribution is the separation of knowledge performance from clinically consequential advice. A strong multiple-choice score leaves unanswered whether a patient-facing explanation omits an essential warning or suggests the wrong next step.
For laboratory AI evaluation, this supports examining the specific content of each response against explicit clinical criteria, with omissions and overtly unsafe recommendations assessed separately. Repeated-run testing would address an additional question this study could not answer.
These are implications for evaluation design, not evidence that Atlas is effective or that Yardstick has been externally validated.
Source: Ulutaş KT, Pekmezci A. Laboratory Medicine Decision Support—Beyond Exam Passing: A Blinded 100-Case Text-Based Benchmark of Diagnostic Accuracy, Management Quality, and Safety for ChatGPT, Gemini, and DeepSeek—LLM Decision Support in Laboratory Medicine. Diagnostics. 2026;16(18):3030. Published September 18, 2026.