HealthRadar Yardstick 2.0
A paired internal benchmark examining whether structured HealthRadar clinical context changes material clinical failures in patient-facing laboratory interpretations across three model families.
Research, evidence, and commentary
Show the work.
This library publishes HealthRadar methods, working papers, study findings, and selected supporting research. Each item is labeled so readers can distinguish current results, developmental work, and external literature.
Research and updates
Versioned publications and media.
Each item has a stable landing page, explicit status and peer-review labels, and a visible version record. Research status and limitations are stated with each publication.
HealthRadar Yardstick: How reliably do leading AI models explain laboratory results?
HealthRadar Yardstick is a paired benchmark of three leading language-model families using 15 synthetic laboratory cases, repeated generation, masked assessment, and physician adjudication. The study found descriptive differences in clinical quality, meaningful within-model variability, and a shared safety signal in one complex case.
VisitPrep early consumer evidence
Two small exploratory studies examined whether adults understood VisitPrep, perceived potential value in it, and would purchase a stand-alone $19 report. Participants generally understood the concept, but the strongest signal favored embedded or sponsored access rather than individual payment.
Evaluating patient-facing laboratory interpretations with blinded cross-model review
To develop and stress-test a blinded, evidence-preserving method for evaluating patient-facing laboratory interpretations, and to describe early signals from a small synthetic-case pilot.
Clinician concordance with a rules-based patient-facing laboratory interpretation system: a pilot review of HealthRadar with HRLens
An exploratory clinician face-validity study of HealthRadar with HRLens found high agreement for clinical reasonableness, diagnostic restraint, and overall interpretation, while identifying urgency and follow-up language as the principal refinement target.
Development of a rules-based framework for patient-facing interpretation of laboratory results
HealthRadar is a physician-informed, rules-based framework designed to move laboratory reporting beyond isolated abnormality flags. This paper describes its architecture for integrating related biomarkers, identifying clinically meaningful physiologic patterns, stratifying severity and urgency, reducing contradictory messaging, and generating bounded patient-facing explanations. The framework is intended as an interpretive communication layer rather than a diagnostic or autonomous clinical decision-making system.
Synthetic Video Publishing Demonstration
A synthetic entry demonstrating an indexable video landing page, consent-on-play external media, a written summary, transcript handling, references, and version metadata.
Synthetic Working Paper Demonstration
A nonclinical example showing how a HealthRadar working paper can be published, downloaded, versioned, superseded, and archived without changing the shared website templates.
The science behind HealthRadar
Recent evidence on laboratory-result communication, patient understanding, interpretive workflows, and rules-based clinical logic.
HealthRadar editorial summaries identify why selected publications are relevant to the questions guiding our development. The collection does not imply endorsement, collaboration, affiliation, or validation of HealthRadar.
Explore the literature collection
External work
Selected external research and commentary
HealthRadar is being developed within a broader body of research examining patient access to laboratory information, health-information comprehension, clinical reasoning, artificial intelligence, and safe communication of medical information. The following independent work has helped inform the questions HealthRadar is intended to address. Inclusion does not imply endorsement of HealthRadar by the authors or organizations listed.
Zhe He and colleagues — Understanding laboratory results
Research from He and colleagues has examined the difficulties patients encounter when interpreting laboratory results and their need for contextual explanations, educational support, and guidance about next steps. Related work has also evaluated large-language-model responses to patient questions about laboratory testing.
Bruce Bassi, MD, MS — Patient access and medical-record usability
Bassi has written about patient-portal usability and develops technology intended to give patients greater access to and control over their medical information. This work represents an adjacent approach to the broader challenge of making increasingly available health data more usable to patients.
Thomas Thesen, PhD — Constrained generative AI in medical education
Thesen and colleagues have investigated generative-AI systems constrained to curated expert materials, including retrieval-augmented approaches intended to provide individualized support while reducing hallucination risk. His group has also developed an AI Patient Actor for bounded clinical communication training.
Dennis Bergau, PhD — AI augmentation, limitations, and clinical judgment
Bergau has emphasized that machine learning and generative AI should be understood as tools whose value depends on appropriate human oversight, knowledge of their limitations, and critical evaluation of bias, variability, and unintended consequences.
Can AI reliably evaluate AI-generated health information?
The studies below examine AI peer review, agreement with clinician assessments, and the limits of automated evaluation. They provide context for questions HealthRadar Yardstick is being designed to investigate—not results from Yardstick itself.
Editorial note: These are HealthRadar editorial summaries of independent research, not the authors' abstracts. “Relevance to Yardstick” is HealthRadar's interpretation. Inclusion does not imply author or institutional endorsement, affiliation, or validation of HealthRadar. Publication status is identified for each entry.
Let LLMs Judge Each Other: Multi-Agent Peer-Reviewed Reasoning for Medical Question Answering
Authors: Zaifu Zhan, Shuang Zhou, Rui Zhang
Publication status and date: Journal of the American Medical Informatics Association (JAMIA); published online June 15, 2026. Peer reviewed. DOI: 10.1093/jamia/ocag042.
Editorial summary
Five language models generated answers and explanations, then evaluated one another's reasoning across three medical question-answering benchmarks. The system selected the most highly rated reasoning to determine its final answer. Its best configuration achieved mean accuracy of 82.0%, compared with 77.7% for the strongest individual model and up to 78.9% for majority voting. This supports peer evaluation as an answer-selection method in the tested setting. It does not establish reliable independent grading of patient-facing laboratory reports; the authors also identify possible benchmark exposure during training as a limitation.
Relevance to Yardstick
A useful precedent for reciprocal model assessment, but answer improvement and independent evaluation of already-generated reports are different research objectives.
Human evaluators vs. LLM-as-a-Judge: toward scalable evaluation of GenAI in global health
Authors: Gwydion Williams, Samuel Rutunda, Floris Nzabakira, Bilal A. Mateen
Publication status and date: npj Digital Medicine; peer-reviewed accepted article, published online July 20, 2026. Publisher version reviewed was labeled article in press.
Editorial summary
Five AI judges and six clinicians assessed responses to Rwandan health workers' questions. Under the study's prespecified equivalence criteria, the best individual judge matched clinician ratings on four of eleven dimensions; a weighted AI jury reached five. Models differed in scoring leniency, and evaluation performance declined in Kinyarwanda compared with English. These results demonstrate that agreement with clinicians depends on the model, evaluation dimension, and language. The four-of-eleven result describes criterion-level rating equivalence—not a percentage of medically correct answers.
Relevance to Yardstick
Reciprocal AI scores should be evaluated against a human-governed rubric rather than treated as an independent source of clinical truth.
Evaluating clinical AI summaries with large language models as judges
Authors: Emma Croxford and colleagues
Publication status and date: npj Digital Medicine. 2025;8:640. Peer-reviewed article published November 5, 2025.
Editorial summary
Researchers evaluated AI judges against clinician ratings of summaries assembled from electronic health records, using the PDSQI-9 clinical-summary rubric. They compared median ratings from seven model runs with those of seven human evaluators. The best single judge, o3-mini with five examples in its prompt, achieved an intraclass correlation coefficient of 0.818 (95% confidence interval, 0.772–0.854). Multi-agent discussion did not outperform that single judge. The findings support task-specific, rubric-based evaluation while showing that adding judges need not improve agreement. The coefficient measures rating agreement, not clinical accuracy as a percentage.
Relevance to Yardstick
A methodological example for validating judges and aggregating repeated ratings; its results concern clinical summaries, not laboratory-report interpretation.
LLM-as-a-Judge in Healthcare: A Scoping Analysis of Applications, Methods, and Human Alignment
Authors: Lingyao Li and colleagues
Publication status and date: arXiv preprint, version 1, posted May 24, 2026. Not a peer-reviewed journal publication in the version summarized here.
Editorial summary
This scoping analysis reports 134 eligible studies after screening 541 records from five databases, covering January 2023 through February 2026. Applications included clinical decision support, documentation, medical questions, and communication. Among studies with human validation, agreement was often moderate to strong but varied across tasks and methods. The authors highlight differences in rubrics, reference standards, and study design, alongside limited reporting of reproducibility and operational performance. The review maps a heterogeneous evidence base; it does not establish one universally reliable judging method.
Relevance to Yardstick
Supports explicitly separating generation accuracy, repeated-output consistency, and judge validity. Its conclusions remain preliminary because this version is a preprint.