The potential of AI applications in the healthcare sector is immense, over $70B was invested into AI and AI-enabled startups in 2023. The proliferation of generative AI models like OpenAI's GPT-4, Anthropic's Claude, Meta's LLaMA-2, and most recently Google's Gemini has given rise to thousands of new AI applications, thanks to low barriers to entry and ease of implementation.
According to McKinsey, one-third of organizations in its Global Survey have already adopted AI within one or more departments, with plans to accelerate investments owing to advancements in generative AI. Although the transformational potential of AI, particularly in healthcare, is undeniable, recent meta-analyses found that only 10-20% of commercially available AI-enabled applications have been externally validated or undergone academic-level reviews.
We believe the lack of rigor did not stem from malice or ill intent, but rather from the absence of standards and a validation framework. To put it in layman's terms: when it comes to validating models, most founders and executives don't know where to start.
Objective: Testing Computer-assisted Diagnosis with AI
The goal of this report is not to establish a standard framework for validation across the healthcare sector, but rather to provide a preliminary insight into the capabilities of Google's Gemini Pro in handling semi-complex tasks in healthcare. In this instance, we are evaluating Gemini's medical domain knowledge through two tests: (1) physical and (2) dermatological conditions. We are eager to test the capabilities of Gemini Ultra; if you have a connection at Google, please contact us.
Although it is the most controversial use case, we are particularly interested in benchmarking AI's capability for clinical diagnosis. This interest is driven by the recent consensus that clinical diagnosis is the holy grail application for unlocking productivity in population health.
Testing Method
Models used
OpenAI's GPT-4-32K and Google's Gemini Pro (Gemini Ultra is unavailable for commercial access at the time of testing). Disclaimer: we have not sponsored or received any monetary compensation from either Google or OpenAI.
Prompt
Beginning in early 2022, we at Mira started experimenting with applied AI, actively incorporating generative AI into various aspects of our business. Additionally, we launched a care companion feature named Beacon. We are applying the prompt engineering techniques acquired during the development of Beacon to conduct these tests.
Design
Test 1: Physical Conditions – In this test, our objective is to evaluate both Gemini and GPT-4 using 45 clinical vignettes, mirroring a prior study conducted by Semigran HL, Linder JA, Gidengil C, and Mehrotra A titled "Evaluation of Symptom Checkers for Self-Diagnosis and Triage," originally published in 2015 and updated 2023.
Test 2: Dermatological Conditions – For this test, we assess Gemini's performance on the 20 most common skin conditions, supplemented by 20 additional novel cases sourced publicly from Dermnet. In total, 40 vignettes are utilized for this evaluation.
Credit
Shridhar Gambhir, our Product Manager and John Murcia, our engineer also took part in constructing and completing the report.
Baseline
Physical conditions
We are using the accuracy metrics provided in Mehrotra A. et al.'s 2023 report as a baseline for our evaluations. Here, accuracy is defined as the ability to "list the correct diagnosis within the top three options."
Dermatological conditions
We are using the sensitivity and accuracy measures from a 2020 meta-analysis by Abhishek De, Aarti Sarda, Sachi Gupta, and Sudip Das as baselines for our tests.
| Study Design | Research | Results |
|---|---|---|
| Deep convolutional neural network (AI) vs 13 board-certified dermatologists (MDs): 14 diagnoses malignant and benign conditions | Fujisawa et al. 2019 | 92.4% AI vs. 85.3% MDs (accuracy) |
| Deep learning algorithm to classify the clinical images of 12 skin diseases including melanoma | Han et al. 2018 | 85.1% (sensitivity) |
| 804 biopsy-proven dermoscopic images of melanoma and nevi were presented to dermatologists (MDs) and convolutional neural network (AI) | Brinker et al. 2019 | 82.3% AI vs. 67.2% MDs (sensitivity) |
Results
When analyzing both Gemini Pro and GPT4-32K in the physical condition tests, it was found that both models performed with an equivalent sensitivity of 93.33%. Both were successful at identifying the correct diagnosis in 42 out of 45 clinical vignettes. However, Gemini Pro was found to have slightly higher confidence levels, with an average of 71.40% confidence compared to GPT4's 65.24%. This suggests that while both models demonstrated similar diagnostic accuracy, Gemini Pro showed a marginal superiority in its degree of certainty when drawing conclusions.
| Overall | Total Vignettes | Correct | Sensitivity | Confidence % |
|---|---|---|---|---|
| Gemini Pro | 45 | 42 | 93.33% | 71.40% |
| GPT4-32K | 45 | 42 | 93.33% | 65.24% |
| Emergent | Total | Correct | Sensitivity | Confidence % |
|---|---|---|---|---|
| Gemini Pro | 15 | 15 | 100.00% | 70.00% |
| GPT4-32K | 15 | 15 | 100.00% | 64.33% |
| Non-emergent | Total | Correct | Sensitivity | Confidence % |
|---|---|---|---|---|
| Gemini Pro | 15 | 13 | 86.67% | 72.14% |
| GPT4-32K | 15 | 14 | 93.33% | 62.50% |
| Self-care Appropriate | Total | Correct | Sensitivity | Confidence % |
|---|---|---|---|---|
| Gemini Pro | 15 | 14 | 93.33% | 72.14% |
| GPT4-32K | 15 | 13 | 86.67% | 69.23% |
In the dermatological condition tests, there was a divergence between Gemini Pro's efficiency in recognizing novel skin cases and the top 20 common skin conditions.
For novel skin cases, Gemini Pro accurately identified the condition in 60.00% of cases, with 9 out of 20 cases correctly diagnosed. It also managed to recognize similar conditions in 15% of the cases, rendering an overall effectiveness of 60.00% and an average confidence level of 55%.
When it came to the analysis of the most common skin conditions, Gemini Pro clearly excelled, scoring an accuracy rate of 85.00% with 17 out of 20 cases correctly diagnosed, showing a solid performance in recognizing prevalent dermatological conditions.
| Novel Skin Cases | Total | Exact | Similar | Sensitivity | Confidence % |
|---|---|---|---|---|---|
| Gemini Pro | 20 | 9 | 3 | 60.00% | 55% |
| Top 20 Common Skin Conditions | Total | Correct | Sensitivity | Confidence % |
|---|---|---|---|---|
| Gemini Pro | 20 | 17 | 85.00% | 69% |
Conclusion
Compared to previous research, both Gemini Pro and GPT4-32K tested with Mira's Beacon showed excellent diagnostic capabilities for physical conditions, with an overall sensitivity of 93.33%. This is significantly higher than the 51% accuracy reported for symptom checkers in the 2015 Mehrotra A. et. al study, and marginally higher than the 85% accuracy showcased by physicians in 2016. This result outperforms the performance of ChatGPT3-3.5 at 82% and is close to ChatGPT4's previous diagnostic accuracy score of 87% reported in 2022.
For the novel dermatological condition tests, Gemini Pro, with an average diagnostic accuracy of 60.00%, somewhat lags behind the benchmark set by deep learning algorithms in Han et al's 2018 study, which recorded a sensitivity of 85.10%. This suggests that, while promising, the model's ability to recognize complex skin cases is still in nascent stages compared to existing AI technology.
However, when it came to recognizing the top 20 most common skin conditions, Gemini Pro showed an impressive accuracy level of 85.00%, nearly matching the previous AI benchmark of 85.3% set by Fujisawa et al in 2019 and significantly outperforming the earlier reported physician's sensitivity of 67.2% in Brinker et al's 2019 study.
In conclusion, AI models like Gemini Pro and GPT4-32K mark a significant milestone in healthcare, demonstrating a transformative potential in diagnosis, especially when compared to physicians' traditional diagnostic methods. However, it's also clear that further refinements are needed for these models to consistently achieve or exceed the highest benchmarks set by their AI predecessors.
Appendix
Results from Gemini Pro performance in identifying physical conditions:
| Condition | Input | Output | Correct (w/in top 3) | Confidence Level |
|---|---|---|---|---|
| Acute Liver Failure | 48-year-old wife has a history of migraine headaches. Earlier today she was confused, disoriented and over the last few hours has become increasingly drowsy. She also has mild tenderness in the right upper side of her abdomen. She had consistently been taking pain medication such as acetaminophen but recently has been taking even more for her lower back pain. | { "summary": "A 48-year-old female with a history of migraine headaches presents with confusion, disorientation, drowsiness, and right upper quadrant abdominal tenderness. She has been taking acetaminophen for pain, including for recent lower back pain.", "dx_1":{ "dx":"Acetaminophen toxicity", "probability":"70%" }, "dx_2":{ "dx":"Migraine with aura", "probability":"20%" }, "dx_3":{ "dx":"Cholecystitis", "probability":"10%" } } | Yes | 70% |
| Appendicitis | 12-year-old daughter suddenly has severe abdominal pain, nausea, vomiting, and diarrhea. She currently has a temperature of 104°F (40°C). She won't let me touch her abdomen. | { "summary": "A 12-year-old girl presents with severe abdominal pain, nausea, vomiting, diarrhea, and fever. She has guarding on abdominal examination.", "dx_1":{ "dx":"Acute gastroenteritis", "probability":"70%" }, "dx_2":{ "dx":"Appendicitis", "probability":"20%" }, "dx_3":{ "dx":"Intussusception", "probability":"5%" } } | Yes | 20% |
| Asthma exacerbation | a 27-year-old woman with a history of persistent asthma and over the past 3 days I have had progressive worsening of shortness of breath, wheezing, and cough. I did not receive significant relief from my rescue inhaler with worsening symptoms, despite using it more frequently. I also have had recent exposure to a person who had a runny nose and a hacking cough. | { "summary": "A 27-year-old woman with a history of persistent asthma presents with a 3-day history of progressive worsening of shortness of breath, wheezing, and cough despite increased use of her rescue inhaler. She also reports recent exposure to a person with a runny nose and hacking cough.", "dx_1":{ "dx":"Asthma exacerbation", "probability":"90%" }, "dx_2":{ "dx":"Pneumonia", "probability":"5%" }, "dx_3":{ "dx":"Influenza", "probability":"5%" } } | Yes | 90% |
| COPD Flare | 67-year-old woman with a history of COPD and for the past 3 days I have had worsening shortness of breath and increased coughing. My cough is also productive of green sputum. I have also had a fever of 100°F (37.7°C) for the past 3 days and have been using my rescue inhaler more often. I have been consistent about taking my other medications. | { "summary": "A 67-year-old woman with a history of COPD presents with a 3-day history of worsening shortness of breath, increased coughing productive of green sputum, fever, and increased use of rescue inhaler. She has been consistent with her other medications.", "dx_1":{ "dx":"COPD exacerbation", "probability":"90%" }, "dx_2":{ "dx":"Pneumonia", "probability":"7%" }, "dx_3":{ "dx":"Heart failure", "probability":"3%" } } | Yes | 90% |
| Deep vein thrombosis | 65-year-old woman with a history of high blood pressure and have had leg pain and swelling for the past 5 days in one of my legs. I was recently hospitalized for pneumonia and had been recuperating at home. After I started walking, my right leg became red, painful, tender, and swollen. | { "summary": "A 65-year-old woman with a history of hypertension presents with right leg pain, swelling, erythema, and tenderness for 5 days following hospitalization for pneumonia.", "dx_1":{ "dx":"Deep Vein Thrombosis", "probability":"70%" }, "dx_2":{ "dx":"Cellulitis", "probability":"20%" }, "dx_3":{ "dx":"Baker's Cyst", "probability":"10%" } } | Yes | 70% |
| Heart Attack | 64-year-old male and have had chest pain for the past 24 hours. One day ago, I began to experience severe pain in the middle of my chest, as well as sweating and shortness of breath with chest tightening. The pain initially improved with Tylenol, however my symptoms have since worsened. | { "summary": "A 64-year-old male presents with a 24-hour history of worsening chest pain, sweating, and shortness of breath. The pain initially improved with Tylenol, but his symptoms have since worsened.", "dx_1":{ "dx":"Acute coronary syndrome (ACS)", "probability":"90%" }, "dx_2":{ "dx":"Pericarditis", "probability":"5%" }, "dx_3":{ "dx":"Pleurisy", "probability":"5%" } } | Yes | 90% |
| Hemolytic uremic syndrome | 4-year-old son has been having pain in his stomach for 7 days and bloody diarrhea that became bloody after the first day. Three days before all this started we visited the county fair with our family and he had eaten a hamburger. | { "summary": "A 4-year-old boy presents with a 7-day history of abdominal pain, bloody diarrhea, and a possible exposure to a foodborne pathogen at a county fair three days prior to symptom onset.", "dx_1":{ "dx":"Shigellosis", "probability":"70%"}, "dx_2":{ "dx":"Salmonella", "probability":"20%"}, "dx_3":{ "dx":"Escherichia coli (E. coli)", "probability":"10%"} } | Yes | 10% |
| Kidney Stones | 45-year-old husband suddenly started having severe pain in the lower left side of his stomach. He said the pain is also radiating into his groin. He is writhing in pain, and it is unrelieved by position. He also has nausea and is vomiting. | { "summary": "The patient is a 45-year-old male who presented with severe pain in the left lower abdomen and groin, associated with nausea and vomiting. The pain is unrelieved by changes in position.", "dx_1":{ "dx":"Renal colic", "probability":"70%" }, "dx_2":{ "dx":"Ureteral stone", "probability":"20%" }, "dx_3":{ "dx":"Diverticulitis", "probability":"10%" } } | Yes | 90% |
| Malaria | 28-year-old man and for the past 5 days I have had a fever, chills, rigors, and diarrhea and it has not been improving with acetaminophen (paracetamol). I had been traveling in Central America for 3 months, returning 8 weeks ago. I had been bitten by mosquitoes on multiple occasions, and although I initially took malaria prophylaxis, I discontinued it due to mild nausea. | { "summary": "A 28-year-old man presents with a 5-day history of fever, chills, rigors, and diarrhea. He recently returned from a 3-month trip to Central America, where he was bitten by mosquitoes. He had taken malaria prophylaxis but discontinued it due to mild nausea.", "dx_1":{ "dx":"Malaria", "probability":"70%" }, "dx_2":{ "dx":"Dengue fever", "probability":"20%" }, "dx_3":{ "dx":"Chikungunya virus", "probability":"10%" } } | Yes | 70% |
| Meningitis | 18-year-old male student and have had a severe headache and fever for 3 days. My eyes have also been really irritated by bright light and my neck is stiff. | { "summary": "An 18-year-old male student presents with a 3-day history of severe headache, fever, photophobia, and neck stiffness.", "dx_1":{ "dx":"Meningitis", "probability":"70%" }, "dx_2":{ "dx":"Encephalitis", "probability":"20%" }, "dx_3":{ "dx":"Subarachnoid hemorrhage", "probability":"10%" } } | Yes | 70% |
| Pneumonia | 65-year-old man with hypertension and degenerative joint disease and for the past 3 days I have had a productive cough and fever. My temperature is 38.3°C (101°F). | { "summary": "A 65-year-old male with history of hypertension and degenerative joint disease presents with a 3-day history of productive cough and fever. Temperature is noted to be 38.3\u00b0C (101\u00b0F).", "dx_1":{ "dx":"Pneumonia", "probability":"70%" }, "dx_2":{ "dx":"Influenza", "probability":"20%" }, "dx_3":{ "dx":"Bronchitis", "probability":"10%" } } | Yes | 70% |
| Pulmonary embolism | 65-year-old husband has suddenly started having shortness of breath for the past 30 minutes. He also has left-sided chest pain that worsens when he breathes deeply. A week ago, he underwent a total left hip replacement and, following discharge, was on bed rest for 3 days due to poorly controlled pain. I also noticed swelling in his left calf, which is tender, and he has a fever of 100.4°F (38.0°C). | { "summary": "A 65-year-old male presents with acute onset of shortness of breath, left-sided pleuritic chest pain, and fever 30 minutes ago. He also has left calf swelling and tenderness. He underwent total left hip replacement a week ago and was on bed rest for 3 days due to poorly controlled pain.", "dx_1":{ "dx":"Pulmonary embolism (PE)", "probability":"70%" }, "dx_2":{ "dx":"Deep vein thrombosis (DVT) of the left lower extremity", "probability":"60%" }, "dx_3":{ "dx":"Pneumonia", "probability":"30%" } } | Yes | 70% |
| Rocky Mountain Spotted Fever | 8-year-old son in Oklahoma has a fever, chills, joint pain, and a headache. He also has a rash that is most prominent on his wrists and ankles. | { "summary": "An 8-year-old boy in Oklahoma presents with fever, chills, joint pain, headache, and a rash on his wrists and ankles. The most likely diagnosis is Rocky Mountain spotted fever.", "dx_1":{ "dx":"Rocky Mountain spotted fever", "probability":"80%"}, "dx_2":{ "dx":"Lyme disease", "probability":"10%"}, "dx_3":{ "dx":"Scarlet fever", "probability":"5%"} } | Yes | 80% |
| Stroke | 70-year-old father has nausea and is vomiting. The right side of his body is beginning to look weak and he has increasing difficulty speaking and comprehending language. The symptoms started with only mild slurred speech before progressing over several minutes to severe difficulty speaking and right arm paralysis. | { "summary": "A 70-year-old man presents with nausea, vomiting, right-sided weakness, and difficulty speaking over the course of several minutes. This suggests a cerebrovascular accident (CVA) affecting the left hemisphere of the brain.", "dx_1":{ "dx":"Ischemic stroke", "probability":"80%" }, "dx_2":{ "dx":"Hemorrhagic stroke", "probability":"15%" }, "dx_3":{ "dx":"Transient ischemic attack (TIA)", "probability":"5%" } } | Yes | 80% |
| Tetanus | 63-year-old man and recently cut himself on his hand while gardening. He cannot open his mouth and contraction of his muscles causes painful spasms for minutes. He is also experiencing sweating and his heart rate feels very fast. He also did not complete his tetanus immunization schedule. | { "summary": "A 63-year-old male presented to the clinic today due to an inability to open the mouth, muscle spasms, and tachycardia. The patient also notes increased sweating. The patient recently cut his hand while gardening. The patient did not complete his tetanus immunization schedule.", "dx_1":{ "dx":"Tetanus infection", "probability":"80%" }, "dx_2":{ "dx":"Cellulitis of hand", "probability":"15%" }, "dx_3":{ "dx":"Myocarditis", "probability":"5%" } } | Yes | 80% |
| Acute otitis media | 18-month-old toddler has had 1 week of runny nose, cough, and congestion. She is irritable, sleeping restlessly, is not eating well, and has developed a fever. She attends day care and both my husband and I smoke. | { "summary": "The 18-month-old toddler presents with a 1-week history of upper respiratory symptoms, including runny nose, cough, and congestion, as well as irritability, restless sleep, poor appetite, and fever. She is exposed to secondhand smoke due to smoking parents and attends daycare, increasing her risk for respiratory infections.", "dx_1":{ "dx":"Acute bronchitis", "probability":"60%" }, "dx_2":{ "dx":"Pneumonia", "probability":"20%" }, "dx_3":{ "dx":"Influenza", "probability":"10%" } } | No | |
| Acute pharyngitis | 7-year-old daughter has a fever of 101°F (38.5°C), nausea, vomiting, and a sore throat. She's not coughing, does not have runny nose, or nasal congestion. She does have swollen lymph nodes and her tonsils have a coating. | { "summary": "A 7-year-old girl presents with fever, nausea, vomiting, sore throat, and swollen lymph nodes with coated tonsils. She does not have a cough, runny nose, or nasal congestion.", "dx_1":{ "dx":"Streptococcal pharyngitis", "probability":"80%" }, "dx_2":{ "dx":"Viral pharyngitis", "probability":"15%" }, "dx_3":{ "dx":"Tonsillitis", "probability":"5%" } } | Yes | 80% |
| Acute pharyngitis | 24-year-old man and have a sore throat, fever, and headache. My symptoms started 2 days ago with a sudden sore throat and fever to 102.2 degrees Fahrenheit. I have had no cough, but I do have an abscess in the back of my throat. I have otherwise been in good health and am on no medications except for ibuprofen for fever. I have no drug allergies. | { "summary": "The patient is a 24-year-old male who presents with a sore throat, fever, and headache. His symptoms began 2 days ago with a sudden onset of sore throat and fever. He denies cough but reports an abscess in the back of his throat. He has no past medical history and is not taking any medications. He has no known drug allergies.", "dx_1":{ "dx":"Streptococcal pharyngitis", "probability":"80%" }, "dx_2":{ "dx":"Influenza", "probability":"15%" }, "dx_3":{ "dx":"Peritonsillar abscess", "probability":"5%" } } | Yes | 80% |
| Acute sinusitis | 35-year-old woman and have had nasal congestion for the past 15 days. I have had facial pain and green nasal discharge for the last 12 days, but I have had no fever. I am otherwise healthy, except for mild obesity. I am on no medications, except for an over-the-counter decongestant. I also have no drug allergies. | { "summary": "This 35-year-old female presents with a 15-day history of nasal congestion, 12-day history of facial pain and green nasal discharge, and is otherwise healthy except for mild obesity. She is not on any medications.", "dx_1":{ "dx":"Acute sinusitis", "probability":"70%" }, "dx_2":{ "dx":"Allergic rhinitis", "probability":"20%" }, "dx_3":{ "dx":"Nasal polyposis", "probability":"10%" } } | Yes | 70% |
| Back pain | 5-year-old man who developed low back pain after shoveling snow 3 weeks ago. I noticed I have trouble lifting part of my left foot and have partially lost sensation in my left foot. These symptoms have been lasting for the last three weeks. | { "summary": "A 5-year-old man with low back pain, left foot weakness, and left foot sensory loss for the last three weeks.", "dx_1":{ "dx":"Lumbosacral radiculopathy", "probability":"70%"}, "dx_2":{ "dx":"Spinal cord injury", "probability":"15%"}, "dx_3":{ "dx":"Cauda equina syndrome", "probability":"10%"} } | Yes | 70% |
| Cellulitis | 45-year-old man and have had pain and redness of the skin in my lower leg. I also have a low fever and the area on my lower leg has redness, edema, and tenderness. | { "summary": "A 45-year-old man presents with pain, redness, edema, and tenderness in the lower leg, accompanied by a low-grade fever. These symptoms suggest a localized infection of the skin and underlying soft tissues.", "dx_1":{ "dx":"Cellulitis", "probability":"80%" }, "dx_2":{ "dx":"Erysipelas", "probability":"15%" }, "dx_3":{ "dx":"Deep vein thrombosis (DVT)", "probability":"5%" } } | Yes | 80% |
| COPD Flare (milder) | 56-year-old woman with a history of smoking and have had shortness of breath and a cough for several days. My symptoms began 3 days ago with a runny nose. I have had a chronic morning cough that produces white sputum, which has increased over the past 2 days. I have had similar episodes each winter for the past 4 years. I have smoked 1 to 2 packs of cigarettes per day for 40 years and continue to smoke. I don’t have chills but have not received any relief from over-the-counter cough preparations. | { "summary": "A 56-year-old woman with a 40-year history of smoking presents with a 3-day history of shortness of breath, cough, and a runny nose. Her symptoms began with a runny nose and have progressed to a chronic morning cough with increased white sputum production. The patient has had similar episodes each winter for the past 4 years. She has not experienced relief from over-the-counter cough medications.", "dx_1":{ "dx":"Pneumonia", "probability":"70%"}, "dx_2":{ "dx":"COPD exacerbation", "probability":"20%"}, "dx_3":{ "dx":"Bronchitis", "probability":"10%"} } | Yes | 20% |
| Influenza | 30-year-old woman and for the past two days I have had a fever, cough, headache, and generally feel very weak. I was in a usual state of health before suddenly feeling these symptoms. A few viral illnesses have affected me during the current winter, but not to this severity. I did have sick contacts at work though and did not receive the seasonal influenza vaccine this season. | { "summary": "A 30-year-old female presents with a 2-day history of fever, cough, headache, and weakness. She reports having been exposed to sick contacts at work and did not receive the seasonal influenza vaccine.", "dx_1":{ "dx":"Influenza", "probability":"50%" }, "dx_2":{ "dx":"Common Cold", "probability":"30%" }, "dx_3":{ "dx":"Sinusitis", "probability":"20%" } } | Yes | 50% |
| Mononucleosis | 16-year-old female and have had a fever, sore throat, and fatigue for about a week. My symptoms are gradually getting worse, and I have had difficulty swallowing. I have had a fever every day, and I could hardly get out of bed this morning. I also have enlarged lymph nodes, inflamed tonsils that are producing a clear fluid, and have a flat, red rash on my chest area and arms. | "dx_1":{ "dx":"Infectious mononucleosis", "probability":"80%" }, "dx_2":{ "dx":"Streptococcal pharyngitis", "probability":"15%" }, "dx_3":{ "dx":"Scarlet fever", "probability":"5%" } } | Yes | 80% |
| Peptic Ulcer Disease | 40-year-old man and for the past two months I have periodically had pain in my abdomen. It often radiates from my upper abdomen. The pain feels like a dull and gnawing ache, and it sometimes wakes me at night. I’ve noticed that it’s relieved by food and by drinking milk and is helped partially by ranitidine. I had a similar but milder episode about 5 years ago. | { "summary": "A 40-year-old man with a two-month history of recurrent epigastric pain, worse at night, and relieved by food and milk. He had a similar but milder episode 5 years ago.", "dx_1":{ "dx":"Peptic ulcer disease", "probability":"85%" }, "dx_2":{ "dx":"Gastroesophageal reflux disease (GERD)", "probability":"10%" }, "dx_3":{ "dx":"Gastritis", "probability":"5%" } } | Yes | 85% |
| Pneumonia | 6-year-old son with a history of mild, persistent asthma has had a cough for five days. His temperature continues to be elevated (101.6°F) despite taking Tylenol. His appetite is good, but his cough produced yellowish sputum at home. | { "summary": "A 6-year-old boy with a history of mild persistent asthma presents with a 5-day history of cough, fever, and yellowish sputum production. Despite taking Tylenol, his fever persists.", "dx_1":{ "dx":"Pneumonia", "probability":"70%" }, "dx_2":{ "dx":"Bronchitis", "probability":"20%" }, "dx_3":{ "dx":"Asthma exacerbation", "probability":"10%" } } | Yes | 80% |
| Salmonella | 14 years old and has nausea, vomiting, and diarrhea. Yesterday, he was at a picnic where he ingested undercooked chicken along with a variety of other foods. He has had regularly sized, non-bloody stools occurring 6 times a day. He also has mild abdominal cramps and a fever of about 100.1 degrees Fahrenheit. | { "summary": "The patient presented with symptoms of gastroenteritis, including nausea, vomiting, and diarrhea, following the ingestion of undercooked chicken at a picnic. The patient has had multiple loose, non-bloody stools, mild abdominal cramps, and a low-grade fever.", "dx_1":{ "dx":"Food poisoning", "probability":"90%" }, "dx_2":{ "dx":"Viral gastroenteritis", "probability":"10%" }, "dx_3":{ "dx":"Bacterial gastroenteritis", "probability":"1%" } } | Yes | 90% |
| Shingles | 77-year-old man and for five days I have had a burning and aching pain on the right side of my chest. On the same part of my chest, I also have redness and small bumps. These rashes have also begun to develop clear vesicles. I also have a headache and just a general feeling of discomfort. | { "summary": "A 77-year-old man presents with a five-day history of right-sided chest pain, redness, and small bumps that have developed clear vesicles. He also reports a headache and general malaise. Differential diagnoses include herpes zoster, contact dermatitis, and cellulitis.", "dx_1":{ "dx":"Herpes zoster", "probability":"75%" }, "dx_2":{ "dx":"Contact dermatitis", "probability":"15%" }, "dx_3":{ "dx":"Cellulitis", "probability":"10%" } } | Yes | 75% |
| Urinary tract infection | 26-year-old woman and have had painful urination, an increasingly urgent need to urinate, and more frequent urination for the past 2 days. I am sexually active, but I don't have any fever, chills, nausea, vomiting, back pain, vaginal discharge, or vaginal pruritus. | { "summary": "The patient is a 26-year-old woman who presents with symptoms of dysuria, urgency, and frequency of urination for the past 2 days. She is sexually active but denies fever, chills, nausea, vomiting, back pain, vaginal discharge, or vaginal pruritus. These symptoms are suggestive of a urinary tract infection (UTI).", "dx_1":{ "dx":"Urinary tract infection (UTI)", "probability":"80%" }, "dx_2":{ "dx":"Cystitis", "probability":"10%" }, "dx_3":{ "dx":"Urethritis", "probability":"10%" } } | Yes | 80% |
| Vertigo | 65-year-old woman and have been experiencing dizziness. It is a sudden and severe spinning sensation that often happens after rolling over in bed onto my right side. The symptoms typically last less than 30 seconds. They have occurred nightly over the last month and occasionally during the day when I tilt my head back to look upward. However, I don’t have any hearing loss, ringing in my ears, muscle weakness, or loss of sensation. | { "summary": "Based on the information provided, the most likely cause of the dizziness is benign paroxysmal positional vertigo (BPPV). This is a common condition that causes brief episodes of dizziness when the head is moved in certain positions. It is usually caused by a small piece of calcium carbonate (otoconia) that has become dislodged from the inner ear and is floating in the fluid-filled canals of the inner ear. This can cause the signals sent to the brain from the inner ear to become confused, leading to dizziness.", "dx_1":{ "dx":"Benign paroxysmal positional vertigo (BPPV)", "probability":"High" }, "dx_2":{ "dx":"Vestibular neuritis", "probability":"Low" }, "dx_3":{ "dx":"Meniere's disease", "probability":"Low" } } | Yes | 70% |
| Acute bronchitis | 34-year-old woman and have been coughing for the last 12 days. Initially, I had nasal congestion and a mild sore throat, but now my symptoms are all related to coughing. I don’t have a fever. | { "summary": "A 34-year-old woman presents with a 12-day history of cough, initially with nasal congestion and mild sore throat. She denies fever.", "dx_1":{ "dx":"Acute bronchitis", "probability":"70%" }, "dx_2":{ "dx":"Post-infectious cough", "probability":"20%" }, "dx_3":{ "dx":"Pneumonia", "probability":"10%" } } | Yes | 70% |
| Acute bronchitis | 61-year-old woman and have had a cough for 4 days that produces of yellow sputum. My symptoms started with a runny nose and a productive cough. I initially had fevers as high as 101 for 2 days, but those have now subsided. | { "summary": "A 61-year-old woman presents with a 4-day history of a productive cough and yellow sputum, preceded by a runny nose and fever (up to 101 F for 2 days).", "dx_1":{ "dx":"Acute bronchitis", "probability":"70%" }, "dx_2":{ "dx":"Pneumonia", "probability":"25%" }, "dx_3":{ "dx":"Influenza", "probability":"5%" } } | Yes | 70% |
| Acute conjunctivitis | 14-year-old son has been developing a red, irritated right eye for the past three days and today it spread from the right to the left eye. He has watery discharge from both eyes, and they are stuck shut in the morning. He has recently had congestion and a cough and several children at his day camp recently had pink eye. He doesn’t have significant pain or sensitivity to light and does not wear contact lenses. | { "summary": "A 14-year-old male presents with bilateral conjunctivitis. He has a history of recent congestion and cough, and several children at his day camp have had pink eye.", "dx_1":{ "dx":"Viral Conjunctivitis", "probability":"70%" }, "dx_2":{ "dx":"Bacterial Conjunctivitis", "probability":"20%" }, "dx_3":{ "dx":"Allergic Conjunctivitis", "probability":"10%" } } | Yes | 70% |
| Acute pharyngitis | 26-year-old man and have had a sore throat, headache, and cough for the past 2 days. I don’t have a fever and am otherwise in good health. | { "summary": "A 26-year-old man presents with a 2-day history of sore throat, headache, and cough without fever. He is otherwise in good health.", "dx_1":{ "dx":"Acute pharyngitis", "probability":"70%" }, "dx_2":{ "dx":"Common cold", "probability":"20%" }, "dx_3":{ "dx":"Influenza", "probability":"10%" } } | Yes | 70% |
| Allergic rhinitis | 22-year-old male and for the past 5-years I have had worsening nasal congestion, sneezing, and nasal itching. The symptoms are year-round but worse during the spring season. I also have had very itchy eyes, redness, and have been tearing up more often. My mouth and throat have also felt very itchy during the spring season. My mom told me I also used to have eczema in infancy. | { "summary": "The patient is a 22-year-old male with a 5-year history of nasal congestion, sneezing, and nasal itching that is worse during the spring season. He also has itchy eyes, redness, tearing, and itchy mouth and throat during the spring season. He has a history of eczema in infancy.", "dx_1":{ "dx":"Allergic rhinitis", "probability":"90%" }, "dx_2":{ "dx":"Eczema", "probability":"80%" }, "dx_3":{ "dx":"Conjunctivitis", "probability":"70%" } } | Yes | 90% |
| Back pain | 38-year-old man with no significant history of back pain but suddenly developed lower back pain when lifting boxes 2 weeks ago. I don't have any leg pain or weakness. I also don't have any fevers, chills, no weight loss, and no recent infections. Over-the-counter ibuprofen has helped somewhat, but I have taken it only twice a day for the past 3 days because I do not want to become dependent on painkillers. | { "summary": "A 38-year-old man presents with acute onset of lower back pain following heavy lifting 2 weeks ago. He denies radicular pain, fevers, chills, weight loss, or recent infections. Over-the-counter ibuprofen has provided some relief.", "dx_1":{ "dx":"Lumbar strain", "probability":"70%" }, "dx_2":{ "dx":"Lumbar herniated disc", "probability":"20%" }, "dx_3":{ "dx":"Spondylolisthesis", "probability":"10%" } } | Yes | 70% |
| Bee sting without anaphylaxis | 9-year-old son was stung by a bee at a picnic. He was crying hysterically and after 15 minutes of calming him down, it looks like he has a swollen and tender upper lip but no tongue swelling, no drooling, no trouble breathing, no rash, and no other complaints. | { "summary": "A 9-year-old boy presented with localized swelling and tenderness of the upper lip following a bee sting. There was no evidence of tongue swelling, drooling, difficulty breathing, or rash.", "dx_1":{ "dx":"Insect bite or sting, localized", "probability":"90%" }, "dx_2":{ "dx":"Allergic reaction, unspecified", "probability":"10%" } } | Yes | 90% |
| Canker sore | 17-year-old male and have had repeated mouth ulcerations for a year. My mother said she had similar mouth ulcers as a teenager, and I don’t have lesions anywhere else. I don’t smoke or drink alcohol. I also don’t do drugs or take any medication. | { "summary": "A 17-year-old male presents with a one-year history of recurrent mouth ulcerations, similar to those his mother experienced during her teenage years. He has no other lesions and does not smoke, drink alcohol, use drugs, or take any medications.", "dx_1":{ "dx":"Recurrent aphthous stomatitis (RAS)", "probability":"90%" }, "dx_2":{ "dx":"Behcet's disease", "probability":"5%" }, "dx_3":{ "dx":"Herpes simplex virus (HSV) infection", "probability":"5%" } } | Yes | 90% |
| Candidal yeast infection | 40-year-old woman and for the past 2 days I have had vaginal itching and a thick white discharge. I don't have any abdominal pain or fever. | { "summary": "Based on the provided information, you may have a vaginal yeast infection (candidiasis). It is a common fungal infection that affects the vagina and the surrounding area. It is not usually a sexually transmitted infection (STI).", "dx_1":{ "dx":"Vulvovaginal candidiasis (yeast infection)", "probability":"80%"}, "dx_2":{ "dx":"Bacterial vaginosis", "probability":"15%"}, "dx_3":{ "dx":"Trichomoniasis", "probability":"5%"} } | Yes | 80% |
| Constipation | 5-month-old son has had difficulty and delay in passing hard stools. He strains for several hours and may even miss a day before passing stool while screaming. He also has occasional spots of fresh blood on the stool or diaper. I recently weaned from him from breastfeeding to cows' milk formula, which he had been reluctant to drink initially, but he is now feeding normally. | { "summary": "A 5-month-old male presents with difficulty and delay in passing hard stools, straining for several hours, and occasional spots of fresh blood on the stool or diaper. This began after weaning from breastfeeding to cows' milk formula.", "dx_1":{ "dx":"Intussusception", "probability":"50%" }, "dx_2":{ "dx":"Anal fissure", "probability":"30%" }, "dx_3":{ "dx":"Constipation", "probability":"20%" } } | Yes | 20% |
| Eczema | 12 years old and has dry, itchy skin in front of her elbows, behind her knees, and in front of her ankles. Her cheeks also have patches of dry, scaly skin. She has symptoms of hay fever and has recently been diagnosed with an egg and milk allergy. She also has a brother with asthma and an uncle and several cousins who have been diagnosed with eczema. | { "summary": "A 12-year-old girl presents with dry, itchy skin in characteristic locations and a personal and family history of atopy. She also has a personal history of egg and milk allergy and symptoms of hay fever.", "dx_1":{ "dx":"Atopic dermatitis", "probability":"90%" }, "dx_2":{ "dx":"Contact dermatitis", "probability":"10%" }, "dx_3":{ "dx":"Nutritional deficiency", "probability":"1%" } } | Yes | 90% |
| Stye | 30-year-old man and have had a painful, swollen right eye for the past day. It is tender and there is also redness on the lower eyelid of my right eye. I have never injured my eye and there is no crusting or change in vision. I have no history of allergies or any eye conditions and haven't used any new soaps, lotions, or creams. | { "summary": "A 30-year-old man presents with a painful, swollen right eye for one day associated with right lower eyelid redness. He denies any history of trauma, ocular allergies, or recent changes in hygiene products. His vision remains unaffected.", "dx_1":{ "dx":"Hordeolum", "probability":"70%" }, "dx_2":{ "dx":"Dacryocystitis", "probability":"20%" }, "dx_3":{ "dx":"Chalazion", "probability":"10%" } } | Yes | 70% |
| Viral upper respiratory infection | 56-year-old man and for the past 6 days I have had a cough, nasal congestion, and green nasal discharge. I have also had intermittent fevers as high as 100.8 and a runny nose. | { "summary": "A 56-year-old male presents with a six day history of cough, nasal congestion, green nasal discharge, intermittent fevers as high as 100.8, and a runny nose.", "dx_1":{ "dx":"Acute sinusitis", "probability":"80%" }, "dx_2":{ "dx":"Common cold", "probability":"10%" }, "dx_3":{ "dx":"Influenza", "probability":"10%" } } | No | |
| Viral upper respiratory infection | 30-year-old man and have had a runny nose and sore throat for 2 days. I feel hot and sweaty, have a mild headache, am coughing up clear sputum and have muscle aches. I don't have a fever or neck stiffness. | { "summary": "A 30-year-old man presents with a 2-day history of runny nose, sore throat, feeling hot and sweaty, mild headache, clear sputum production, and muscle aches. He denies fever or neck stiffness.", "dx_1":{ "dx":"Common Cold", "probability":"70%" }, "dx_2":{ "dx":"Influenza", "probability":"20%" }, "dx_3":{ "dx":"Allergic Rhinitis", "probability":"10%" } } | Yes | 70% |
| Vomiting | 2-year-old son has a fever of 100.5 and vomited twice. I was worried about dehydration, so I gave him a sippy cup of apple juice, but he immediately vomited up the juice. | { "summary": "A 2-year-old boy presents with fever and vomiting. He has vomited twice and is unable to keep down fluids. He is at risk for dehydration.", "dx_1":{ "dx":"Gastroenteritis", "probability":"60%"}, "dx_2":{ "dx":"Dehydration", "probability":"40%"}, "dx_3":{ "dx":"Intussusception", "probability":"5%"} } | Yes | 60% |