Icon Legend

This session is not in your schedule.

This session is in your schedule. Click again to remove it.

Presentation Icons

Audio Recorded Session

Poster Icons

Best in Basic/Clinic Science

Best in Show

Honorable Mention in Business of Medicine

Honorable Mention in Comprehensive Otolaryngology

Honorable Mention in Endocrine Surgery

Honorable Mention in Care Equity, Quality and Safety

Honorable Mention in Facial Plastic and Reconstructive Surgery

Honorable Mention in Global Perspectives in Otolaryngology

Honorable Mention in Head and Neck Surgery

Honorable Mention in Laryngology/Broncho-Esophagology

Honorable Mention in Otology/Neurotology

Honorable Mention in Pediatric Otolaryngology

Honorable Mention in Professional and Personal Development

Honorable Mention in Rhinology/Allergy

Honorable Mention in Sleep Medicine

18 Views

Otology/Neurotology

(828) De Novo Performance of AI Chatbots for Otoscopic Image Classification

Saturday, October 17, 2026

Presenting Author(s)

Steven X. Chen, MD, PhD

Resident Physician
Indiana University Department of Otolaryngology-Head and Neck Surgery
Indianapolis, Indiana, United States
Steven X. Chen, MD, PhD

Resident Physician
Indiana University Department of Otolaryngology-Head and Neck Surgery
Indianapolis, Indiana, United States

Co-Author(s)

AB

Aditya M. Bhatt, BS

Medical Student
Indiana University School of Medicine
Fishers, IN, United States
AB

Aditya M. Bhatt, BS

Medical Student
Indiana University School of Medicine
Fishers, IN, United States
MP

Matthew P. Partain, MD

Assistant Professor of Clinical Otolaryngology-Head & Neck Surgery
Indiana University School of Medicine Department of Otolaryngology - Head and Neck Surgery
Carmel, Indiana, United States
MP

Matthew P. Partain, MD

Assistant Professor of Clinical Otolaryngology-Head & Neck Surgery
Indiana University School of Medicine Department of Otolaryngology - Head and Neck Surgery
Carmel, Indiana, United States

Senior Author(s)

VR

Vijay R. Ramakrishnan, MD

Professor of Otolaryngology-Head & Neck Surgery
Indiana University Department of Otolaryngology-Head and Neck Surgery
Indianapolis, IN, United States
VR

Vijay R. Ramakrishnan, MD

Professor of Otolaryngology-Head & Neck Surgery
Indiana University Department of Otolaryngology-Head and Neck Surgery
Indianapolis, IN, United States

Introduction: Direct-to-consumer smartphone otoscopes and AI chatbots are increasingly used for at-home triage and medical questions, yet the diagnostic reliability of general-purpose multimodal large language models for otoscopic images is unclear. We compared de novo performance of Google Gemini, OpenAI ChatGPT, and Anthropic Claude on common otologic diagnoses.

Methods: A publicly available dataset of 160 de-identified otoscopic images (40 each: normal, myringosclerosis, earwax plug, chronic otitis media) was presented to each model using standardized prompts without task-specific fine-tuning. Each image was independently classified once per model; overall accuracy, diagnosis-specific sensitivity, and diagnostic tendency were assessed. Model versions included Gemini 2.5 Pro, ChatGPT GPT-4o, and Claude Sonnet 4.5.

Results: Overall accuracy was 64% for ChatGPT, 63% for Gemini, and 33% for Claude. ChatGPT and Gemini showed higher sensitivity for earwax plug (both 100%). ChatGPT performed best for myringosclerosis (43% sensitivity), whereas Gemini performed best for chronic otitis media (100% sensitivity). No significant differences were observed for normal tympanic membranes. Claude frequently labeled abnormal images as normal, while ChatGPT and Gemini more often classified images as abnormal.

Conclusions: General-purpose multimodal chatbots show moderate, diagnosis-dependent performance on otoscopic image classification without specialized training and exhibit systematic biases. Prospective validation and guardrails are needed before clinical or consumer at-home use for diagnosis or triage.