Skip to main content
AAO-HNSF 2026 Annual Meeting & OTO EXPO Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.

Scientific Oral Presentation

Facial Plastic and Reconstructive Surgery

Validation of a Locally Deployed Large Language Model for Procedure-level Data Extraction in Rhinoplasty

Monday, October 19, 2026
1:06 PM - 1:12 PM PT
Location: 404A

    Presenting Author(s)

  • KC

    Kevin Z. Chen

    Research Fellow
    Washington University School of Medicine
    Palatine, IL, United States

    • Co-Author(s)

  • Amrita Hari-Raj, MD, MSCI

    Resident Physician
    Washington University School of Medicine in St. Louis
    Saint Louis, Missouri, United States

  • DK

    Dorina Kallogjeri, MD

    Associate Professor
    Washington University of St. Louis
    St Louis, Missouri, United States

    • Senior Author(s)

  • Emily A. Spataro, MD

    Associate Professor
    Washington University in St. Louis
    St. Louis, Missouri, United States

Disclosure(s):
Kevin Z. Chen:

faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Emily A. Spataro, MD:

faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Introduction: Accurate extraction of operative details from free-text surgical notes is essential for outcomes research and quality improvement in rhinoplasty but is traditionally performed through manual chart review. Large language models (LLMs) may offer a scalable alternative, yet real-world performance in operative note abstraction remains incompletely characterized.

Methods: We conducted a retrospective validation study of patients undergoing rhinoplasty at a single tertiary academic center between January 2019 and June 2025 (n = 504). Surgeon-authored operative notes were extracted and processed using a locally deployed large language model (gpt-oss-20b, hosted via Ollama) to identify whether predefined rhinoplasty procedures were performed. Manual chart review by trained reviewers served as the reference standard. Model performance was evaluated using sensitivity, specificity, precision, recall, F1 score, and inter-method agreement. Error patterns were additionally reviewed qualitatively to characterize sources of model misclassification.

Results: The locally deployed LLM demonstrated high accuracy for procedure-level data extraction, with sensitivity ranging from 92-96%, specificity from 91-95%, precision from 94-98%, and F1 scores between 0.93 and 0.96 across evaluated procedures. Most errors represented systematic false positives arising from negated statements, implied procedures, or templated operative language rather than random misclassification.

Conclusions: A locally deployed large language model achieved high accuracy in extracting procedure-level information from rhinoplasty operative notes compared with manual chart review. Errors were concentrated in a limited subset of notes with negated or ambiguous phrasing, suggesting that targeted prompt refinement and limited human auditing may further improve performance. LLM-assisted abstraction represents a scalable and reproducible approach for surgical outcomes research and quality improvement.