Clinical AI Learns to Fail Like Medicine, Not Like Software

Ophthalmology residents waste time searching retinal scans for pathology and documenting biomarkers. Expert gaze and dictation patterns could be distilled into AI guidance, but would it actually speed diagnosis without sacrificing accuracy?

Method: Two expert-distilled models—a gaze-aligned Vision Transformer producing fixation-aligned areas of interest and an ontology-bounded VLM pre-filling biomarker summaries—were deployed with residents. When combined, correct diagnoses per minute increased by 40% and comment editing time fell by 67%, with no accuracy loss. The striking result: neither modality improved efficiency alone during guidance (US2), but together they produced efficiency gains substantially exceeding either modality (US3).

Caveats: Tested only on retinal OCT with ophthalmology residents. Generalization to other imaging modalities or specialties unverified.

Reflections: Why did combined guidance unlock efficiency gains that neither modality achieved independently during guidance? · Do the perceptual efficiency gains from AOI guidance persist beyond the immediate post-guidance carryover period? · Would attending physicians show similar efficiency gains, or is the benefit specific to trainees?

Frontier LLMs can generate UIs by selecting pre-built components from catalogs (A2UI), but API costs and latency make production deployment impractical. Can small fine-tuned models match frontier quality at small-model cost?

Method: Fine-tuned 4B models (Qwen 3.5) recovered approximately 98% of teacher semantic quality and approximately 97% of teacher visual quality at more than an order of magnitude lower cost than frontier API calls. Two augmented training strategies—Perturbed-catalog and Constrained-GT—Pareto-dominated the unaugmented Full-catalog baseline. Even small models handled and benefited from relatively large component catalogs.

Caveats: Tested only on React/TypeScript domains. Transfer to other UI frameworks or design systems unverified.

Reflections: What catalog size threshold causes small models to degrade in selection accuracy? · Do the augmentation strategies transfer to non-UI catalog-conditioned generation tasks? · Can even smaller models (sub-1B) achieve acceptable quality with more aggressive augmentation?

Clinical AI failures happen in production, but aggregate model monitoring and patient safety reporting don't explain how risk emerges across AI systems, clinicians, workflows, and institutional controls. Individual failures go unexamined.

Method: AI M&M provides a structured, blameless framework for case-based review, classifying each event across four linked dimensions: Trigger, Mechanism, Clinical Pathway, and Corrective Action. Two clinician reviewers independently applied all four axes to five outpatient medication and clinical decision-support cases, reaching agreement across all 20 axis-level classifications. The framework separates the condition that exposed a vulnerability from the process that produced risk.

Caveats: Demonstrated on five illustrative cases with perfect inter-rater agreement. Prospective evaluation across institutions and AI systems needed.

Reflections: What failure patterns emerge across institutions when AI M&M is deployed at scale? · How should corrective actions be prioritized when multiple AI systems share common failure mechanisms? · Can AI M&M classifications predict which AI systems pose highest institutional risk?