The expert eval loop: a design frameworkfor expert evaluation in healthcare AI

Abstract

This paper argues that the evaluation of healthcare and life-science AI has become a design discipline, not only an engineering one. It introduces the Expert Eval Loop, a four-stage framework (Calibration, Elicitation, Adjudication, Drift Monitoring) for designing the interfaces through which domain experts contribute the irreplaceable judgment AI evaluation requires. Production patterns illustrate how interface design directly shapes evaluation quality.