Meet the AI avatar helping to train ASU’s next generation of medical-engineering students

At ASU’s John Schufeldt School of Medicine and Medical Engineering, students are learning how to walk patients through a difficult diagnosis, calm scared individuals and ask the hard questions — all before they meet an actual human patient.

These students now train with AI-generated avatars that talk and behave like real patients, creating a virtual space to hone their skills before entering clinical practice. Developed by ASU for use in ASU’s new Shufeldt School, the AI-enabled platform called Meletai, one of the most immersive tools of its kind in medical education today, launched this semester.  

“There's no other platform that looks as real, has such low latency, and is able to scale in the way that we envision,” said Dr. Jordan Coulston, assistant dean of clinical education for the John Shufeldt School of Medicine and Medical Engineering. 

Behind realistic patient-provider interactions, an agentic workflow called Case Creator is used to build complex patient-case scenarios with multiple agents working in sync to create a patient’s medical background, current condition and previous clinical appointments. The agentic workflow is powered by ASU’s CreateAI Platform. 

Once a patient case is created, multimodal AI systems across video and audio are used to develop realistic avatars that simulate standard patient encounters. As students interact with the avatars, the system uses AI to generate a student evaluation that scores an interaction across 25 unique areas, including how they gathered and used information, reached a diagnosis, communicated uncertainty and involved patients in the plan.

How it works

Meletai works like a telehealth visit. A student logs in and receives a brief explanation of why the AI-simulated patient is there that day before meeting an interactive avatar on screen. The student asks questions, explores the chief complaint and works to solve the clinical problem. 

The simulation can also provide diagnostic information, such as vital signs, laboratory results, imaging or other relevant clinical data. Students must decide what information to seek, interpret the findings, explain them to the patient and determine the appropriate next steps.

Beyond the standardized patient

Medical schools have long used trained actors to portray patients, giving students practice before clinical rotations. But actors are expensive to scale, hard to repeat on demand and limit how thoroughly students can be evaluated. In contrast, Meletai generates a full transcript of each encounter, which can then be run through multiple layers of assessment simultaneously, from basic communication skills for a first-year student to more advanced clinical reasoning for a third-year student. Virtual patients therefore enable a new pedagogy where repeatable, adaptable clinical encounters become a rich source of evidence about learner competency.