A picture of a laptop screen showcasing one of the AI avatars

Meet the AI avatar helping to train ASU’s next generation of medical-engineering students

At ASU’s John Schufeldt School of Medicine and Medical Engineering, students are learning how to walk patients through a difficult diagnosis, calm scared individuals and ask the hard questions — all before they meet an actual human patient.

These students now train with AI-generated avatars that talk and behave like real patients, creating a virtual space to hone their skills before entering clinical practice. Developed by ASU for use in ASU’s new Shufeldt School, the AI-enabled platform called Meletai, one of the most immersive tools of its kind in medical education today, launched this semester.  

“There's no other platform that looks as real, has such low latency, and is able to scale in the way that we envision,” said Dr. Jordan Coulston, assistant dean of clinical education for the John Shufeldt School of Medicine and Medical Engineering. 

Behind realistic patient-provider interactions, an agentic workflow called Case Creator is used to build complex patient-case scenarios with multiple agents working in sync to create a patient’s medical background, current condition and previous clinical appointments. The agentic workflow is powered by ASU’s CreateAI Platform. 

Once a patient case is created, multimodal AI systems across video and audio are used to develop realistic avatars that simulate standard patient encounters. As students interact with the avatars, the system uses AI to generate a student evaluation that scores an interaction across 25 unique areas, including how they gathered and used information, reached a diagnosis, communicated uncertainty and involved patients in the plan.

How it works

Meletai works like a telehealth visit. A student logs in and receives a brief explanation of why the AI-simulated patient is there that day before meeting an interactive avatar on screen. The student asks questions, explores the chief complaint and works to solve the clinical problem. 

The simulation can also provide diagnostic information, such as vital signs, laboratory results, imaging or other relevant clinical data. Students must decide what information to seek, interpret the findings, explain them to the patient and determine the appropriate next steps.

Beyond the standardized patient

Medical schools have long used trained actors to portray patients, giving students practice before clinical rotations. But actors are expensive to scale, hard to repeat on demand and limit how thoroughly students can be evaluated. In contrast, Meletai generates a full transcript of each encounter, which can then be run through multiple layers of assessment simultaneously, from basic communication skills for a first-year student to more advanced clinical reasoning for a third-year student. Virtual patients therefore enable a new pedagogy where repeatable, adaptable clinical encounters become a rich source of evidence about learner competency.

The team behind the technology

Building something this immersive required pulling together product, engineering and design expertise in one place, work handled through ASU Enterprise Technology’s AI Accelerator team. “We partnered with the AI Accelerator team to help bring together different dimensions of technology required for the project, allowing us to take an educational idea all the way to the student-facing software that you see today,” Coulston said.

For Meletai’s creators, the technology isn’t meant to replace the human core of medicine — it’s meant to protect it, giving students the chance to make their mistakes, build their instincts and find their bedside manner long before a real patient is on the other side of the conversation.

The new medical school is part of ASU Health, an initiative that transforms access to  high-quality health care across Arizona. 

“In order to achieve that vision, it's critical that we train the next generation of physicians to leverage new technologies like medical devices, artificial intelligence and health systems design,” said Dr. Sarah “Holly” Hollingsworth Lisanby, founding dean and foundation professor of the John Shufeldt School of Medicine and Medical Engineering, where students earn both an MD and a master’s degree in medical engineering. 

“These technologies can be force multipliers, allowing our graduates to care for more patients, have a broader geographical reach and improve access to affordable, effective health care,” she continued.