Researchers at West Virginia University have found that ChatGPT-5 Pro can generate realistic educational case studies for psychiatry training, while concluding that human oversight remains essential to address patient safety and ethical considerations. The study, published in npj Digital Medicine, evaluated whether the AI model could produce clinical vignettes for medical students and psychiatry residents that reflect emerging issues related to patients’ use of AI chatbots for mental health support. The research was conducted by scientists and physicians from the West Virginia University School of Medicine and the Rockefeller Neuroscience Institute.
The researchers developed prompts asking ChatGPT-5 Pro to generate simulated patient scenarios that included chatbot interactions, psychiatric symptoms, diagnostic information, and multiple-choice educational questions with explanatory answers. The cases covered conditions including schizophrenia spectrum disorders, anxiety disorders, mood disorders, major depression, and psychosis.
According to the study, the AI-generated vignettes demonstrated strong performance in language quality, diagnostic accuracy, and educational value. Faculty reviewers also found that the scenarios contained sufficient clinical detail to support learning.
The evaluation also identified limitations related to safety and ethics. The researchers concluded that AI-generated teaching materials should be used within a human-supervised educational framework, with instructors guiding discussions and incorporating structured debriefings on diagnostic reasoning, patient risk assessment, management strategies, and the role of chatbot use in clinical care.
Study co-leader Dr. Wanhong Zheng, a professor in the Department of Behavioral Medicine and Psychiatry, said the project was intended to address a gap in psychiatry education as patients increasingly use AI tools for mental health support. He said trainees are more likely to encounter reports of AI-related mental health issues in journals, news coverage, and social media than in routine clinical practice.
Co-leader Gangqing “Michael” Hu, an associate professor in the Department of Microbiology, Immunology, and Cell Biology, said clinicians need to understand how patients’ interactions with AI systems may influence psychiatric conditions. He noted that while AI chatbots possess extensive knowledge, their conversational style may present challenges if they reinforce unusual beliefs rather than encouraging critical evaluation. Hu added that additional research is needed to better understand safe communication between AI systems and users and the potential effects of prolonged chatbot interactions.
The vignettes were assessed by Zheng and board-certified psychiatrists Dr. Dilip Chandran and Dr. Daniel Elswick across four areas: language, diagnostic accuracy, safety and ethics, and educational usefulness. Although the AI performed well in most categories, the reviewers emphasized that training materials should continue to prioritize assessment of patient safety, including evaluation of suicide and homicide risk where clinically appropriate.
The researchers said the findings suggest AI-generated cases could become a component of psychiatry education as training programs seek to improve physicians’ familiarity with AI technologies. They also noted that further research is needed before AI can be more broadly incorporated into mental health education or practice.
The team suggested future work could examine patterns in published case reports and news coverage involving AI and mental health to improve the quality of simulated training cases. They also recommended studying how factors such as age, medical history, and existing health conditions influence the relationship between chatbot use and psychiatric symptoms. Zheng said AI is becoming an increasingly common part of everyday life but should not replace mental health professionals for people experiencing psychiatric conditions.
