ChatGPT, an artificial intelligence chatbot, shows promise in providing accurate and understandable information to parents of critically ill children, according to a new study.
“Our study highlights ChatGPT’s ability to effectively answer parent questions, even when dealing with medical complexity and jargon in unstructured medical notes,” authors wrote in “Using ChatGPT to Provide Patient-Specific Answers to Parental Questions in the PICU” (Hunter RB, et al. Pediatrics. Oct. 7, 2024).
Families of children in the pediatric intensive care unit (PICU) often search online for information about their child’s condition but report finding this content unreliable or irrelevant to their specific situation.
Authors of the study sought to determine whether ChatGPT, which is built on a large language model (LLM), could provide more useful information. They set up three scenarios in which a PICU patient was experiencing respiratory failure, septic shock or status epilepticus. Each was accompanied by an assessment and plan.
The team asked ChatGPT eight typical parental questions on diagnosis, pathophysiology, management and prognosis and asked for responses at a sixth-grade level.
Six PICU physicians evaluated the responses and found a high level of accuracy. On a six-point scale, only four of 144 scores were below four points, and none were below three points. Reviewers determined none of the low-scoring answers would have been harmful to the patient.
Answers also scored well on empathy, completeness and understandability, although the answers had a median grade level of 8.7, which was higher than requested. Authors acknowledged real-world patient scenarios may be more complex than their models. But they also said back-and-forth dialogue in a clinical setting could enhance responses.
Authors also appeared to be impressed with ChatGPT’s use of patient-specific information to answer the questions, something a standard website can’t do. In one case, it explained why the patient was receiving certain medications even though the plan did not explicitly detail the rationale.
“This use of information is noteworthy given that ChatGPT operates as an advanced text completion algorithm that lacks true comprehension of the queries it addresses,” authors wrote.
They said their findings lay the groundwork for future studies of LLMs.
“This feasibility study highlights ChatGPT’s potential as a tool to enhance parent education in PICUs by providing patient-specific explanations about a child’s medical condition,” they wrote, “potentially addressing limitations in existing digital resources for parents of critically ill children.”