In my clinical practice, more than 20% of our patients’ families speak languages other than English—although most of these families speak Spanish, we also frequently have families who speak Dari, Pashto, Urdu, Nepali, Chinese, and Korean. While we have in-person and video interpreters who can convey the spoken information, I am always at a loss when it comes to providing written instructions. I admit to having used language websites, such as Google Translate, to translate my usual written instructions into the family’s preferred language. But am I doing more harm than good in doing that?
This week, Pediatrics is early releasing an article and accompanying video abstract entitled, “Performance of ChatGPT and Google Translate for Pediatric Discharge Instruction Translation,” which looks at that very question (10.1542/peds.2023-065573). Dr. Ryan Brewster, Dr. Priscilla Gonzalez, and colleagues from Harvard Medical School and Boston University used 3 ways to translate 20 discharge instructions for 16 common diagnoses into Spanish, Brazilian Portuguese, and Haitian Creole:
- Professional translation by a certified medical translator
- Google Translate
- ChatGPT
Pediatricians who were native speakers in the target language and certified as bilingual providers then rated the translations for:
- Accuracy: How much of the information in the original document is conveyed in the translation?
- Meaning: How well does the translation keep the original meaning and intent of the original document?
- Fluency: Is the grammar and readability acceptable?
- Severity: Does the translation say anything that could potentially cause harm or delay necessary care?
- Preference: Which translation do they prefer?
The results are simultaneously reassuring and dismaying:
- Google Translate and ChatGPT were just as good as a professional translation for Spanish and Portuguese. The evaluators actually preferred the Spanish translations from Google Translate and ChatGPT to the professional translation.
- However, for Haitian Creole, professional translations were much better than Google Translate or ChatGPT translations.
- Additionally, there were some potentially clinically important translation errors with Google Translate or ChatGPT, particularly for Haitian Creole. For example, the word “take” in “take inhaled asthma medicines” was translated as “levar,” which means to physically move or transport.
For more common languages, it sounds as though language websites do a good job of accurately translating medical information. However, we need to be careful when using these for languages that are not as commonly heard in the US.