ChatGPT Passed National Medical Exam: How did you fare against students across the country?

Like every year in Peru, Medical students Those wishing to obtain a medical residency must pass a series of exams to obtain this professional degree, which in many cases is associated with certain difficulties.

Before receiving the title Residentsdoctors must first pass National Medical Examination (ENAM) and their qualifications make it easier for them to choose medical centers for the development of their Rural-Urban Marginal Health Service (SERUMS). However, not everyone gets a favorable result, so the researcher turned to the technology to see how complex it is.

Thus, Javier Flores Cohaila, professor and researcher of medical education in Southern Scientific University and the lead author of the study, consulted ChatGPT and other artificial intelligence (AI) devices if they can pass the exam while posing as Peruvian medical students. The result will surprise you.

It should be noted that 43% of Peruvian medical graduates entered SERUM they didn’t approve ENAMaccording to a study that analyzed banknotes from 2009 to 2019.

At Flores’ request, it turned out that artificial intelligence ChatGPT-4 (the most updated version) managed to pass the National Medical Examination (ENAM) with an accuracy of 86% in the 2022 test.

According to the researcher, the study is based on data ENAM 2022

, which consisted of 180 multiple choice questions. Thus, the AI ​​result was compared with a sample of 1025 medical students who took the same exam.

The research team analyzed various factors such as question type, specific knowledge about Peru, discrimination, difficulty and quality of questions, and topic to determine their impact on incorrect answers.

The results showed that ChatGPT received 156 correct answers, while other similar AI applications passed with lower scores. For example, BingAI – 148; GPT-3, 120; Bard, 120; and Claude, 118.

Javier Flores Cohaila He clarified that this study does not aim to compare the IQ of medical students with AI and does not seek to demonstrate that artificial intelligence can replace a doctor; because their work goes beyond simply answering questions.

However, he suggested that these results could stimulate universities and research groups to assist in medical education.

As we remember, December 3 ENAM 2023. Since 2006, this test has been a requirement for applying to the National Medical Residency System, so it provides a score.

In addition, this score also represents 70% of the score that is taken into account when allocating seats in Rural-Urban Peripheral Health Servicebetter known as whey.