ChatGPT will not replace pediatricians: AI has low accuracy in pediatric diagnoses

A recent study published in JAMA Pediatrics indicates that the use of artificial intelligence chatbots such as ChatGPTunreliable for diagnosing diseases in children.

The study showed that the accuracy of diagnosing childhood diseases with this artificial intelligence system is only 17%, which is significantly low.

According to the study, this shows that the expertise of pediatricians remains indispensable and highlights the importance of their clinical knowledge. Despite this, many health professionals acknowledge that integration AI in medical care is probably inevitable.

artificial intelligence In the healthcare sector, it is rapidly developing and is used for a wide range of applications.

These include analyzing large volumes of health data to identify patterns that help prevent and treat disease, developing algorithms for more accurate diagnoses, personalizing patient care, and automating administrative tasks to improve the efficiency of health care services.

However, a recent study conducted in Cohen Children’s Medical Center V NY found that the newest version ChatGPT He is not yet ready to diagnose diseases in children. Children differ from adults in that they change a lot with age and cannot even really tell what is happening to them.

In an experiment with ChatGPTScientists used texts from 100 real cases of children’s illnesses and asked the system to try to determine what disease they had. Then two medical specialists looked at whether the AI’s responses were good, bad or so-so.

Sometimes, ChatGPT It spoke of a disease that had something to do with it, but it was incorrect because it was very general. For example, ChatGPT thought the baby had some kind of lump on his neck, but in fact he had a genetic disorder that also affects the ears and kidneys and can cause these lumps.

Of the 100 cases they tested, ChatGPT He gave the correct answer only 17 times. He did not get a complete answer in 11 cases and was completely wrong in 72. Moreover, out of 83 times he did not answer correctly, in 47 cases the disease, according to him, was in the correct part of the body, but even in this case the diagnosis was wrong .

The researchers noted that AI She didn’t understand very well what experienced doctors know. For example, this is not due to the fact that a child with autism may get scurvy because he does not eat enough vitamin C.

This can happen because sometimes people with autism don’t eat enough different foods and don’t get enough vitamins. Doctors know they need to look out for vitamin problems in children in countries where they are typically well-nourished. But chatbot He did not see this and thought that the child had another, much less common disease.

The chatbot didn’t perform quite right in this test, but the researchers said it could have been improved by being better trained on specialized medical books rather than sometimes incorrect information from the Internet.

They added that if the chatbot can use updated medical data, it will be able to make better diagnoses. They call it “regulate» system so that it works much more optimized.

“This provides researchers with the opportunity to test whether specific training and customization of medical data helps improve the diagnostic accuracy of chatbots based on large language models,” conclude the authors of the medical study.