A recent study published by the University of Cambridge’s School of Clinical Medicine shows that OpenAI’s ChatGPT-4 performed almost as well in an ophthalmology exam as eye specialists while beating lesser-qualified doctors.

According to Dr. Arun Thirunavukarasu, who led the study while at the University of Cambridge but is now with Oxford University Hospitals NHS Foundation Trust, the results reveal a potential for large language models, or LLMs, to benefit society. Although eye specialists don’t have to fear losing their jobs, GPT-4’s performance on the exam could mean shorter waiting times in the future when diagnosing people’s eye issues. This “could be revolutionary for how care is delivered,” said Thirunavukarasu.

The exam featured 87 multiple-choice questions based on a textbook for trying out trainee ophthalmologists. The questions covered areas such as eye itchiness, lesions and light sensitivity. The AI was pitted against the results of five expert ophthalmologists, three trainee ophthalmologists and two unspecialized junior doctors.

ChatGPT-4 scored an impressive 69%, above the 59% of the trainee ophthalmologists and way above the 43% scored by the nonspecialist junior doctors. It also scored higher than OpenAI’s ChatGPT-3.5, Meta Platforms Inc.’s Llama and Google LLC’s Palm2. The expert ophthalmologists scored 76%.

Importantly, the eye exam questions in the textbook are not publicly available, so there was no direct training on the data. The researchers noted that LLMs have a tendency to “hallucinate,” and they also lack the ability to understand nuance, but in terms of cutting down on waiting times, they show great potential, said the researchers.

“We could realistically deploy AI in triaging patients with eye issues to decide which cases are emergencies that need to be seen by a specialist immediately, which can be seen by a GP, and which don’t need treatment,” explained Thirunavukarasu. “With further development, large language models could also advise GPs who are struggling to get prompt advice from eye doctors. People in the U.K. are waiting longer than ever for eye care.”

Photo: Freepik