Researchers at the University of Washington have created an artificial intelligence (AI) system that can solve geometry problems about as good at the average 11th-grade student. That means about 500 points (out of 800) on the SAT geometry section, or about 49 percent accuracy.

The holy grail of AI is to provide not just computational power, but the ability to “think” and perceive about as good as a human. In most cases this means natural language processing, which is a complex and context-sensitive process. Understanding language takes more work than just raw computation, it means being able to put together abstract relationships and use reason to predict outcomes. This is why researchers at the Allen Institute for Artificial Intelligence (AI2) at the University of Washington chose geometry problems for their AI project to solve.

The system, called GeOS, uses computer vision, natural language processing and a geometric solver to get these apparently mediocre scores on geometry problems. The takeaway here is not that the computer is capable of an average success when solving SAT geometry problems, but that multiple disparate systems could be brought together to understand words, formulas, and graphs at all. Any other system for solving geometry problems usually needs a human to interpret the problem into a programming language before a solution can be had. The GeOS system manages to become its own human interpreter.

“Unlike the Turing Test, standardized tests such as the SAT provide us today with a way to measure a machine’s ability to reason and to compare its abilities with that of a human,” said Oren Etzioni, CEO of AI2. “Much of what we understand from text and graphics is not explicitly stated, and requires far more knowledge than we appreciate. Creating a system to be able to successfully take these tests is challenging, and we are proud to achieve these unprecedented results.”

How GeOS manages to be its own human interpreter

The paper outlining the research by the AI2 team, “Solving Geometry Problems: Combining Text and Diagram Interpretation,” was a joint effort between between the UW Computer Science & Engineering department and AI2.

The research describes GeOS as a system designed to view and interpret SAT plane geometry problems. It does this by processing the problem’s text and interpreting the associated diagram into a logical format that can be used by the geometry problem solving software. After that, the system compares the answer from the geometry solving software to the multiple choice answers and picks the closest match.

Complications arise in that SAT questions often contain unstated assumptions. For example, diagrams may not be to scale or wording in text may be ambiguous (that may be understood easily by a human looking at the diagram).

According to researchers, GeOS currently has a 96 percent accuracy for problems the system feels confident enough to solve. The system also only knows how to solve plane geometry problems but AI2 researchers mean to develop the system to solve other geometry-related problems in the next three years.

AI2 has also released its data sets, research and software so that other researchers can use it to expand their own projects.

featured image credit: A Health Blog via photopin cc