On Wednesday in an exam room in Chengdu, Sichuan province, a tall dark pile of servers known as AI-MATHS, took two versions of a mathematics entrance exam for university.
The AI machine scored 105 out of 150, achieving so in 22 minutes. Liberal arts students from Beijing did the same test last year and scored an average of 109 having two hours to complete the test. The AI also tried another version of the test and scored 100 in 10 minutes.
This test of the AI’s abilities was not just the mathematical equations of the test but also of its ability to understand the language of the questions, to make it engage and reason with numbers in the same way humans do. This is where it really struggled. As AI is usually good at crunching hardcore numbers and similar challenges, as only last month we were shown the power of AlphaGo in defeating the best human player in the world playing an ancient Chinese board game.
In this instance though, the AI-MATHS created by, Zhunxingyunxue Technology from Tsinghua University, really struggled with language. For instance it could not get its head hertz(?) around the meaning or difference between “students” and “teachers”, and thus lost some points.
The builders of AI-MATHS are hoping that it will get a higher mark next year, and expand its ability to see, understand and engage with numbers and such problems in the same way as humans do.
The machine did not however do any of the other tests in Science comprehension or a foreign language that human students have to pass, thus it did not earn a place at the university.