Microsoft"s Build 2016 keynote passed by in a blur today, as the company crammed in all sorts of announcements - from the launch of its new Bot Framework and Xbox One dev kit functionality, to Bash support on Windows, new Universal apps, and the upcoming Windows 10 "Anniversary Update".
But one of the most remarkable moments of the keynote came right at the very end when as part of its new Cognitive Services framework - which includes 22 new intelligence APIs - company CEO Satya Nadella showed off "Seeing AI", an exciting research project that has the potential to change how people see the world around them, with a particular focus on those who can"t see anything at all.
Saqib Shaikh is a Microsoft developer and software engineer from London, and he has played a key role in developing the Seeing AI project, having lost his sight when just 7 years old.
"One of the things I"ve always dreamt of, since I was at university," he said in a video published by Microsoft today, "was this idea of something that could tell you at any moment what"s going on around you."
He was able to help take the first steps towards realizing that vision with the help of his colleagues. "I teamed up with like-minded Microsoft engineers," he explained, "to make an app that lets you know who and what is around you. It"s based on top of the Microsoft intelligence APIs... and the app runs on smartphones, but also on the Pivothead smart glasses."
With a swipe across his smart glasses, Seeing AI tells Saqib exactly what"s in front of him. "I think it"s a man jumping in the air, doing a trick on a skateboard," it tells him, as the scene plays out in front of him. Saqib also said that "the app can tell me the age and gender of people around me, and what their emotions are", pointing out the usefulness that this can have when engaged in conversations.
The app can also help with simple everyday tasks that many of us take for granted, such as "reading" a menu. It"s even intelligent enough to be able to deal with voice commands, such as "read me the headings", rather than simply regurgitating every piece of text across the entire menu at once.
Saqib humbly notes that his efforts and those of his colleagues with whom he worked on the project would not have been possible without the years of Microsoft research that came before them - but even so, he says, a project like Seeing AI is "just the beginning". Nonetheless, while Seeing AI may be just a small step in the much broader picture of artificial intelligence development, with further development, it has the potential to genuinely transform the lives of many people around the world.
You can view the full video below: