Microsoft Seeing AI - the best ever app for blind people just got even better

Microsoft’s revolutionary app Seeing AI took the blind world by storm when released earlier this year. Now it's been updated, with several new functions including handwriting and colour recognition - and it’s still free. It's launching today and I’m personally wriggling in my office chair with excitement, whilst simultaneously hitting refresh in the iOS app store.

Take a look at the features below to see why it's such an amazing step forward for disabled people.

Seeing AI just got even better

Microsoft Seeing AI app describing sceneSince it was launched in mid-2017 Seeing AI has been downloaded more than 100,000 times and has assisted users with over 3 million tasks. It was released with features such as the ability to identify a product audibly using the barcode, as well as being able to describe images, text and faces of friends and family as they come into view.

Today (14 December) Microsoft has announced new features that provide new user experiences including currency, handwriting and colour recognition, as well as light detection. It's also now available in 35 countries, including the European Union.

Seeing AI in action

As you can see from my video I've been using Seeing AI for reading magazines as well as handwritten notes left by my family.

A whole new set of features

New features in Seeing AI v2 include: 

  • Colour recognition: Getting dressed in the morning just got easier with this new feature, which describes the colour of objects, like the garments in your closet.  
  • Currency recognition: Seeing AI can now recognize the currency of US dollars, Canadian dollars, British Pounds and Euros. Checking how much change is in your pocket or leaving a cash tip at a restaurant is much easier.
  • Musical light detector: The app alerts you with a corresponding audible tone when you aim the phone’s camera at light in the environment. A newly convenient tool so you don’t have to touch hot bulbs to know that a light is switched off, or the battery pack’s LED is on.
  • Handwriting recognition: Expanding on the ability of the app to read printed text, such as on menus or signs, the newly improved ability to read handwriting means you can read personal notes in a greeting card, as well as printed stylized text not usually readable by optical character recognition.  
  • Reading documents: Seeing AI can read you the document aloud without voiceover, with synchronised word highlighting. Also includes the ability to change the text size on the Document channel.
  • Choose voices and speed: Personalization is key, and when you’re not using VoiceOver, this feature lets you choose between the voice that is used and how fast it talks.

Raise your glasses to a brighter future for the blind

As amazing as it is the development of the app is also a sign of the power that AI will bring in the future.

The video above shows a prototytpe that uses the Seeing AI engine in a pair of smart glasses. While the Google Glass smart spectacles fell a little flat, there’s no doubt that head-mounted cameras (with or without a screen) are going to play a major role in the future of wearable tech and it’s only a matter of time before this functionality is added.

For the users of Seeing AI, it makes perfect sense to have the camera that is working so hard to tell us about our surroundings mounted on our heads and looking in the direction most important to us. This is important in terms of warning us about upcoming obstacles, people we’re interacting with (or wanting to avoid physically or possibly even socially) and also in terms of street signs, shop fronts, notices and hoardings etc (all of which will soon be able to be automatically and effortlessly read out to us).

It is also possible to combine this breathtakingly useful level of awareness of our surroundings with the spoken cues of navigation apps - my personal top GPS app with added accessibility sparkles is BlindSquare. This means that people who are blind, severely dyslexic or have a learning disability will now have a whole new world of support wherever we go.

Thank you, Microsoft

I just want to end this quick post by spelling out my gratitude to Microsoft for bringing real cutting-edge machine learning to a group of users with such evident needs in this area. While none of the smarts within Seeing AI are solely or even primarily intended for blind users, it takes a company as acutely aware of the importance of accessibility as Microsoft to do such an excellent implementation that brings the best of AI to those who benefit most.

Robin Christopherson is head of digital inclusion for AbilityNet

Related links