Tech4Good Accessibility Award 2018: Facebook and Microsoft reach final with AI creations to help blind people

The category finalists of this year’s coveted Tech4Good Accessibility Award have a strong focus on sight loss and blindness, with four of the five finalists entering technology developments which help people see and understand images and their surroundings better.

The estimated number of people living with sight loss in England is 1.7 million (Office of National Statistics, 2015). Finalists such as Microsoft are using Artificial Intelligence to provide more context of blind people’s surroundings, whereas fellow finalist Be My Eyes is using simpler technology, but harnessing the goodwill of more than a million sighted volunteers and connecting them to blind people for vision support on everyday tasks.

The fourth project in the accessibility award final which supports people with sight loss and visual impairment is GiveVision. This organisation has built a combined augmented reality and virtual reality headset to magnify a persons' view of the world around them.

More independent navigation for people with a learning disability

Finally, the fifth finalist in the accessibility category WaytoB has created a smart phone/smart watch combination which helps people with learning disabilities navigate journeys, with help from others and makes everyday journeys more simple, safe and comfortable.

This is the eighth year of the awards which are organised by tech and disability charity AbilityNet and supported by BT. Categories include BT Young Pioneer and Connected Society, Digital Volunteer of the Year Award and the People's Award. 

Robin Christopherson, head of digital inclusion at AbilityNet and one of the awards’ judges says: “We’ve had more entries of a higher quality than ever before, and have had to extend the shortlist in several categories to five finalists instead of the usual four. There was a huge amount of variety in the entries, from those making the most of the very latest technology advances like AI and VR, to others using technology very simply, but combined with a huge amount of people power, to great effect.”

More about the 2018 Tech4Good AbilityNet Accessibility Award finalists:

Be My Eyes man smiling holding smartphone camera to his eyeBe My Eyes app

Be My Eyes is a free smartphone app for blind people and those who are partially-sighted. Using their phone camera and an internet connection, the person without sight can quickly access a network of more than one million sighted volunteers who will help them see the world around them by explaining what they see on their camera link-up.

Used in more than 150 countries, with help in 180 different languages, users can ask for advice on choosing what clothes to wear, reading a bus timetable, checking use-by dates on food and much more.  Blind people and those who partially-sighted also have the option to call certain companies through the app, paid for by the companies.

Check out this Be My Eyes film by the BBC.

Facebook Automatic Alt Text technology

Facebook is using artificial intelligence and machine learning services to make images more accessible for people with low vision and vision loss. At present, more than a billion photos are shared on Facebook daily. In 2016, the company added Automatic Alt Text (AAT), a feature that uses object recognition to describe photos to people who are blind or who have low vision and use screen readers.

In December 2017, the social networking platform also launched a face recognition tool that can tell people who use screen readers which friends appear in photos in their news feed, even if they aren't tagged.

AAT can currently detect more than one hundred concepts, such as the number of people in a photo, whether people are smiling, physical objects like a car, tree, mountain and others.

A spokesperson for Facebook told the awards panel: “We strongly believe that AI is the future of improving additional interaction experiences at scale, whether they are visual in nature or otherwise. As AI systems get better at understanding images, video, audio, and other media, we believe that more novel and robust innovations in accessibility will follow.

Read more about Automatic Alt Text in this Facebook Newsroom Post.

GiveVision VR headsetsGiveVision: SightPlus headset

GiveVision is developing the next generation of low vision aids. Its first prototype, called SightPlus combines augmented reality (AR) and virtual reality (VR) headsets to help visually impaired people do a multitude of daily tasks easier or more independently. SightPlus currently works as an advanced digital magnifier and a pair of binoculars, but the team is working on a more discreet option.

With the technology users can capture more visual information by adjusting the zoom, contrast or brightness reducing the amount of assistance they would otherwise need.

According to GiveVision, the most commonly sold sight aids are handheld magnifiers, which have limited fields of view and functionalities compared to SightPlus. While the use of a smartphone with accessibility features or applications has increased, GiveVision says its research and testing shows it’s more useful and desirable for some people, particularly older generations, to have a dedicated magnifying device.

See GiveVision in action on BBC Breakfast news. 

Microsoft Seeing AI app

Seeing AI is a free app from Microsoft that narrates the visual world for the blind and low vision community. It’s an ongoing research project which was born at a hackathon event held by the company. The app harnesses the power of Artificial Intelligence to open up the visual world and describe nearby scenes, people, food in supermarkets, for example, and more, with spoken audio. Its key features include real-time text reading, document structure understanding, audio-based barcode locator and product recogniser, face recognition and emotion/age/gender description, currency recognition, colour recognition, audible light detector and handwriting reader.

Since launch in July 2017, the app has assisted users in completing over seven million tasks and has been downloaded by 200,000 users. The app can also describe inaccessible images in other apps, including Twitter, and WhatsApp, etc.

Find a short promo film about Microsoft Seeing AI here. 

WaytoB smartwatch navigation tool

WaytoB has developed an integrated smartphone and smartwatch platform to help people with a learning disability navigate their environment independently. The development is currently being trialled and tested in Ireland.

To use WaytoB, a friend, family member or carer can add safe routes for a person with a learning disability to the platform with their smartphone. The person adding the routes is then able to track the location, heart rate and battery life of the person with the learning disability, as well as get notified of key journey events (e.g getting lost, stopping for too long, showing high levels of anxiety, etc.). The person using WaytoB who has a learning disability follows icon-based instructions on their watch to better navigate their environment, with the watch vibrating when there's a new instruction.

Way2B smartwatch receives navigation instructions from smartphone

Popular smartphone apps for wayfinding are not suitable for those with learning disabilities, according to WaytoB ’s research - both from a cognitive perspective, and from a safety aspect, ie walking while trying to follow a smartphone screen presents risks from traffic, crime, and others.

In 2015, Public Health England estimated that there were more than one million people with a learning disability in England alone. Often people with a learning disability rely on others for transport and assistance to access their community, so WaytoB has the potential to open up independence.

Watch a film on how WaytoB works.

Finalists will be announced on the 17 July 2018 in London.