Dictating with speech challenges: A tester's experience with Google’s Project Relate speech app

Guest blogger: Colin Hughes

Colin Hughes discusses the Android beta research app, Project Relate, that aims to help people with non-standard speech communicate more easily with others and with the Google Assistant.

As someone living with muscular dystrophy, my speech isn't exactly what you'd call "standard." I struggle to speak in long sentences and get a little slurred when I'm tired. And don't even get me started on the ventilator I need to use parts of each day and the nasal mask that comes with it - they both take a toll on my speech.

All of this makes using voice dictation apps like Apple and Android's on-device dictation, and Dragon and Apple’s Voice Control frustrating.

I rely on these apps because typing is almost impossible for me, but the speech recognition they offer is very unforgiving, and is just not accurate enough. I end up constantly correcting errors in emails, documents, and WhatsApp messages, which is a major productivity killer.

Dictation accuracy factors

Admittedly, there are a lot of factors that go into dictation accuracy, like environment acoustics, microphone quality, the app you're using, and even how tired you are.

However, Google aside, tech companies generally don't seem to be putting enough effort into improving speech recognition accuracy. If you've ever had trouble dictating a message with Siri or Bixby on your phone, or getting an Amazon Echo or Apple HomePod to understand you, you know what I'm talking about.

I have what could be described as a standard British accent, but my slower speech delivery and respiratory issues can make me a little difficult to understand sometimes by some voice dictation apps. 

So, it is against this background, that I became intrigued when a friend recently sent me a link to a BBC article about Google's Project Relate app, which opened to UK users in December.

What is Project Relate?

Project Relate is all about bridging the communication gap between people and technology. It's aimed at making it easier for anyone, regardless of age or ability, to access and use digital technologies by making them more personalised and natural to use. 

The accompanying Project Relate app is a mobile app that uses machine learning to understand your speech better - you just have to record 500 phrases for the app to learn from.

Google is looking for English-speaking testers in Australia, Canada, India, New Zealand, UK, and the US to try out the app and give feedback. 

As soon as heard about it I was all over it, I even bought a Google Pixel 7 Pro (I'm an iPhone guy, so this was a big deal!) just to test the app since it's only available on Android phones.

Getting started with Project Relate

To use the Project Relate app, you need an Android phone running Android OS 8 or later. I filled out an interest form on the Project Relate website, and within a few days, I received a link to download the app from the Google Play Store.

You can express interest in using the Project Relate app by filling out Google’s interest form: 

Use the Project Relate app

Four key features of the Project Relate app

1. The Listen function transcribes your speech to text in real-time, allowing you to copy-paste text into other apps or let others read what you want to tell them.
2. The Repeat function allows you to restate what you have said using a clear, synthesised voice. This feature is incredibly helpful in face-to-face conversations or when giving voice commands to home assistant devices.
3. The Assistant function allows you to speak directly to Google Assistant from within the app, making it easy to complete tasks such as turning on lights or playing music.
4. The feature I use the most is the Project Relate keyboard. This has proved to be incredibly powerful and has significantly improved my dictation in Gmail, WhatsApp, Facebook Messenger, and indeed anywhere you can dictate text on an Android device like my Pixel phone.

How Google understands my speech

I've been using Project Relate for the past few weeks and I'm really impressed.

The app was easy to use and only took a couple of hours to set up by recording my first 500 phrases. After processing, I got an alert that Google now understands my speech better and I was good to go. The app even copes with my slower speech delivery and softer voice, not to mention how my speech changes when I use my ventilator.

There are up to 5640 pre-set words and phrases for you to record but you don't need to record them all at once. I have recorded almost 3000 now and dip in and out of the app when I have time - the screenshot shown here is taken from inside my Project Relate app showing my progress in recording cards.

Google has told me the app should be able to improve its recognition of your speech if you record more than the first 500 phrases. You receive a new notification every time a new speech model has been created.

Adding custom words to the app

In addition to a 5640 pre-set list of words and phrases, you can also create Custom Cards to record phrases in your own choice of words and you can create and record as many Custom Cards as you’d like.

A significant drawback with other voice dictation apps like Apple Voice Control is their failure to understand custom names and places.

The Project Relate app lets you record custom names and places that are important to you. This is incredibly useful and means if you have friends who have foreign names you can record their names and when you come to dictate them the Project Relate app has no problem understanding you. For me, this represents a huge leap forward in day-to-day dictation accuracy.

Improvements needed 

As the Project Relate app is a beta research app it is a bit rough around the edges. Hopefully, Google will work on updates before a final release. 

Things I would like to see in future app updates are a “new paragraph” voice command for dictating longer text, a “send” command for sending emails and messages by voice, and the ability to dictate emoji. 

Conclusion: the keyboard is a game changer

The Project Relate keyboard has been the major game changer for me and has helped me produce the fastest and most accurate dictation I have ever been able to produce on a mobile phone. It is, quite simply, breathtakingly accurate!

Google is hoping that the Project Relate app, and feedback from testers, will help build a future where individuals with disabilities can more easily communicate and be understood. I just hope other tech companies like Apple and Microsoft make similar strides in the quality of their dictation apps.

Whilst the Project Relate app is still a research beta app it is already changing my choice of phone to use every day and I am in no hurry to give up on my Google Pixel phone and return to my iPhone any time soon.

Further resources:

• Blog: How Project Relate can help those with atypical speech patterns
• Blog: How Alexa can change the life of a disabled person
• Find out about assistive technology in our 'How to use assistive technology at work, in education and at home' training
• Learn more about assistive technology
• Find out about hearing assistive technology from our recent webinar
• Listen to our TechShare Procast - AbilityNet's podcast