The hidden cost of AI bias: Why disability may be the ultimate test of trustworthy AI
Robin Christopherson | 17 Aug 2026For most of my career, accessibility has sat slightly outside mainstream technology conversations. It has certainly been recognised as important, especially in the areas of legislation, risk and compliance, but it has too often been treated as a specialist concern rather than a central measure of whether technology is working well.
Artificial intelligence may be about to change that. As a blind person who relies on technology throughout every working day, and as someone who has spent more than 25 years helping organisations create more inclusive digital services, I have watched successive waves of technology arrive with enormous promise. We begin by focusing on what the technology can do, then only later discover the assumptions hidden within it and who those assumptions leave behind.
We are seeing that happen with AI. The conversation is moving beyond model performance and productivity gains towards a more practical question: can people trust these systems enough to rely on them? Accessibility professionals have been asking versions of that question for decades, but this time the answer may determine whether organisations realise the value they hope AI will deliver.
Trust is becoming the real AI challenge
Most organisations have now moved beyond asking whether AI is useful. The more pressing question is whether it can be trusted enough to become part of everyday workflows, customer journeys and business decisions.
If staff feel the need to check every AI recommendation, summary or analysis, much of the promised efficiency disappears. If customers receive outcomes that feel inconsistent or unfair, confidence quickly starts to erode. If a system works well for most people but repeatedly struggles with particular groups, those users learn very quickly that they were not properly considered.
That is why I find the growing emphasis on trust in the responsible AI conversation so encouraging. Research from PwC increasingly links responsible AI practices with customer confidence, adoption, innovation and business value. That matters because it moves the discussion beyond ethics alone. Trust is becoming a commercial issue.
Organisations are investing heavily in AI. The question is whether employees and customers will trust it enough for those investments to deliver real returns.
AI is transforming financial services, but is it accessible?The people AI may understand least
Much of the discussion around AI fairness focuses on race, gender and ethnicity. Those conversations are essential, but disability still receives surprisingly little attention, despite disabled people representing one of the largest and most diverse groups in society.
A blind screen reader user. Someone whose speech is affected by a neurological condition. A neurodivergent customer interacting with a chatbot differently from most users. A person with dyslexia processing information in a different way. Someone living with fatigue whose needs vary from one day to the next. These are not unusual edge cases. They are the reality of human diversity.
The Microsoft Research paper Toward Fairness in AI for People with Disabilities: A Research Roadmap highlights a challenge that many accessibility professionals will recognise immediately: disabled people are frequently underrepresented in the data, testing and evaluation processes used to create AI systems. That absence matters because systems learn patterns from what they are trained on, and they can only respond well to people they have been designed and tested to understand.
If disabled people are missing from the research, training or testing, organisations should not be surprised when the resulting systems fail to understand them.
A problem organisations may already have
The risk is not theoretical. Many organisations are already deploying AI-powered customer support, search, recruitment tools, content creation systems, virtual assistants and internal productivity tools.
In our work with organisations across multiple sectors, we are increasingly seeing the same concern emerge: AI is being added to products and services much faster than it is being evaluated from an accessibility perspective. An organisation may thoroughly test a website, app or customer journey, then introduce an AI-powered feature without applying the same level of scrutiny.
A retailer launches an AI shopping assistant that is difficult to use with screen readers. A bank introduces an AI-driven support journey that struggles when customers communicate in unexpected ways. An organisation deploys AI-generated content that unintentionally creates accessibility barriers. An employer adopts AI-assisted recruitment tools without fully understanding how disabled applicants may experience them.
None of these decisions are malicious. Most are made with good intentions. The issue is that accessibility and inclusion often lag behind the technology itself.
Accessibility has taught us this lesson before
One reason I find this discussion so familiar is that accessibility has spent decades uncovering assumptions hidden within technology. Websites and applications assumed users could see the screen, use a mouse and navigate information in predictable ways. Voice technologies often assumed a narrow range of speech patterns.
AI is exposing a new generation of assumptions. Speech recognition systems may struggle with atypical speech, computer vision systems may be trained largely on standard objects and experiences, and large language models may reflect the majority while failing to understand less common perspectives.
These shortcomings are not always signs of poor engineering but rather a reflection of incomplete understanding. That is why I increasingly see disability as one of the most useful tests of AI quality.
When a system performs well for people whose needs, behaviours and experiences sit outside the assumed norms, there is a good chance it will prove more resilient, reliable and useful for everybody else too. That makes disability a practical quality test when, today, it’s still most often perceived as a niche issue for accessibility teams alone.
Why this matters when decisions affect people
Some AI failures are frustrating. Others can affect whether someone gets support, finds information, secures employment, accesses financial services or receives help from a public service. The more consequential the interaction, the more important it becomes to understand how the system behaves for people whose circumstances differ from the average.
Banking and insurance provide good examples. Many organisations in financial services are investing heavily in AI while simultaneously focusing on consumer outcomes, vulnerable customers and trust. A customer may be blind, neurodivergent, recovering from illness, managing debt or experiencing a temporary impairment. Any of those circumstances can affect how they navigate services, communicate needs or respond to automated systems.
The same principle applies well beyond financial services. Retailers using AI in customer support, employers using AI in recruitment, universities using AI in assessment and public bodies using AI to triage requests all face a similar challenge; real people do not fit neatly into the patterns a system may expect.
The AI Now Institute has argued that disability remains underrepresented in conversations about AI bias despite automated systems increasingly helping determine who receives opportunities, resources and support. For organisations, that creates a simple question: how confident are you that your AI systems work as well for diverse users as they do for ‘everybody else’?
Many organisations do not yet know the answer. Many organisations have not yet, in fact, even asked the question.
Good governance starts with real people
One of the most encouraging developments in recent years is the growing recognition that governance need not be a brake on innovation. Done well, it helps organisations move with greater confidence because they better understand how systems behave in the real world.
The AI Risk Management Framework from NIST promotes the idea of building trustworthiness into the design, development, use and evaluation of AI systems. That principle resonates strongly with accessibility.
Trustworthy AI cannot be achieved through policy documents alone. It requires testing, evidence and an understanding of how real people experience systems once they leave the lab and enter everyday life.
Organisations that involve disabled people in user research, evaluate AI-powered customer journeys, assess AI-generated content, review accessibility during procurement and test with assistive technologies are already putting themselves in a stronger position than those that do not. Those activities reduce risk, improve customer outcomes and increase confidence in the technology itself.
The lesson we should already know
Throughout the history of technology, the people who sit furthest from the assumptions built into a product have often revealed its most important weaknesses. Time and again, improving technology for those users has led to improvements that benefit everybody else.
I suspect AI will follow the same path. Disability deserves a far more prominent place in conversations about trustworthy AI, not simply because inclusion matters, although it does. Nor because organisations want to avoid regulatory, reputational or operational problems, although those concerns are real.
The deeper reason is that disability provides one of the clearest ways to assess whether an AI system genuinely understands the diversity of the people it is intended to serve. If it can do that well, trust becomes far easier to earn. Increasingly, trust looks likely to determine which organisations succeed with AI and which struggle to turn enthusiasm into lasting value.
Where does your organisation stand on AI and accessibility?
Complete our annual Digital Accessibility Survey and help benchmark progress across the UK.
TechShare Pro 2026
This year's theme for Europe's largest digital accessibility is AI and Accessibility: Risk and Reward.
How AbilityNet can help
Many organisations are asking the same practical questions. How do we evaluate AI-powered customer journeys? How do we test AI tools with more diverse users? How do we identify usability and accessibility barriers before they become customer complaints, delivery problems or governance issues?
AbilityNet is already helping organisations across financial services, technology, retail and the public sector answer those questions. Through disabled-user research, accessibility audits, AI accessibility reviews, consultancy and training, we help organisations understand how AI performs for the people who are most likely to expose its hidden assumptions.
If disability is one of the best tests of trustworthy AI, involving disabled people should be one of the first places organisations start. Our Accessible AI Integrations service can help you identify barriers, test AI-powered journeys with disabled users and build accessibility into your approach from the outset.
Talk to us about accessible AI integrations
More from Robin Christopherson on AI and accessibility
The EU AI Act has a message for accessibility teams: get involved earlier
- Published 10 Aug 2026. The EU AI Act's requirement for human oversight highlights the need to involve accessibility experts and disabled users much earlier in AI development and governance to identify risks, improve inclusion and build more trustworthy AI systems
Using AI tools to drive an accessible dev process
- Published 05 Aug 2026. How do we move beyond AI-generated code and start using AI to reduce accessibility debt before it reaches production?
AI could make financial services more inclusive. But only if accessibility is part of the infrastructure
- Published 27 July 2026. Why accessibility must be treated as core infrastructure if AI is to improve financial inclusion.
The trust gap: why AI's edge cases matter to customer loyalty
- Published 30 June 2026. Why accessibility and edge cases are critical to building trust in AI-powered services.
The delicious irony at the heart of trusting AI
- Published 17 June 2026. A personal reflection on trust, accessibility and the unexpected ways AI can both help and hinder.
Executive briefings for senior leaders
Robin offers invite-only executive briefings exploring AI, accessibility, governance, customer trust, regulation and digital inclusion.
