How do I measure accessibility maturity and compliance?

Guest blog by Alistair McNaught*

When you ask adults to recall their most memorable and enjoyable learning experiences it nearly always comes back to relationships – a fun teacher, an inspiring lecturer, a lively community. Such human interactions neatly illustrate the difference between maturity and compliance.

A “mature relationship” sounds a lot more attractive than a “compliant” one. Yet many institutions are offering compliance to disabled learners rather than mature relationships.

A group of people sitting in a lecture theatre

Maturity and compliance are different

In the UK, the Public Sector Bodies Accessibility Regulations (PSBAR) set clear legal requirements for universities and colleges - and slightly less clear legal requirements for schools.

Many colleges and universities have made significant efforts to move towards compliance. It’s a good first step but the worry is when compliance becomes the goal.

Accessibility maturity is about ensuring the things that promote good practices continue to grow. This means being embedded in cultural practice and consciousness. There are implications far beyond a time-limited steering group.

Maturity in an educational context

Among public sector bodies, education has unique challenges.

  • Engagement, challenge, and assessment – technology enhanced learning has to engage all the users with all the content. It must challenge them to persevere with content they find difficult. It must set them tasks and test the extent of their understanding. No other public sector body requires their audience to access and learn all the content provided. This has implications on the media, formats and activities teachers need to use.
  • Third party content - an online teaching module may have links to YouTube videos, journal articles, e-books, research papers, industry publications and so on. All of these have their own copyright and Intellectual Property Rights (IPR) conditions. These hinder tutors from making wholesale improvements to accessibility, even when technology exists to make such improvements.
  • Skills, resources, and training - most public sector bodies have a small team of trained people in charge of online content. Quality assurance is easy to guarantee. But a university or college may have a thousand different people with different skill sets uploading content daily.
  • Within this context, a compliance-only approach, is at best, very hard to achieve. At worst, it is counter-productive. If organisations retreat from digital diversity to rely on hardcopy printouts (where no accessibility standards apply), disabled students are massively disadvantaged.

Free webinar: How to promote digital accessibility at your institution, with University of Derby: Tuesday 13 April, 1pm BST 

Register now


Accessibility maturity is based, instead, on a holistic approach that:

  • acknowledges the generic benefits of digital content over traditional handouts/hard copy,
  • encourages diverse digital approaches that are mindful of different accessibility needs,
  • recognises that people with different disabilities (or indeed none) benefit from different resources in different ways,
  • moves beyond compliance to culture, seeking to identify and evolve “best achievable practice” that draws on good pedagogy, good policy, and good quality assurance.

Measuring maturity

The critical thing for any consideration of accessibility maturity is that it needs to have resonance and relevance for the people employing it. That’s why other “high level” maturity models have less traction in educational contexts. They don’t get to the nitty gritty of practice or influence.

The AbilityNet/McNaught maturity model originated from observations and conversations with dozens of leaders across the higher education sector. Throughout 2020, I worked with AbilityNet to:

  • build on the original model (The TechDis Accessibility Maturity Model effectively disappeared when the Jisc advisory services were disbanded at the end of 2014),
  • extend and update it to reflect developments in the last 10 years,
  • make it more granular,
  • make it more measurable.        

We’ve developed 2 versions of the model - one at institutional level and one at course/module level. These reflect the relative responsibilities and spheres of influence of different staff roles. We’ve tested the 2 models with 18 different pilot institutions.

This is the first of a series of blog posts exploring the findings of those pilots and the implications for real-world practice.

    Are you looking for accessibility training courses aimed at higher and further education professionals?


    Sign up for our courses
     


    Institutional level maturity: 8 lenses... and an overview

    In total, there are 8 lenses through which organisations are invited to measure themselves. There is some overlap between them because accessibility is pervasive, not discrete.

    The lenses we use are:

    1. Main driver - Where is energy being expended and what is measured as success?
    2. Responsibility - Who are the actors. Do they have sufficient authority?
    3. Model of disability - Is the perception "users with issues" or "systems and content with barriers"?
    4. Focus of effort - Is accessibility a "task and finish" project or a long-term quality improvement?
    5. Skills and expertise - What is the focus of training? Who gets it? Is it considered important?
    6. Digital accessibility in policies - Digital accessibility is a vital equality issue. Is it visible in policies?
    7. Culture - Is the focus on minimising risk? Or maximising user experience? Does accessibility straitjacket online learning? Or encourage innovation and experimentation?
    8. User's digital experience - How consistent is the user experience? How well designed?

    In subsequent posts will explore the findings, lens by lens. We will end this post by exploring two top-level messages from the pilot.

    Takeaway message 1- there are different routes to excellence.

    The pilot institutions included 3 organisations with very similar final scores, ranging between 51 to 57%. Yet how they achieved the same kind of score was markedly different, as shown in the graph below.

    Graph illustrating visually what is described in the text included in the body copy of the post

    Institution “a” scored well for policies, culture and student experience but had much lower scores for clarity of responsibility and skills/training. By contrast, institution “i” scored well for responsibility, culture, and student experience but poorly for policies and model of disability.

    Institution "c" had far more consistency across each lens, being let down only by skills/training.

    Using the model allows organisations to identify their strengths, weaknesses and inconsistencies. This helps focus efforts. Areas identified for improvement can be coordinated with other broad initiatives across the institution, saving time, resource, and effort.

    Takeaway message 2 – perceptions within an organisation can vary significantly

    Many institutions struggle with effective internal communication. This may result in vastly different perceptions about an organisation’s accessibility progress. Identifying differences and exploring the realities behind them is a vital part of maturity.

    It is easy to believe all areas of the organisation are as confident (or unconfident) as your own. We recommend many people are involved in the self-reflection process. Divergent views give insight into the effectiveness of communication or pervasiveness of good practice.

    The screenshot below shows the differing scores of two people working in different parts of the same pilot organisation. While they closely agree on the quality of the organisation’s culture, their scores are highly divergent for the institution’s policy framework, the effective allocation of responsibility and even what they perceive the main drivers to be. One view is more pessimistic. Finding such discrepancies is vital to unearthing information and experience that otherwise remains hidden.  

    Graph illustrating visually what is described in the text included in the body copy of the post

    What next?

    In upcoming blog posts I will share further pilot results from each of the lenses. Hopefully this will provide useful comparative data for your own organisation to learn from.

    Meanwhile I would encourage you to:

    * This is an edited version of Alistair's blog that can be found in full on LinkedIn.

    Further resources

    AbilityNet provides a range of free services to help disabled people and older people. If you can afford it, please donate to help us support older and disabled people through technology