Skip to main content
Subscribe — first month free
Legal InsightThe Access Advantage

The Problem with EMMA: Crash Test your Tech

When AI can't understand the customer, and what it means for every customer-facing business.

20 August 2026·Chris Fry·Legal Insight
A person holding a smartphone displaying a facial recognition or biometric scan, illustrating AI verification and the Equality Act 2010 reasonable adjustments duty

Photo: Pexels

AI, biometric verification and automated customer service are rapidly becoming part of ordinary business. Banks use facial recognition. Retailers require app authentication. Utilities automate customer contact. Voice recognition increasingly sits between a customer and a human adviser. There's nothing inherently inaccessible about any of this. The risk arises when the automated route becomes the only route.

Following the commencement of the revised EHRC Code of Practice for Services, Public Functions and Associations on 5 August 2026, businesses should be reviewing these systems through the anticipatory reasonable adjustment duty under the Equality Act 2010.

The practical question is simple: what happens when your technology doesn't work for a disabled customer?

Real world, right now: EMMA and the Rotherham GP patients

This isn't theoretical. On 20 August 2026 the Guardian reported that patients in Rotherham are hanging up on their GP practice's AI telephone receptionist, EMMA, because it can't understand their accents. Healthwatch Rotherham had flagged the problem a month earlier, on 10 July 2026, citing digital exclusion and communication breakdowns.

One patient told Healthwatch: “I could never get it to understand me, I ended up just hanging up and not bothering to try and book an appointment.” Others are walking to the surgery in person because the phone route no longer works for them. Healthwatch's manager, Kym Gleeson, put it plainly: the system can't always understand broad Yorkshire accents, and accents across South Yorkshire vary a lot.

EMMA is built by QuantumLoopAI and is the first DTAC-certified AI reception platform for NHS GP surgeries. The company says EMMA supports 17 languages besides English, is trained on a wide range of accents and dialects, and that any caller can ask to speak to a human at any time. Where EMMA can't understand or handle a request, the call is meant to transfer to the reception team.

So the exception journey exists, on paper. The question is whether it's genuinely accessible. When patients would rather hang up and walk to the surgery than use the phone, the fallback isn't working as a reasonable adjustment. It's working as a barrier.

Healthwatch Rotherham made the legal point clearly: GP practices still have a legal duty to make reasonable adjustments for patients who need them. That duty sits on the practice, and on the NHS commissioning authorities behind them. It doesn't transfer to the software supplier, and DTAC clinical-safety approval doesn't outsource it either.

This is the live example the rest of this article is about. AI doesn't remove the reasonable-adjustments duty. It can create a new one.

The full article walks through four composite case studies, the Strategic Access Test for deliberately failing your own technology, how to design an exception path, section 20(7) on cost, and the commercial opportunity in getting AI implementation right.

Case Study 1: A bank introduces voice authentication. Customers repeat a phrase before being transferred to telephone banking. A customer who stammers repeatedly fails authentication. After three attempts the system treats the failures as a potential security issue and locks telephone access. The technology has operated exactly as designed. That's the problem.

Case Study 2: A FinTech business requires customers to authenticate through facial recognition. A customer has a condition affecting facial movement. The software repeatedly rejects the scan. Customer support tells them: “You'll need to keep trying until the app accepts it.” That isn't an accessibility strategy.

The Access Advantage — Subscribers Only

This article continues in The Access Advantage

The full article walks through four composite case studies (bank voice authentication, FinTech facial recognition, app-only retail, and AI fraud flags), the Strategic Access Test for deliberately failing your own technology, how to design an exception journey, section 20(7) on cost, and the commercial opportunity in better AI implementation.

Subscribers get monthly legal briefings translated into operational language, direct access to ask one question per month answered personally, and first sight of every tool and framework before public release.

Subscribe to read in full

First month free — £14.99/month or £150/year after that.

What's in the full article

Four case studies

Bank voice authentication, FinTech facial recognition, app-only retail, and AI fraud flags mistaking disability for suspicious behaviour. Composite scenarios drawn from real patterns.

The Strategic Access Test

Deliberately fail your own technology, then follow the customer journey. The eight questions that tell you more than any supplier accessibility statement.

Design the exception path

Map two routes, primary and exception. Why disability-related failure belongs alongside fraud and forgotten-password paths, and what section 20(7) really means for cost.

CF
Chris Fry

Accessibility specialist and legal professional. Postgraduate legal qualifications, CEDR mediation accreditation. Experience in Court of Appeal and Supreme Court cases, contributions to House of Lords and Commons Select Committee proceedings. Director, Strategic Access Advisory Limited.

The EMMA section describes a real, reported case drawn from Healthwatch Rotherham's blog of 10 July 2026 and national press coverage from 20 August 2026. The four case studies are illustrative composite scenarios and do not describe individual reported cases. This article provides general legal and operational information and doesn't constitute legal advice. For organisation-specific guidance, contact us.

⚖️

Dynamic Disability Assessment™

A risk assessment tool grounded in 15 years of accessibility case law. Audit your automated customer journeys and exception paths before a claim does it for you.

Contact Chris to find out more

The Access Advantage

Monthly briefings. Operational language. One question per month answered personally. First month free.

Subscribe — first month free

Share this article

Share on LinkedIn

Related reading

The warning period has ended. The replacement statutory Code now applies.

The revised EHRC Code commenced 5 August 2026. What every service provider needs to do now.

Read article →

Reasonable Adjustments Examples: What Works in Sport, Music, Entertainment and Live Events

What a good reasonable adjustment looks like in practice, and the patterns that end up in Court.

Read article →

General legal information, not legal advice. For organisation-specific guidance, contact us.

Crash test your tech before a claim does it for you

The Access Advantage translates legal developments into operational language every month. First month free, no credit card required.

Subscribe to The Access Advantage

All articles

Insights & Updates

Practical guidance on accessibility law, compliance, and customer experience from the people who work at the sharp end.

View all articles