Can AI Really Sign?

Stephen Ellis-Menton
7 days ago
11 min read

Reflecting on Dr Ben Saunders’ session about what AI research can teach us about sign language

A train is cancelled, the platform changes and an announcement needs to reach everyone quickly.

At a railway station, an AI-generated BSL message may be able to provide Deaf passengers with information that would otherwise only be available through spoken announcements and display screens.

Now imagine a very different situation.

A Deaf person is receiving a medical diagnosis, discussing their mental health or being interviewed by the police. The communication is personal, interactive and may carry serious consequences. Meaning can shift through a small change in language, expression or response.

Should the same technology be used in both settings?

Dr Ben Saunders’ presentation at The Together Conference 2026 did not offer AI as a single answer to every access problem. Instead, it asked us to look more closely at what the technology currently does, what it cannot yet do and where human professional knowledge remains essential.

His session, What Signapse Have Learned About Sign Language Through Our AI Research, was as much about the complexity of signed languages as it was about technology.

It also gave interpreters, translators and Deaf professionals an opportunity to move beyond two unhelpful extremes: assuming AI will shortly replace everyone, or refusing to engage with it at all.

What are we actually talking about?

Artificial intelligence is a broad term. During Ben’s session, the focus was specifically on technology that produces sign language from written or spoken content.

Signapse develops translations from English into BSL and ASL. Ben was clear that these are separate products using separate language data and grammar. The company does not currently work in the opposite direction by using AI to recognise a person’s signing and translate it into written or spoken language.

That distinction matters.

An AI system producing one version of a signed message is not the same as a system being able to understand any Deaf person who signs towards it.

People have different language backgrounds, regional signs, movements, appearances and ways of expressing themselves. An AI recognition system would need to understand that range of real-world variation, which Ben described as a much more difficult technical task.

Signapse uses what it calls digital signers. These are photorealistic rather than cartoon-style avatars, so the finished videos are designed to look like a real person signing.

Some are based on the appearance and signing of real Deaf people. Ben also showed examples of anonymous digital signers created for the company’s ASL work after users expressed a preference for the content not to be associated with one recognisable individual.

The video may look like a person is signing the whole sentence, but the completed message is generated by the system from language and movement data.

This is already a useful reminder. When we see an apparently fluent signer on screen, we should ask what we are actually watching.

Is it a recorded Deaf translator?

Has an existing translation been edited digitally?

Is it a generated signer based on recorded language data?

Has a human checked the final message?

The visual result alone does not tell us how the translation was produced or who was involved in it.

It is not simply replacing English words with signs

One of the most useful parts of Ben’s presentation was his explanation of what sits behind an AI-generated translation.

Signapse approaches the process in two broad stages.

The first stage translates an English sentence into a written gloss sequence intended to represent the order and structure of the BSL message.

The second stage uses that sequence to produce the final sign language video.

Even the gloss stage is far more complicated than matching one English word to one sign.

The system’s labels include information about:

different sign variants
which sign may be appropriate in a particular context
directional verbs, such as who is helping whom
non-manual features
grammatical order
the movement needed for a particular form of the sign

The large language model is trained using thousands of English and gloss translation examples. From these, it begins to identify patterns in how a new English sentence might be expressed in BSL.

Deaf translators remain involved in creating the examples, checking the gloss and reviewing translations produced for clients. Ben explained that some of the videos shown during the session had been through human quality checks and adjustments, even though the final visual production was AI-generated.

This is not an insignificant detail.

The phrase “AI translation” can give the impression of a machine independently understanding English and producing BSL.

Ben’s explanation showed a process built from human language data, translation decisions, labels, recordings, technical design and continued quality checking.

The AI can learn patterns from the information it receives, but people still decide what information it receives and what a satisfactory translation should look like.

What has AI research revealed about BSL?

Trying to teach a machine to produce BSL makes some of the language’s complexity particularly visible.

A person can change their body position naturally when showing role shift. They can establish people or objects within the signing space, refer back to them and change the direction of a verb without consciously breaking the process into separate technical instructions.

An AI system needs those features to be labelled, recorded or learned through sufficient examples.

Ben discussed several areas that remain difficult.

Role shift and body movement

A human signer can turn their shoulders, alter their eye gaze and show the actions or perspective of another person as part of one smooth message.

For AI, changing the body position while keeping the movement natural and maintaining a smooth transition into the next sign is much harder.

Ben described role shift and natural shoulder movement as one of Signapse’s largest continuing challenges.

Movement between signs

A translation is not simply a series of accurate dictionary signs placed beside one another.

The movement between signs is part of what makes the language appear natural. If those transitions are stiff, abrupt or visually inconsistent, the message may be understandable but still not feel fluent.

Non-manual information

Facial expression, mouth patterns, head movement and body position can carry grammatical and emotional information.

These cannot be added as decoration after the manual signs have been selected. They need to be part of the system’s understanding of how the message should be produced.

Direction and placement

The direction of a verb may show who is carrying out an action and who receives it.

The signing space can also establish people, locations and relationships. These features require much more than translating the English vocabulary in the sentence.

Regional and individual variation

BSL varies across regions and communities. Different Deaf people may prefer different signs or styles of translation.

A human translator can consider the intended audience and make choices accordingly. A digital translation placed on a public website may be viewed by someone in Newcastle, London or elsewhere, without the system knowing who the user will be.

Ben discussed the possibility of future systems allowing users to select a regional preference or translation style. At present, however, one generated version cannot reflect every form of BSL used across the community.

He also noted differing preferences between people who wanted translations closer to BSL and those who preferred something closer to SSE. Trying to create one version for an undefined audience makes these decisions difficult.

This raises a wider point for translators.

There is rarely one neutral or universal sign language translation. Decisions about language, audience, register, regional variation and presentation still need to be made, whether the final product is recorded by a person or produced through AI.

Understandable is not the same as fluent

Ben shared results from a user evaluation in which members of the Deaf community were asked whether AI-generated translations were understandable.

He reported that 76 per cent of the translations assessed were considered understandable.

That result showed clear progress, but Ben was also careful about what it did not mean.

Understandable does not necessarily mean fluent BSL.

He acknowledged that the system can still lean towards SSE because it begins with an English sentence. Role shift, natural transitions and other aspects of BSL remain difficult.

This honesty was important.

Technology is often presented through carefully chosen examples showing its strongest output. A polished demonstration may lead viewers to assume the same quality can be achieved consistently with any subject, sentence or context.

The more useful question is not whether AI can produce one impressive BSL video.

It is whether the system can produce accurate, appropriate and natural language reliably across the full range of content it may be asked to handle.

A technology can be useful before it is perfect, but its limits need to be stated clearly so that people do not use it in situations where those limits could cause harm.

The right tool for the right setting

Ben made a clear distinction between settings where Signapse believes AI-generated sign language may currently be useful and settings where it should not be used.

He identified medical diagnoses, police interviews, mental health appointments and situations involving emotion or significant nuance as unsuitable.

In these settings, the consequences of an inaccurate or incomplete translation may be serious. Communication also needs to move in both directions, respond to the individual and change as the conversation develops.

A human interpreter can notice uncertainty, ask for clarification, respond to language variation and work with the participants in real time.

Current AI sign language generation does not provide that interaction.

Ben suggested that the technology is better suited to routine, one-way information, particularly where large amounts of written or spoken content currently have no sign language version.

Examples included:

transport announcements
platform changes
delays and cancellations
routine public information
static website content
some pre-recorded videos
repeated messages with a predictable structure

Railway stations have been an early area of Signapse’s work. Train announcements often follow repeated patterns, while information such as a new platform or delay needs to be updated quickly.

This is a very different task from interpreting a conversation where participants interrupt, ask questions, express uncertainty or disclose new information.

The distinction should not be between “AI is good” and “AI is bad”.

A better set of questions might be:

What is the purpose of this communication?
Is it one-way information or an interaction?
How serious would an error be?
Does the user need to ask questions?
Is the message routine or highly individual?
Is emotional nuance involved?
Has a qualified person reviewed the translation?
Is AI adding access where none currently exists, or replacing a service that people still need?

Adding access or reducing it?

One of the concerns raised during the conference discussion was that commissioners may use AI because it is cheaper, even when a human interpreter or translator would be more appropriate.

Ben acknowledged this concern.

An organisation unfamiliar with Deaf communities may see a signed video and assume its access responsibilities have been met. It may not understand the difference between routine information, translation and live interaction.

The risk is that a tool developed to increase the amount of sign language available could instead be used to reduce access to qualified professionals.

Ben said Signapse has a governance framework setting out where it believes the technology should and should not be used. He also described education as part of the company’s responsibility when working with public organisations and other clients.

That responsibility cannot sit only with the technology provider.

Commissioners, service providers and public bodies also need to understand that different communication needs require different forms of access.

A generated train announcement may provide information that was previously inaccessible.

An AI signer replacing a qualified interpreter during a medical diagnosis would remove essential interaction and professional judgement.

Those are not equivalent uses of the same tool.

What does this mean for interpreters and translators?

Ben repeatedly stated that Signapse does not see its technology as a replacement for interpreters.

His view was that AI could provide routine translations at a scale that the current workforce could not realistically cover, while human professionals continue to work in settings requiring interaction, nuance and judgement.

For interpreters and translators, that may lead to new professional roles as well as new concerns.

Deaf translators may contribute to:

creating training data
developing gloss systems
recording sign variants
reviewing AI output
advising on audience and register
testing whether translations are understandable
setting quality standards
identifying unsuitable uses
contributing to governance and policy

Interpreters may need to explain why live, two-way communication cannot be replaced by a generated message.

Educators may need to prepare students to understand these systems and discuss them accurately with service providers.

Professional bodies may need to consider what accountability looks like when access is produced through a mixture of human decisions and automated output.

Researchers may examine how Deaf audiences respond to different forms of digital signing and whether systems reproduce bias from the data used to train them.

AI does not remove the need for professional knowledge. It may make that knowledge more important, particularly when organisations are deciding whether and how to use the technology.

Deaf involvement must be more than consultation

Ben explained that Deaf people are involved across Signapse, including within the AI team, translation work, product development and the company’s wider activities.

The signs used by the system were translated or recorded by registered Deaf sign language translators. Deaf translators also check and amend client translations.

This matters because sign language technology cannot be developed responsibly through technical knowledge alone.

However, Deaf involvement also raises questions that the profession and technology sector will need to keep discussing:

Who owns the language data used to train a system?
How is consent obtained from the people whose appearance or signing is recorded?
Can someone withdraw that consent later?
Who is paid when their language data continues to generate new content?
How are regional and cultural differences represented?
Who has authority to decide whether the output is good enough?
What happens when Deaf users disagree about the preferred translation?
Is Deaf involvement present throughout decision-making, or only at the final testing stage?

Ben’s session did not claim to resolve every one of these questions. It did, however, show why Deaf translators, communities and linguists need to remain central as the technology develops.

What are we passing on about AI?

The 2026 conference theme asked us to consider what we pick up and pass on through our professions.

AI is an area where information can be passed on very quickly, whether it is accurate or not.

We may pass on fear that every interpreting and translation role is about to disappear.

We may repeat marketing claims suggesting that technology can already understand and produce fluent sign language in any situation.

We might dismiss all AI work without examining whether a particular use could give Deaf people access to information that currently has no BSL version.

None of these positions gives us a full picture.

A more responsible approach is to pass on questions.

What exactly does this system do?
Which language direction does it work in?
Who created and checked the translation?
What are its known limitations?
Where should it not be used?
Is it creating access or replacing appropriate access?
Who is accountable when it goes wrong?

Ben’s presentation encouraged informed caution rather than either excitement or rejection without scrutiny.

That may be one of the most useful professional responses we can pass on.

New insights require shared responsibility

The theme for The Together Conference 2027 is New Insights, Shared Purpose.

There is unlikely to be a shortage of new developments in AI before the next conference. The harder task will be deciding which developments are useful, ethical and genuinely connected to Deaf people’s priorities.

The discussion cannot be left only to technology companies.

It needs contributions from Deaf communities, translators, interpreters, linguists, educators, researchers, professional bodies, service providers and commissioners.

Perhaps you are already researching sign language technology.

You may have tested AI-generated translations with Deaf audiences.

You might have experience of a service replacing human access with an automated tool.

You may be developing guidance, teaching materials, governance frameworks or ways of reviewing AI output.

You might also be using technology in a focused way that has genuinely increased access.

These experiences could contribute to the programme for The Together Conference 2027.

The Call for Papers opens on Monday 5 October 2026. You do not need to decide whether your proposal belongs in the live programme or the extended conference. Submit your idea and the review panel will consider where accepted contributions fit best.

Poster submissions will also be welcomed for developing research, early findings, professional projects and practice-based reflections.

AI will continue to change.

Our shared purpose must be to ensure that access, language quality, accountability and Deaf leadership do not become secondary concerns while it does.

Using this article for unstructured CPD

You may wish to reflect on one or more of the following questions:

Before this session, what did you understand AI-generated sign language to be?
How is producing sign language different from recognising and understanding someone else’s signing?
Which elements of BSL appear particularly difficult for AI to reproduce?
What is the difference between an understandable translation and a fluent one?
In which settings might AI-generated sign language add useful access?
In which settings should a human interpreter or translator remain essential?
How could commissioners mistakenly use AI as a cheaper replacement rather than an additional access tool?
What role should Deaf translators and communities have in the design and review of these systems?
What questions would you ask before recommending or using an AI sign language product?
Is there an experience, project or piece of research from your work that could become a 2027 conference proposal or poster?