How Ishara AI is Rebuilding Digital Classrooms

Most online courses handle accessibility for deaf students by simply turning on text captions. While captions are great for watching movies, they fall short when you are trying to learn complex technical skills. Written text follows the exact grammar rules of spoken speech. Sign language does not. Kenyan Sign Language utilizes its own syntax, relying on distinct hand shapes paired with facial expressions. For a deaf student, reading a standard textbook layout is like trying to translate a foreign language on the fly, which makes learning exhausting.

Nairobi startup Ishara AI just won the top innovation award at the Inclusive Africa Conference by fixing this exact linguistic gap. Instead of forcing students to read flat text, they built a system that lets users interact with a digital learning space using their native language.

By decoupling the translation engine from heavy video streaming, the architecture shifts the processing burden away from user hardware.

Interactive Avatars Replace Heavy Video

Standard web video tools are not built for sign language instruction. A flat, regular video locks the camera in one position, making it hard to see the exact depth or side-view of a hand movement.

The Ishara Learning platform replaces heavy, data-draining video files with a lightweight 3D animated assistant. Because the avatar renders dynamically right on the student's screen, users can rotate the camera angle or slow down the speed. This lets them see exactly how a tricky programming or design sign is formed from the side or back, without eating up expensive mobile data packages.

Translating live sign language requires a massive amount of visual processing. The software uses an advanced computer vision model called VisionLLaMA to track rapid movements.

Instead of isolating finger movements, the model maps hand coordinates simultaneously with facial expressions, like a change in the eyebrows or mouth. In sign language, these facial shifts act like punctuation and verb tenses. By tracking the face and hands together, the AI prevents translation errors where two identical hand gestures mean completely different things. The system works both ways: it can turn a live video of someone signing into text, or take spoken words and instantly turn them into an avatar's movements.

Opening the Framework to External Developers

To scale this tool outside their own virtual classroom, the team launched a web SDK and API. This allows external developers to embed sign language translation directly into retail banking apps and public utility portals with just a few lines of clean code:

import Ishara from '@ishara/sdk'

const client = new Ishara({
  apiKey: process.env.ISHARA_API_KEY,
})

// Translating a live video stream into text
const result = await client.translate({
  input: videoBuffer,
  from: 'ksl',
  to: 'en',
})