Sema: I built a reading companion for my brother
The person I built this for
My elder brother is the most capable person I know. He can fix a car, wire a house, repair almost any electronic you put in front of him. He built his whole livelihood with his hands, and he lives well.
He also has strong dyslexia.
Growing up in Africa, not many people understood what that meant. They saw someone who couldn't read easily and decided he was lazy, or not clever. He was neither. When we were kids, he was the one who ironed my clothes and shined my shoes so school would be easier for me. Our parents pushed him from one class to the next until he finished high school, even though his grades were the worst in the room; not because he didn't try, but because he couldn't read the way school demanded.
To this day, when he needs to send an email, he calls me and I write it. When he's stuck on an app, he photographs the screen and I talk him through it, step by step. When a letter or a form arrives, I'm the one who reads it. For years, I have quietly been his eyes for text. He hides the dyslexia from most people, because the world taught him to be ashamed of it.
One day he asked me a simple question: "Is there an app that could just do this for me?"
That question is why Sema exists. (Sema means "read" or "speak" in Kiswahili.) I decided to build the exact thing I had been doing for him by hand — so he could have his independence and his dignity back, without needing to call anyone to read his own life.
What Sema does
Sema is a Gradio app with three tools, each replacing one of the things he used to call me for.
ReadEasy - reading. Paste any text — a message, a bill — and Sema simplifies it, summarises it, or reads the original aloud. The output is shown in a dyslexia-friendly font (OpenDyslexic), one sentence per line, with generous spacing, and it's read out loud so he can listen along. It works in English and Kiswahili.
VoiceOut - writing. He just speaks. "Tell my client the job is done and the invoice is attached." Sema works out who the message is for and writes it in the right tone - a clear email for a client, a warm note for family — ready to copy and send. He picks his language before recording so it transcribes him correctly.
SnapGuide - navigating. He photographs a screen he's stuck on, asks his question (typed or by voice), and Sema gives him simple step-by-step instructions, highlighting exactly what to tap. Upload the next screen and it continues the same task.
How I built it
This section is as much about lessons as about architecture.
Small models, chosen for the job. Everything runs on open models under the 32B cap:
- Qwen2.5-7B-Instruct for ReadEasy and VoiceOut (text understanding and writing, multilingual).
- MiniCPM-V from OpenBMB for SnapGuide — a ~1.3B vision model that punches far above its size at OCR and document understanding. For an app whose whole job is reading screens and forms, this was the obvious fit.
- Whisper-large-v3 for turning speech into text.
The lessons that cost me the most time:
Accents break speech recognition silently. My brother speaks English, but his accent sometimes made the transcriber guess Swahili mid-sentence. The fix was letting the user pick their language before recording and passing that hint to the model, so it stops guessing.
"The model exists" is not "the model is available to you." I lost a lot of hours to errors that said a vision model wasn't "supported by any provider you have enabled." The real lesson: a model existing on paper means nothing until you confirm your own account can actually reach it through a working provider. Checking my inference-provider settings — instead of guessing model names — is what finally solved it.
Build a safety net. SnapGuide tries MiniCPM-V first, but if that endpoint is ever busy, it automatically falls back to a second open vision model so the app never simply breaks for the user. A person relying on this to read their mail can't be met with an error screen.
Accessibility is in the small details. The dyslexia-friendly font, the one-sentence-per-line spacing, audio on every output, plain-language responses, and Kiswahili support aren't extras — they are the app. An assistant that's hard to read would defeat its own purpose.
What's next
If this goes anywhere, I want to add phonics, transform this into a professional app with ease to use UI Design — to help people with dyslexia not just work around reading, but actually begin to learn to read and write. Sema today gives my brother control. The next version could give someone the skill itself.
The why, one more time
I didn't build Sema to fix my brother. He was never broken. The world just measured him with the one ruler he couldn't use, and ignored everything he was brilliant at.
I built it so he doesn't have to call anyone to read his own life. Small models, real help — for my brother, and for everyone like him.
Live App
▶️ https://build-small-hackathon-sema-dyslexic-assistant.hf.space/
Demo video
▶️ https://youtu.be/6VZcPdaC75s
Social post
🔗 https://x.com/JamesGo01862630/status/2065063242280423869?s=20