Audiodidact
About the project
🎧 Audiodidact: understand any open-source repo by ear
Audiodidact turns a repository’s architecture diagrams into a spoken, voice-navigable tour, so blind and visually impaired developers can understand a project and start contributing.
The problem
Open-source projects explain themselves with diagrams: system architecture, database ER diagrams, data flows. For blind and visually impaired developers, these are a wall. Alt text is usually missing or just says “architecture diagram,” with nothing about the components or arrows. A developer can read every line of code and still lack the mental model everyone else gets in ten seconds from one image.
What it does
- Paste a GitHub repo URL. Audiodidact finds the diagrams in the README and docs.
- Gemma reads them. The multimodal model converts each diagram into a structured graph of components, connections, and layers, with a confidence score and warnings for anything unclear.
- ElevenLabs narrates them. The graph becomes an audio tour with distinct voices for different system layers (frontend, backend, data), so you know where you are by ear.
- Talk to it. Say “Drill down into the database” or “What depends on the payments service?” and get a spoken answer.
-
Go from understanding to contributing. It surfaces
good first issuetasks tied to the parts you just explored.
Grounded by design
A blind user can’t glance at the image to double-check, so answers come only from the extracted diagram data. If something isn’t in the diagram, Audiodidact says so: “That isn’t shown in the diagrams I have for this repository.” We chose honesty over a confident guess.
Built with
- Gemma (multimodal): diagram image to validated JSON graph
- ElevenLabs: multi-voice narration
- DigitalOcean Gradient AI: agent with a per-repo knowledge base for Q&A
- DigitalOcean Spaces: caching diagrams, graphs, and audio
- DigitalOcean App Platform: hosting
- Next.js, TypeScript, Tailwind, Zod: accessible full-stack app with validated model output
- Web Speech API: push-to-talk voice commands, with a typed fallback
- GitHub API (Octokit): diagram discovery and good-first-issue lookup
Accessibility
Fully keyboard-usable, transcript for every audio response, and a text outline of the diagram graph so voice is optional, not required.
Open source
MIT licensed and self-hostable, so any maintainer can run it for their own project.
Honest limitations
Diagram understanding can be wrong on dense or low-resolution diagrams, so we show confidence and warnings. We currently support architecture and ER diagrams, and we haven’t yet validated it with blind or visually impaired developers. That feedback is our first next step.
What’s next
Sequence diagrams and flowcharts, Mermaid/PlantUML parsing, spatial audio per layer, a browser extension for “explain this diagram” on any GitHub page, self-hosted Gemma on a GPU Droplet, and an accessibility audit with blind contributors.
Built with
AI tools
Prize Categories Entered
- Submitted
Best Use of Gemma 4
- Event prizeSubmitted
Hacktoberfest Hack Day Winners
- Event prizeSubmitted
ElevenLabs Track Prize