Skip to main content

About the project

🎧 Audiodidact: understand any open-source repo by ear

Audiodidact turns a repository’s architecture diagrams into a spoken, voice-navigable tour, so blind and visually impaired developers can understand a project and start contributing.

The problem

Open-source projects explain themselves with diagrams: system architecture, database ER diagrams, data flows. For blind and visually impaired developers, these are a wall. Alt text is usually missing or just says “architecture diagram,” with nothing about the components or arrows. A developer can read every line of code and still lack the mental model everyone else gets in ten seconds from one image.

What it does

  1. Paste a GitHub repo URL. Audiodidact finds the diagrams in the README and docs.
  2. Gemma reads them. The multimodal model converts each diagram into a structured graph of components, connections, and layers, with a confidence score and warnings for anything unclear.
  3. ElevenLabs narrates them. The graph becomes an audio tour with distinct voices for different system layers (frontend, backend, data), so you know where you are by ear.
  4. Talk to it. Say “Drill down into the database” or “What depends on the payments service?” and get a spoken answer.
  5. Go from understanding to contributing. It surfaces good first issue tasks tied to the parts you just explored.

Grounded by design

A blind user can’t glance at the image to double-check, so answers come only from the extracted diagram data. If something isn’t in the diagram, Audiodidact says so: “That isn’t shown in the diagrams I have for this repository.” We chose honesty over a confident guess.

Built with

  • Gemma (multimodal): diagram image to validated JSON graph
  • ElevenLabs: multi-voice narration
  • DigitalOcean Gradient AI: agent with a per-repo knowledge base for Q&A
  • DigitalOcean Spaces: caching diagrams, graphs, and audio
  • DigitalOcean App Platform: hosting
  • Next.js, TypeScript, Tailwind, Zod: accessible full-stack app with validated model output
  • Web Speech API: push-to-talk voice commands, with a typed fallback
  • GitHub API (Octokit): diagram discovery and good-first-issue lookup

Accessibility

Fully keyboard-usable, transcript for every audio response, and a text outline of the diagram graph so voice is optional, not required.

Open source

MIT licensed and self-hostable, so any maintainer can run it for their own project.

Honest limitations

Diagram understanding can be wrong on dense or low-resolution diagrams, so we show confidence and warnings. We currently support architecture and ER diagrams, and we haven’t yet validated it with blind or visually impaired developers. That feedback is our first next step.

What’s next

Sequence diagrams and flowcharts, Mermaid/PlantUML parsing, spatial audio per layer, a browser extension for “explain this diagram” on any GitHub page, self-hosted Gemma on a GPU Droplet, and an accessibility audit with blind contributors.

Built with

geminielevenlabsgemma

AI tools

Coded withGemini 3.8 FlashOlder GeminiAntigravityGoogle AI Studio

Prize Categories Entered

  • Deepmind Google

    Best Use of Gemma 4

    Submitted
  • Event prize

    Hacktoberfest Hack Day Winners

    Submitted
  • Event prize

    ElevenLabs Track Prize

    Submitted