DrishtiAI tells a blind or low-vision person what is in front of them, out loud, in English and Hindi. It runs entirely on the user’s own machine: an open-weight vision model, offline speech in and out, and no cloud calls.

What it does

• Surroundings: warns about physical hazards first (what, direction, rough distance), then gives one line about the scene.
• Read: reads signs, labels and packaging.
• Find: you name an object; it answers found, not found or uncertain.
• Voice commands: say “describe”, “read” or “find my water bottle”. No typing or tapping.
• Phone mode: a phone on the same network becomes the camera and speaker.

How open-source AI is central

• Vision: Qwen3-VL-2B-Instruct (Apache-2.0) running locally through Ollama.
• The model must return strict JSON, validated with Pydantic. An invalid answer is retried once, then marked as a rough description. Hazards are never invented.
• Speech out: Piper. Speech in: Whisper. Both run offline.
• The same pipeline is packaged as an Agent Skill (agentskills.io standard).

Measured, not claimed
On a fixed set of 30 images: 14 of 15 hazards found, 6 false alarms, 14 of 15 texts read correctly, about 1.5 s per image on a laptop GPU. The first real run found only 9 of 15 with 31 false alarms; the README documents what failed and how it was fixed.

Limits: the prompt was tuned on the same images it was scored on, it still over-warns on clear scenes, and Hindi output uses a fixed glossary. It is an experimental aid, not a replacement for a cane or a guide.