DocSaathi: Private, local AI that explains English documents in Tamil, Hindi or simple English

TAGLINE
Photograph a bill or notice. Gemma 4 reads it on your own machine, tells you what to do, by when and how much, and shows the exact source text for every fact. Nothing leaves your laptop. Translation tells you what the words say. DocSaathi tells you what to do.

INSPIRATION
Much of everyday Indian life runs on English paperwork: bank notices, electricity bills, insurance letters, rental agreements, tax and legal notices. Many people who must act on them read only Tamil, Hindi or another regional language. So they hand the document, and often their Aadhaar, PAN and account numbers, to a shopkeeper, agent or cyber-cafe operator. Deadlines get missed, helpers misread or overcharge, and fake “your account will be blocked” notices are hard to tell from real ones. Cloud AI and camera-translate apps mean uploading private documents, and translation alone does not say what to do.

WHAT IT DOES

• Reads a photo of an English document using Gemma 4 vision.
• Extracts the key facts: document type, sender, amount due, due date, reference IDs, required actions and penalties.
• Explains in plain language in Tamil, Hindi or simple English, streamed as it is written, leading with what to do, by when and how much.
• Shows evidence: every fact links to the exact words it came from, shown next to the transcription.
• Answers follow-up questions from that document only. If the answer is not there, it says “this document does not say” and suggests who to ask.
• Redacts Aadhaar, PAN, phone and account numbers.
• Checks for scam signals and gives a verdict: likely genuine, suspicious or likely scam.
• Creates a calendar reminder (.ics file) for the due date.

HOW IT WORKS: CODE HANDLES FACTS, THE MODEL HANDLES LANGUAGE
Language models misread numbers and invent dates, so we do not let the model decide facts.

1. Pass 1, understand: Gemma 4 transcribes the image and returns structured JSON, with a quoted source text for every fact.
2. Verify, in plain code: a deterministic engine checks that each date is valid and plausible, calculates days left and overdue status, confirms the amount appears in its quote, and confirms each quote really exists in the transcription. Anything doubtful sets a “needs paper check” flag, and the app says: “Please check this against your paper copy.”
3. Pass 2, explain: Gemma receives only the verified facts, not the image, and writes the explanation in the chosen language.
4. Q&A: each answer ends with a source quote that is located in the transcription. If it cannot be found, nothing is highlighted, so the app never points at the wrong place.
   Scam checking combines Gemma’s judgement with rules (OTP requests, UPI requests, urgency wording, short links). The rules can raise the verdict but never lower it.

WHERE GEMMA 4 IS USED
Gemma 4: reads a phone photo, classifies the document and extracts facts with source quotes, writes Tamil, Hindi and simple English, answers questions from the document, and judges whether a letter looks like a scam.
Plain code: validates dates and amounts, calculates days left and overdue status, locates every quote in the transcription, detects and redacts personal numbers, and builds the .ics reminder.

WHY OPEN WEIGHTS MATTER
Open weights make the privacy promise possible: your Aadhaar and bank letters never leave your machine. Inference costs nothing per document and works offline. Privacy is enforced in code, not just promised:

• The server refuses to start unless the Ollama address is localhost, 127.0.0.1 or ::1.
• It listens on 127.0.0.1 only, and CORS allows only localhost origins.
• Request bodies, images, transcriptions and model output are never logged.
• Nothing is stored on the server.

BUILT WITH
React, Vite, Node.js, Express, Server-Sent Events, Ollama and Gemma 4. The backend uses only three dependencies (express, cors, dotenv), with no database, vector store or cloud service. The backend has 69 passing automated tests.

CHALLENGES

• Making a small local model reliable with structured output and images. We used schema-constrained output, a prompt-only fallback and one retry.
• Making sure quotes are matched exactly, using Unicode-safe matching that works for Tamil and Devanagari text.
• Keeping latency acceptable on a laptop, with model warm-up and a smaller model fallback.

WHAT WE LEARNED
Splitting the work so that code handles facts and the model handles language made the product more trustworthy and easier to test.

LIMITATIONS
DocSaathi is a reading aid, not legal or financial advice. Accuracy depends on photo quality and the model. Only the first page of a PDF is processed. The scam check is a signal, not a guarantee.

WHAT’S NEXT
More document types and languages, a form-filling guide, and voice read-aloud.

MODEL
Gemma 4 (gemma4:latest) run locally through Ollama.