WITNESS
About the project
Every claim, with proof.
AI chatbots sound just as sure when they are wrong as when they are right. WITNESS checks an AI-written answer against its source documents, sentence by sentence, and shows the exact source line as proof.
How it works
- Hybrid retrieval (BM25 + MiniLM embeddings, merged with reciprocal rank fusion) finds the top 5 passages.
- A small local model (Gemma 4 E2B in LM Studio) writes the answer, or you paste any AI’s answer (for example ChatGPT) to grade it.
- The answer is split into claims with exact character offsets.
- Two independent judges verify every claim: Gemma 4 through the Gemini API, and an offline NLI model (DeBERTa-v3-small).
- A fixed rule combines the votes into Supported, Unsupported or Contradicted. When the judges disagree, the claim is flagged for a human instead of getting a confident wrong colour.
- Click any sentence to see both judges’ reasoning and the exact source sentence. “Fix it” rebuilds a clean answer with only supported claims, each with a citation.
Highlights
- Grade any AI: paste ChatGPT’s answer and it is graded the same way. Or upload your own PDF (kept in memory only, never saved).
- Replay mode answers in under 200 ms with no model and no network.
- Measured on 42 labelled claims: Gemma alone 90.5%, NLI alone 69.0%, both judges combined 83.3%. The labels are drafts and the set is small, and we say so.
- Offline automated test suite, plus an Agent Skill (backend/skill/witness-verify).
Built only on open-weight models (Gemma 4 E2B, Gemma 4 26B-A4B, DeBERTa-v3, MiniLM). Apache-2.0.
Team Alpha: Aryan, Mudit, Himanil, Rajat.
Built with
pythonfastapireacttypescriptviteTailwind CSSgemmaGemini APILM Studiosentence-transformerspytorchpydanticpytest
AI tools
Coded withClaude CodeGoogle AI StudioGemma 4 26B A4BClaude Opus 5.5Claude Sonnet 5.5
Prize Categories Entered
- Submitted
Best Use of Gemma 4
- Submitted

Best Open-Source AI Project
- Event prizeAccepted
Hacktoberfest Hack Day Winners