Skip to main content

About the project

Every claim, with proof.

AI chatbots sound just as sure when they are wrong as when they are right. WITNESS checks an AI-written answer against its source documents, sentence by sentence, and shows the exact source line as proof.

How it works

  1. Hybrid retrieval (BM25 + MiniLM embeddings, merged with reciprocal rank fusion) finds the top 5 passages.
  2. A small local model (Gemma 4 E2B in LM Studio) writes the answer, or you paste any AI’s answer (for example ChatGPT) to grade it.
  3. The answer is split into claims with exact character offsets.
  4. Two independent judges verify every claim: Gemma 4 through the Gemini API, and an offline NLI model (DeBERTa-v3-small).
  5. A fixed rule combines the votes into Supported, Unsupported or Contradicted. When the judges disagree, the claim is flagged for a human instead of getting a confident wrong colour.
  6. Click any sentence to see both judges’ reasoning and the exact source sentence. “Fix it” rebuilds a clean answer with only supported claims, each with a citation.

Highlights

  • Grade any AI: paste ChatGPT’s answer and it is graded the same way. Or upload your own PDF (kept in memory only, never saved).
  • Replay mode answers in under 200 ms with no model and no network.
  • Measured on 42 labelled claims: Gemma alone 90.5%, NLI alone 69.0%, both judges combined 83.3%. The labels are drafts and the set is small, and we say so.
  • Offline automated test suite, plus an Agent Skill (backend/skill/witness-verify).

Built only on open-weight models (Gemma 4 E2B, Gemma 4 26B-A4B, DeBERTa-v3, MiniLM). Apache-2.0.

Team Alpha: Aryan, Mudit, Himanil, Rajat.

Built with

pythonfastapireacttypescriptviteTailwind CSSgemmaGemini APILM Studiosentence-transformerspytorchpydanticpytest

AI tools

Coded withClaude CodeGoogle AI StudioGemma 4 26B A4BClaude Opus 5.5Claude Sonnet 5.5

Prize Categories Entered

  • Deepmind Google

    Best Use of Gemma 4

    Submitted
  • DigitalOcean

    Best Open-Source AI Project

    Submitted
  • Event prize

    Hacktoberfest Hack Day Winners

    Accepted