Skip to main content

About the project

The problem

A new CVE drops and your lockfile says you have the package. Now what? Most scanners stop there and hand you a pile of “critical” alerts. The question that actually matters is: does this vulnerability reach my code?

Quorum is our attempt at answering that, locally, with no cloud and no hosted LLM.

What it does

You give Quorum a security advisory and a codebase. It breaks the advisory into small, checkable claims (“the vulnerable function is imported”, “untrusted input reaches it”, “the pinned version is affected”) and has three Gemma personas argue about each one:

  • Proposer argues the advisory applies
  • Red Team has to call local MCP tools (grep, read_file, find_callers) and look at the actual code before it’s allowed to judge
  • Referee reads the arguments with the authors hidden and rules on the evidence

The panel runs several times with different seeds. We don’t ask the model “how sure are you?”, because small models are bad at that. Instead the code counts how often the verdict stays the same across runs. Stable votes mean high confidence. Claims where the model keeps changing its mind get flagged as questions for a human.

Facts like versions, imports and symbol hits are collected by plain Python first, so the model can’t invent a version number.

Playground API

Besides the CLI and the Streamlit UI, Quorum has a playground API. You can send it an advisory and a codebase from your own scripts or tools and get the triage result back as JSON. That makes it easy to plug into CI or try ideas without clicking through a UI.

Model switcher

The reasoning and tool-calling models aren’t hard-wired. You can switch models from the UI, the CLI or the API, and compare how different Gemma variants handle the same advisory. Since confidence comes from agreement across runs, it’s also a handy way to see whether a bigger model really is more stable than a smaller one.

Demo

The repo ships two sample apps with the same advisory and the same pinned package version:

  • codebase_a calls the vulnerable function from a web route, so it comes out likely exploitable
  • codebase_b only uses a safe function, so it comes out likely not exploitable

Same CVE, opposite verdicts, and the report shows why.

How we built it

Python, Ollama running gemma3:4b (reasoning) and functiongemma:270m (tool calls), LangChain, FastMCP, Pydantic, Typer and Streamlit. It has a CLI that writes an HTML + JSON report, a Streamlit UI, the playground API, and an optional spoken briefing of only the contested claims. Wi-Fi can be off once the models are pulled.

What we learned

  • Self-reported confidence from a 4B model is basically noise, but agreement across seeded runs is a usable signal
  • Giving the Red Team real tools made it far less likely to hand-wave
  • Getting small models to return clean JSON took more work than the reasoning itself

Try it

pip install -e .
ollama pull gemma3:4b && ollama pull functiongemma:270m
quorum triage samples/advisory.json samples/codebase_a --n 3 --out report_a.html

Licensed MIT. Reports are a triage aid, not a replacement for a human security review.

Built with

pythonstreamlitollamagemmalangchainmcppydantic

AI tools

Coded withClaude CodeCursorGemini 3.8 FlashGemini 3.5 Flash LiteClaude Haiku 5.5Claude Sonnet 5.5Claude Opus 5.5GPT-5.5Older GrokKimi K2.7 Code

Prize Categories Entered

  • Deepmind Google

    Best Use of Gemma 4

    Submitted
  • DigitalOcean

    Best Open-Source AI Project

    Submitted
  • Event prize

    Hacktoberfest Hack Day Winners

    Submitted
  • Event prize

    Best Beginner Hack

    Submitted
  • Event prize

    169pi Track Prize

    Submitted