quorum-cve-triage
About the project
The problem
A new CVE drops and your lockfile says you have the package. Now what? Most scanners stop there and hand you a pile of “critical” alerts. The question that actually matters is: does this vulnerability reach my code?
Quorum is our attempt at answering that, locally, with no cloud and no hosted LLM.
What it does
You give Quorum a security advisory and a codebase. It breaks the advisory into small, checkable claims (“the vulnerable function is imported”, “untrusted input reaches it”, “the pinned version is affected”) and has three Gemma personas argue about each one:
- Proposer argues the advisory applies
- Red Team has to call local MCP tools (grep, read_file, find_callers) and look at the actual code before it’s allowed to judge
- Referee reads the arguments with the authors hidden and rules on the evidence
The panel runs several times with different seeds. We don’t ask the model “how sure are you?”, because small models are bad at that. Instead the code counts how often the verdict stays the same across runs. Stable votes mean high confidence. Claims where the model keeps changing its mind get flagged as questions for a human.
Facts like versions, imports and symbol hits are collected by plain Python first, so the model can’t invent a version number.
Playground API
Besides the CLI and the Streamlit UI, Quorum has a playground API. You can send it an advisory and a codebase from your own scripts or tools and get the triage result back as JSON. That makes it easy to plug into CI or try ideas without clicking through a UI.
Model switcher
The reasoning and tool-calling models aren’t hard-wired. You can switch models from the UI, the CLI or the API, and compare how different Gemma variants handle the same advisory. Since confidence comes from agreement across runs, it’s also a handy way to see whether a bigger model really is more stable than a smaller one.
Demo
The repo ships two sample apps with the same advisory and the same pinned package version:
-
codebase_acalls the vulnerable function from a web route, so it comes out likely exploitable -
codebase_bonly uses a safe function, so it comes out likely not exploitable
Same CVE, opposite verdicts, and the report shows why.
How we built it
Python, Ollama running gemma3:4b (reasoning) and functiongemma:270m (tool calls), LangChain, FastMCP, Pydantic, Typer and Streamlit. It has a CLI that writes an HTML + JSON report, a Streamlit UI, the playground API, and an optional spoken briefing of only the contested claims. Wi-Fi can be off once the models are pulled.
What we learned
- Self-reported confidence from a 4B model is basically noise, but agreement across seeded runs is a usable signal
- Giving the Red Team real tools made it far less likely to hand-wave
- Getting small models to return clean JSON took more work than the reasoning itself
Try it
pip install -e .
ollama pull gemma3:4b && ollama pull functiongemma:270m
quorum triage samples/advisory.json samples/codebase_a --n 3 --out report_a.html
Licensed MIT. Reports are a triage aid, not a replacement for a human security review.
Built with
AI tools
Prize Categories Entered
- Submitted
Best Use of Gemma 4
- Submitted

Best Open-Source AI Project
- Event prizeSubmitted
Hacktoberfest Hack Day Winners
- Event prizeSubmitted
Best Beginner Hack
- Event prizeSubmitted
169pi Track Prize