Less context, richer context

A language model only knows the text you send it. When a chat gets long, the full history no longer fits, so something has to choose what goes in.

RALC (Relational Context Allocation) is an open-source, local-first Python library that makes that choice. It stores conversation history as a graph of messages and the links between them, and, given a question and a token budget, selects the messages that belong together. The question is never modified, RALC never touches the model’s answer, and it works with any LLM.

How it works

1. Every message is stored as a node in a local graph (SQLite + NetworkX).
2. An extractor connects messages: who answered what, which messages mention the same thing, and which decision replaced an earlier one. Gemma 4 can do this step and understands changes a simple pattern can’t.
3. For a question, local embeddings find starting points, RALC follows the links to related messages, ranks them, and fills a strict token budget.
4. Linked messages are kept together instead of being dropped as duplicates.

Gemma’s role

Gemma finds relationships a regex can’t. In our test conversation, the pattern extractor found 0 “this decision replaced that one” links; Gemma found 31. With those links, RALC kept both halves of a decision change (Heroku, then AWS) that similarity-based selection split apart.

Results (pre-registered, from real runs)

• Using relationships found more complete answers than the same system without them, on multi-message questions, at every budget on both test sets.
• On simple one-fact questions, RALC ties ordinary vector search.
• With Gemma’s links, complete answers on multi-message questions went from 0.83 to 1.00 at a 1,000-token budget (one recovered question).

Negative results: stale messages did not confuse Gemma in our test, and linking decisions by topic did not work.

Limitations: one synthetic conversation, 14 to 17 questions, one Gemma model; we measured which messages were selected, not answer quality. 178 tests passing.

Dataset

A synthetic software-team conversation we generated for testing: 234 messages (about 21,000 tokens) in set 1 and 244 messages in set 2, with real decisions, decisions that were later reversed, bugs and their fixes, and lots of unrelated chatter as distractors. Set 2 is harder: it adds near-miss messages on the same topics, important messages placed late, and questions that need 3 or 4 messages. We wrote 14 and 17 questions with the messages needed to answer each, and reviewed those labels by hand. Every experiment was pre-registered before running.

How it can get better

• Test on real conversations and standard benchmarks like LongMemEval, not only our own data.
• Measure answer quality: send the selected context to a model and check whether its answers are correct.
• Ask Gemma directly whether one decision replaces another, since our topic-based linking did not work.
• Run Gemma locally through Ollama, so nothing leaves the user’s machine.
• Add ways to plug it in: import exported chat files, an MCP server, and agent integrations.

Disclosure

RALC’s core pipeline (graph, retrieval, ranking, token allocation, Gemma extractor) was built the day before the event. During the event we ran the Gemma relationship experiments (stale-context test, relation-aware selection, topic linking), hand-reviewed the benchmark labels and recomputed the results, and built the demo page and demo video. The full commit history is public. Built with Claude Code as a coding assistant.