AI Engineer NYC 2026
Join AI Engineer and Major League Hacking (MLH) on Sunday, October 11, for a full day of building, experimenting, and meeting fellow AI engineers.
OpenComputeMesh
Share compute, run models, see every token: peer-to-peer AI inference with real token, latency and credit accounting.
About the project
OpenComputeMesh lets a group pool the AI models they already run locally. Participants join by QR code in a browser, with no account. Contributors run a small outbound-only agent (Python, or a single Rust binary) next to their local model runtime, approve exactly which models to share, and can pause or revoke at any time. The agent never opens an inbound port and talks to the model on loopback only.
A FastAPI coordinator routes each prompt to the least busy node serving that exact model. Tokens stream back live over WebSockets, and every request gets a receipt: runtime-reported input/output tokens, time to first token, model load, prompt evaluation and generation time, plus a latency waterfall. Credits move through an append-only ledger: a worst-case hold first, then the requester pays the measured cost and the node owner earns exactly the same amount, with a consistency checker.
Every layer is pluggable. Runtimes sit behind one adapter interface (Ollama locally, plus a Google Gemini API adapter). Finished jobs stream into Tiger Data (TimescaleDB) as a hypertable with a continuous aggregate behind a live throughput chart. Sentry monitoring is opt-in, with scrubbers that strip prompts, outputs and secrets before anything leaves the machine.
It was verified end to end through a public Cloudflare tunnel from a phone. Python coordinator and agent, Rust agent, React/TypeScript app, 67 Python tests plus 9 Rust tests, and CI on every push.
Built with
AI tools
Prize Categories Entered
- Submitted
Best Use of Gemini API
A GeminiAdapter plugs into the same runtime interface as local Ollama, so a node owner can share Gemini models on the mesh with `--runtime gemini`. It streams streamGenerateContent over SSE, maps Gemini's usageMetadata to runtime-reported token counts on each receipt, and keeps the API key on the node (header only, never sent to the coordinator). Unit-tested against the documented API format; live run pending an API key.
- Submitted
Best Use of Tiger Data
Every finished inference job is written to a TimescaleDB hypertable (ocm_job_metrics) with a compression policy. A continuous aggregate (ocm_job_metrics_1m) computes per-minute throughput, token totals, and p95 time-to-first-token and latency by model, and it powers the live throughput chart in the app. Relational ledger data stays in SQLite while the high-frequency metric stream lives in Tiger Data. Verified against a real TimescaleDB with live jobs.
- Event prizeSubmitted
Best Overall
A working peer-to-peer AI inference mesh: Python coordinator, Python and Rust node agents, and a React app. Real prompts stream from local models on contributors' machines, every request gets a runtime-backed receipt (tokens, TTFT, load/eval/generation time), and credits settle in an append-only, balance-checked ledger. Verified end to end from a phone through a public tunnel; 76 tests and CI.
- Event prizeSubmitted
Sentry Raffle
Opt-in Sentry error monitoring and tracing for the coordinator with privacy-first scrubbers: request bodies, cookies and frame variables are dropped, and join codes, enrollment tokens, node credentials and API keys are redacted before any event leaves the machine, so prompts and outputs never reach Sentry. Verified through the real Sentry SDK with an in-memory transport; live delivery pending a DSN.