PromptDuel helps you choose the right open-weight model for your own task. It runs two models on your prompts, has a judge model pick the better answer (A/B order randomised to reduce bias), and writes a shareable HTML report with answers, winners, latency and tokens/sec. It is one Python file using only the standard library, so there is nothing to install. Open-weight Gemma 4 models (31B and 26B-A4B, via the Gemini API) are the contestants and the judge, and local models through Ollama also work. It also ships as an agent skill (SKILL.md, Agent Skills open standard) so an AI agent can run model comparisons for you. Known limitation: LLM judging is a rough signal, and a judge that is also a contestant may favour itself, so a different judge can be chosen with –judge.