Open Model Evaluation

About the Project

This project compares four open-weight AI models using a consistent set of prompts and a structured evaluation rubric.

Developed for Hacktoberfest 2026 Hack Day at SSIT, Tumakuru.

Models Evaluated

• GPT-OSS-20B — Groq Playground
• GPT-OSS-120B — Groq Playground
• Gemma-4-31B-IT — Google AI Studio
• Gemma-4-26B-A4B-IT — Google AI Studio

Evaluation Criteria

Each model response is scored from 1 to 5 on:

1. Correctness
2. Instruction Following
3. Clarity
4. Honesty

Maximum score per response: 20 points.

Evaluation Categories

• Math & Reasoning
• Factual Accuracy
• Python Coding
• Instruction Following
• Indian Languages

Methodology

• Use at least 25 prompts across five or more categories.
• Give every model the same prompts and system instructions.
• Start a fresh conversation for each prompt.
• Keep temperature consistent wherever supported.
• Use at least two independent reviewers to evaluate responses.

Results

Results will be added after the evaluation is completed and reviewed.

Repository Contents

Evaluation prompts, model responses, scoring data, and analysis will be added as the project progresses.

Event

Hacktoberfest 2026 Hack Day
SSIT, Tumakuru, Karnataka, India.