Open Model Evaluation – HacktoberFest HackDay

This project evaluates and compares multiple open AI models using a standardized set of prompts across different categories, including reasoning, factual accuracy, instruction following, coding, Indian languages, and summarization.

The goal is to identify each model’s strengths, weaknesses, and real-world suitability through a structured scoring rubric based on correctness, instruction following, clarity, and honesty. The evaluation includes overall and category-wise performance comparisons, failure analysis, and practical recommendations for use cases such as student help desks, coding assistants, and Indian-language applications.

The project also highlights testing limitations and the importance of fair evaluation, reproducibility, and evidence-based model selection.