CodeMixBridge
About the project
CodeMixBridge is a Python-based machine learning project that improves English-to-Kannada language conversion by fine-tuning Google’s Gemma 4 E2B instruction model. It supports three tasks: English to Romanized Kannada, English to Kannada-script code-mixed text, and English to formal native Kannada. The project uses the AI4Bharat IndicCMix Kannada dataset, containing English sentences with corresponding Romanized, code-mixed, and native-script outputs. Data is normalized, deduplicated, grouped by source, and divided into deterministic training, validation, and testing sets without leakage. Each source record is expanded into three task-specific examples. The model is loaded in pre-quantized 4-bit format using Unsloth and bitsandbytes, allowing training on a Google Colab Tesla T4 GPU. LoRA parameter-efficient fine-tuning updates small adapter matrices while keeping the original Gemma weights frozen. PyTorch performs GPU computation, Hugging Face Transformers handles tokenization and generation, TRL provides the supervised fine-tuning trainer, and PEFT manages LoRA adapters. Training uses response-only loss, gradient accumulation, an 8-bit AdamW optimizer, and a cosine learning-rate schedule. Generic Gemma and fine-tuned Gemma are evaluated on identical held-out examples using deterministic generation and SacreBLEU chrF scores. Pandas organizes results, while Gradio provides an interactive interface for comparing both models’ Kannada outputs side by side.
Built with
AI tools
Prize Categories Entered
- Event prizeSubmitted
Theme 3: Fine-Tuning Gemma