We’re team cooked.exe, and we picked Problem 3B: pulling the company, date, address and total out of a receipt photo and returning them as JSON.

We started with Gemma 4 E2B (the 4-bit version, so it fits on a free Colab T4) and fine-tuned it with LoRA on 240 receipts from the SROIE dataset. Training took about 6 minutes. Then we tested it on 150 receipts it had never seen, and ran the plain model on the same 150 so the comparison was fair. We also tried the plain model with a long, detailed prompt, because we wanted to know whether better prompting alone would do the same job.

What we found

Exact-match accuracy, plain model vs fine-tuned:

• company: 76.0% to 82.7%
• date: 88.0% to 94.0%
• address: 27.3% to 54.7%
• total: 70.7% to 80.0%

The detailed prompt barely helped. It stayed within a few points of the plain model on every field, and for date it was slightly worse.

The address jump looks huge, but it’s partly an illusion. The plain model was often reading the address correctly and just writing it in a different style from the dataset’s labels, like missing commas. When we scored with a looser fuzzy match, the address improvement dropped from +27 points to +16. We think that’s the honest number, so we report both.

Where it still fails

Total is our weakest field. About 1 in 5 totals is wrong, and most of those are genuinely wrong values, not formatting. We haven’t fully figured out why yet. We also only tested on one dataset of Malaysian receipts, so we can’t say how it does on other receipt styles or on phone photos.

One thing that went wrong along the way

Our first attempt crashed Colab by running out of RAM, because we loaded every receipt as a full-resolution image at once. Resizing the images to 1024 px fixed it, and memory use stayed around 3.4 GB RAM and 8.4 GB VRAM after that.

The demo

We built a small Gradio app that runs the plain and fine-tuned models side by side on whatever receipt you upload, so you can see the difference yourself. It takes a few seconds per receipt. It runs from a Colab session, so it isn’t a hosted product.

The repo has the notebook, the per-receipt results as CSVs, the app code and a README with the full numbers and error breakdown.