Agaram 2.5B
A Tamil–English Language Model Trained from Scratch
arXiv 2610.18420 · open weights (Apache-2.0)
Read on arXiv →What It Is
A 2.5B-parameter Tamil–English language model trained from random weights at Murai Labs. It chats in Tamil script, in romanized Tamil (Tanglish) and in English, and it can call tools. The base model, the chat model, GGUF files and the code were released on 27 October 2026 under Apache-2.0. It is a research model and weak on knowledge: on Tamil MMLU and reading comprehension it scores near chance.
The Problem
Open language models serve Tamil poorly, and part of the reason is the tokenizer. Qwen2.5's tokenizer splits an average Tamil word into 9.4 tokens and an English word into 1.35, so Tamil text costs about seven times as many tokens. Adding Tamil tokens to an existing model leaves them untrained; training from scratch lets the tokenizer be built for Tamil first.
Approach
- A 64,000-token byte-level BPE trained on a sample that was 58% Tamil: 1.55 tokens per Tamil word, vowel signs kept with their letters, line breaks and indentation preserved
- A dense decoder-only Transformer, chosen after hybrid Mamba variants did not beat it in three-seed tests at about 91M parameters
- One pretraining run on 50B tokens (58% Tamil, plus English, math, code and parallel text) on a rented 8×H100 node: about six days and USD 4,765 of compute
- Chat fine-tuning in one run from the base on 222,662 records; about a fifth of the supervised tokens are translation
- Training data scanned against 20 evaluation sets before any score was reported; 42,001 overlapping documents removed
Results
- Better Tamil–English translation than Sarvam-1 in all six directions measured (FLORES chrF: English→Tamil 56.9 vs 49.2, Tamil→English 55.4 vs 44.2)
- On 25 tool prompts: the right tool every time, exactly the expected arguments 16 times (the schemas also appear in its training data)
- An LLM judge rated 25.3 of 50 replies to new everyday prompts usable (mean of three passes)