Author(s): Latent AI Labs

Originally published on Towards AI.

Nandi Carries Every Complaint to the Right Desk: Open-Source AI for Indian Languages

A free model that sorts complaints in 13 Indian languages into 53 categories based on RBI, IRDAI and CPGRAMS lists. What a synthetic test hid, and what real posts showed.

By Arun Prabhakaran · @LatentAILabs

TL;DR

• Nandi (arunp0/nandi-grievance-indic) is a free, open-source model that reads a complaint in Hindi, Hinglish, 10 other Indian languages or English and returns one of 53 categories, the top 3 with probabilities, and an urgency level (P1/P2/P3).
• The categories follow the complaint lists of India’s regulators and grievance systems: RBI for banking, IRDAI for insurance, CPGRAMS for central government services, telecom, and the National Consumer Helpline.
• On 443 real public posts it had never seen, it picks the right category 62.5% of the time and has it in its top 3 81.3% of the time. A 35B-parameter LLM (Qwen3.6, zero-shot) is more accurate at 66.4%, and much better on rare categories.
• On its held-out synthetic test it scored 92.9%, beating the 35B LLM. Why it dropped 30 points on real text is the core lesson of this post.
• Urgency failed its pre-registered rule on real text, so it ships marked experimental.
• Apache-2.0, 560M parameters, runs on your own CPU or GPU, sends nothing to an API.

Parts of this article were drafted with AI assistants; I checked every number against the model card and the evaluation logs.

The routing problem: 13 languages, 53 desks

RBI’s ombudsman scheme received 13.34 lakh complaints in 2024–25. The central government’s grievance portal, CPGRAMS, redressed over 26 lakh in 2024. Behind those official numbers are many more complaints that never reach a regulator: app reviews, tweets, emails and support chats sent to banks, insurers, telecom companies and government offices.

Every one of those messages has to land on someone’s desk first. Here is a typical one (a made-up example):

“UPI se 2,000 rupaye kat gaye par receiver ko nahi mile, aaj subah ki baat hai. Bank helpline koi jawab nahi de raha.” (₹2,000 left my account by UPI this morning but never reached the receiver. The bank helpline isn’t answering.)

Before a support agent can act, triage has to answer three questions: which desk owns this (digital payments, not fraud or loans), how urgent it is (money lost today), and what the fallback categories are if the first guess is wrong.

In Shiva temples, people whisper their wishes into the ear of Nandi, the bull who keeps Shiva’s gate. Nandi does not grant the wish; he makes sure it reaches the right place. That is Nandi’s entire scope: it does not resolve complaints or judge whether they are valid; it routes each message to the right desk.

Nandi is the third model in a small open set for Indian languages. Yama is a zero-shot classifier: it judges text against labels you write at run time. Chitragupta keeps the records: it finds and masks personal data. Nandi exists because on routing into a fixed set of categories, a fine-tuned 560M encoder beats a zero-shot generalist of the same size by 25.9 points on real text.

What Nandi returns

# pip install torch transformers safetensors huggingface_hub
import sys
from huggingface_hub import snapshot_download
path = snapshot_download("arunp0/nandi-grievance-indic")
sys.path.insert(0, path)
from router import ComplaintRouter
r = ComplaintRouter(path) # uses a GPU if available, otherwise CPU
r.route("UPI se 2,000 rupaye kat gaye par receiver ko nahi mile, aaj subah ki baat hai. "
 "Bank helpline koi jawab nahi de raha.")
# {'category': 'bank.digital_banking', 'domain': 'bank', 'confidence': 0.9999, 'urgency': 'P1',
# 'urgency_probs': {'P1': 0.868, 'P2': 0.0012, 'P3': 0.1307},
# 'top3': [('bank.digital_banking', 0.9999), ('bank.fraud_unauthorised_txn', 0.0), ('con.ecommerce', 0.0)]}
r.route("Thank you SBI for the quick help at the Andheri branch today!")
# {'category': 'none.out_of_scope', 'domain': 'none', ..., 'urgency': 'P3', ...}

route_batch(texts) handles lists. Under the hood, Nandi is an XLM-RoBERTa-large encoder with mean pooling and two linear heads: a 53-way category head and a 3-way urgency head.

One warning belongs right next to the code: the confidence number is not calibrated. On real text Nandi often outputs 0.99 on wrong predictions. Treat top3 as suggestions for a human, never as a threshold for fully automatic routing.

53 categories, borrowed from RBI, IRDAI, CPGRAMS and the NCH

I did not want to invent a taxonomy. Indian regulators already publish what people complain about, so the categories follow their lists:

• Banking and payments (14): the complaint form of RBI’s Integrated Ombudsman Scheme: ATM and debit cards, digital banking, credit cards, loans, deposit accounts, KYC and frozen accounts, and so on. Some categories are split out because routing or urgency differs, such as fraud and unauthorised debits, and recovery-agent harassment.
• Insurance (12): IRDAI’s complaint types: death claims, health claims, motor claims, policy servicing, mis-selling, ULIPs and others.
• Central government services (17): published CPGRAMS categories: PF and EPFO, pensions, PM-KISAN, income tax and PAN, Aadhaar, passports, post, railways, LPG, corruption by officials and more.
• Telecom (4) and consumer complaints (5), the latter mostly from the National Consumer Helpline’s sectors: e-commerce, electronics, travel, coaching and others.
• Not a complaint (1): praise, questions, suggestions, news, opinions and chit-chat.

Each category records where it came from: 23 are verbatim items on an official complaint form, 17 are CPGRAMS categories, 4 are National Consumer Helpline sectors, 4 are derived from official buckets, and 5 (railways, ration cards, two telecom groups, coaching) have no published list, so I defined them. The category names belong to the regulators; the mapping is mine, not an official one.

A few boundary rules matter more than they look:

• Fraud beats the payment method. A fraudulent UPI debit goes to fraud, not to digital banking.
• A review criticising an app counts as a complaint, even if it asks for nothing.
• Reports of a crime by someone else are not complaints here. None of these grievance systems handles crime, and there is no police category. Bribery or misconduct by an official is a complaint.
• If a message has several problems, the most time-critical one wins.

Training data: 116,199 complaints that never happened

No openly licensed dataset of labelled Indian complaints exists, and I had a hard rule against training on real complaints, production data or customer data. So every training text is synthetic, written and filtered only by open-weights models (Gemma 4 and Qwen3.6, both Apache-2.0):

1. Code chose the scenario. For the first rounds, code picked the category, the timing facts (when the money left, whether a deadline had passed) and the history, then asked Gemma 4 (Apache-2.0) to write the message in a given language, channel and persona: tweets, app reviews, emails, chat messages, ombudsman forms, formal letters, helpline call transcripts.
2. A second model labelled it blind. Qwen3.6 (Apache-2.0) read each text without seeing the scenario. Where it disagreed with the scenario’s category, the text was dropped.
3. Later rounds wrote freer text that looked more like real reviews and tweets, and two teachers (Gemma 4 31B and Qwen3.6) labelled it. Only texts where both agreed were kept.

The result: 116,199 texts in 13 languages, of which 15,902 are non-complaints and 24,638 are urgent (P1). Every name, account, phone and policy number is a dummy value. No output of a proprietary model is in the training data.

The synthetic test said 93%

I wrote down the ship rules before generating any data: on a synthetic test, macro-F1 at least 0.10 above Yama, urgent recall at least 0.90, and no more than 10% of non-complaints routed to a category.

To keep the test honest, it was written by a different model (Qwen3.6, not Gemma), with channels and personas held out from training. Nandi passed easily:

• Category accuracy 0.929, macro-F1 0.909 (Qwen3.6 zero-shot 0.888, Yama 0.536, TF-IDF with logistic regression 0.604).
• Top-3 accuracy 0.971.
• Urgent recall 0.955 on the rows a blind checker confirmed as urgent.
• 7.3% of non-complaints routed to a category.
• Every language scored between 0.87 (Urdu) and 0.98 (English).

Even with the test written by a different model family and unseen personas, the 560M specialist beat the 35B LLM. Stopping here would have hidden a 30-point drop on real text.

Real posts said 63%

Even when a different LLM writes it, a synthetic test shares the definitions, prompt structure and tidy detail of the training data. So the real decision was made on real text, scored once.

I built a blind set of 443 real public posts: Play Store and App Store reviews from India and posts from a public forum, collected only after the model was frozen. 38% are not in English, mostly Hindi and Hinglish. Apps and companies from my two earlier real sets were excluded. Three independent LLM labellers (Claude agents, not humans) labelled each post from the text alone, using the same written definitions the training teachers used, and a 2-of-3 vote decided. I wrote the evaluation plan and the pass rules before collecting a single post, and scored the set once.

Two cautions about those labels. They are LLM-reviewed, not human-verified. And all three labellers are the same model, so their 91% agreement measures one model’s consistency, not true agreement between independent annotators, and overstates how reliable the labels are.

Every model drops on real text, and Nandi drops the most (30 points).

On real posts the 35B LLM leads on accuracy and, by a lot, on rare categories.

What the real numbers say:

• Nandi picks the right category about 6 times in 10, and has it in the top 3 about 8 times in 10. That is usable for suggesting a desk to a human. It is not good enough to route without one.
• The accuracy gap to Qwen (3.9 points) is not statistically clear on 443 posts (McNemar p ≈ 0.15). The macro-F1 gap (0.43 vs 0.62) is real: Nandi is weak on rare categories.
• About 1 in 5 real non-complaints still get a category, almost exactly as many as Qwen routes.
• Against Yama, the generalist of the same size, Nandi wins clearly (0.625 vs 0.366 accuracy). That margin is why Nandi exists.

Real complaints are shorter, vaguer and messier than anything an LLM writes when you ask it for a complaint. An imagined one-line review such as app is fraud, money gone carries less signal than three tidy sentences about a failed UPI transfer.

A correction, disclosed. After scoring, I found a mistake in my own labelling guide. It told the labellers to put recovery-agent threats under “loans” instead of the taxonomy’s own recovery-harassment category, so models that picked the correct category were marked wrong. On those 35 posts I now accept either label. Before the correction Nandi scored 0.571 accuracy and 0.397 macro-F1 (Qwen 0.607 and 0.593, Yama 0.352 and 0.256). Urgency numbers did not change, and neither did any pass-or-fail decision.

Urgency failed its rule, and that is in the model card

Before collecting the real set, I fixed the bar for calling urgency reliable: the lower end of the 95% interval for urgent (P1) recall had to reach 0.50.

While the set was being labelled, a review of the plan pointed out that about 40% of its urgent posts were loan-app harassment, which might be easy to spot. So before scoring, I added a second condition: the bar also had to hold without the loan-app posts.

The pre-registered bar is the dashed line. Nandi’s intervals sit well to its left.

• Nandi catches about 1 in 4 real urgent posts, and only 1 in 10 once loan-app harassment is left out. The 35B LLM does no better.
• Yama catches more because it calls far more posts urgent. Its precision is 0.32, against 0.70 for Nandi.
• The synthetic test had said 0.955. Real urgent posts rarely spell out their timing the way my training scenarios did (“debited this morning”, “agent threatened me yesterday”).

So the urgency head ships, but the model card calls it experimental: a hint for a human, not a triage decision. That split was the most useful step in the whole evaluation. Without it, a 0.43 recall on the easier loan-app slice would have hidden a 0.10 recall on every other kind of urgent complaint.

Fixing one error broke another: false alarms vs missed complaints

Before freezing the model, I used a public set of tweets as a development set: the Mendeley Enhanced Dataset of Citizen Centric Complaints and Grievances on Twitter (Agarwal, Mittal and Sureka, IIIT-Delhi, CC BY 4.0). It has English tweets to Indian Railways, the Income Tax Department, Delhi Police and Delhi Traffic Police. I relabelled 2,249 of them with this taxonomy and masked the official @handles, which would otherwise give the category away.

The shipped model, v5b, routed 40% of the non-complaint tweets (news, live traffic updates, opinions) to a category. That looked fixable. I generated about 5,600 new look-alike texts, such as traffic updates next to synthetic traffic complaints, and retrained twice.

Both retrained versions cut false alarms, and both started rejecting real complaints.

• v5c cut non-complaints routed from 40% to 26%, but called 59% of real traffic complaints “not a complaint”, up from 10%. The wording of my labelling prompt led the teacher models to call many of the new traffic complaints “not a complaint”, and 844 texts they disagreed on were dropped.
• v5d clarified the labelling prompt. It recovered some of the traffic complaints but still lost a third of them, and failed four of its five pre-set checks, including accuracy on my earlier real set.

I kept v5b and closed that line of work. A model that sometimes routes a news tweet costs an agent a few seconds. A model that drops a real complaint costs the person who wrote it. The model card now lists “about 1 in 5 non-complaints routed” as a known limit, not a solved problem.

How to use it

Mask, route, then a human decides. All three steps run locally.

Nandi works best as a suggestion layer: show the agent the top 3 categories, let them confirm with one click, and log the disagreements. If the text holds personal data, run Chitragupta’s redact() first so far less of it reaches your logs. Nandi's accuracy on redacted text has not been measured, so check the chained pipeline on a sample of your own first.

Where it fits:

• Sorting app reviews, tweets and support emails into teams.
• Pre-filling a category on a complaint form, with the top 3 as choices.
• A small, local first pass before an expensive LLM, or instead of one where data cannot leave your servers.

Where it does not fit:

• Deciding on its own what gets handled first. Urgency is experimental.
• Throwing away messages it calls “not a complaint” without a look. Some real complaints land there, and about 1 in 5 real non-complaints go the other way.
• Rare categories. They are its weakest area.

Known limitations

• Synthetic training data. Accuracy drops from 0.93 on the synthetic test to about 0.63 on real posts.
• Urgency is experimental on real text: it catches about 1 in 4 urgent posts.
• Non-complaints: about 1 in 5 real ones get a category; on public tweets it was 2 in 5.
• Rare categories are weak on real text (macro-F1 0.43).
• Confidence is not calibrated.
• Real-text evaluation is mostly English, Hindi and Hinglish. Other languages are tested only on synthetic text.
• Every real-text label is LLM-reviewed, not human-verified.
• Category lists and turnaround times are as of 2026. Not legal advice.

Three lessons for training small classifiers on synthetic data

1. A synthetic test measures your generator, not your users. Nandi beat a 35B LLM on synthetic text and trailed it on real posts. If you train on LLM-written data, budget for a real, blind test before you believe any number.
2. Write the rules before you see the results, and split the slices people will ask about. The loan-app split turned a misleading 0.43 into an honest 0.10.
3. A fix that moves one error into another is not a fix. Two retraining rounds traded false alarms for missed complaints. Shipping the version with the right kind of error was the better call.

FAQ

What is Nandi? Nandi (arunp0/nandi-grievance-indic) is an open-source complaint classification model for Indian languages and Hinglish. It reads a complaint and returns one of 53 categories based on RBI, IRDAI, CPGRAMS and National Consumer Helpline lists, the top 3 categories with probabilities, and an experimental urgency level.

Which languages does it support? Hindi, Hinglish, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada, Malayalam, Punjabi, Urdu, Odia and Indian English, in native scripts and in Roman letters.

How accurate is it on real complaints? On 443 real public posts labelled blind by three LLM labellers, it picked the right category 62.5% of the time and had it in the top 3 81.3% of the time.

Is it better than using an LLM? Not on accuracy. Qwen3.6–35B-A3B zero-shot scored 66.4% on the same posts and is much better on rare categories. Nandi is about 60 times smaller, runs locally on a CPU, and sends no text to an API.

Can it detect urgent complaints? Only as a hint. It catches about 1 in 4 real urgent posts, so it failed its own reliability rule and is marked experimental.

Is it free for commercial use? Yes. The weights and code are Apache-2.0, the base model (XLM-RoBERTa-large) is MIT, and the training data was generated only with open-weights models (Gemma 4 and Qwen3.6, both Apache-2.0).

Was it trained on real complaints? No. All training text is synthetic. Real posts were used only for evaluation, and they are not released.

Try them

• Nandi (complaint routing): huggingface.co/arunp0/nandi-grievance-indic
• Chitragupta (PII detection and redaction): huggingface.co/arunp0/chitragupta-pii-indic
• Yama (zero-shot classification): huggingface.co/arunp0/yama-indic
• How Yama was built: Before You Invent a New Architecture, Try a Bigger Encoder
• How Chitragupta was built: Yama Judges, Chitragupta Keeps the Records

If you run Nandi on your own complaints and it gets them wrong, open a discussion on the model page. Real failure cases are the most useful thing anyone can send me.

Nandi, Chitragupta and Yama are independent personal releases by Arun Prabhakaran, licensed Apache-2.0. The development tweets come from Agarwal, Mittal and Sureka (IIIT-Delhi), Enhanced Dataset of Citizen Centric Complaints and Grievances on Twitter, Mendeley Data (CC BY 4.0). Complaint volumes: RBI, Annual Report of the Ombudsman Scheme 2024–25; DARPG, CPGRAMS 2024.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI