AGENTANYWHERE VAANI · INDIC VOICE
Vaani
वाणीThe caller speaks. The recording stays home.
AgentAnywhere Vaani is our Indic voice stack. A voice note or a call comes in — Hindi, Hinglish, Urdu, Tamil, Telugu, Marathi or English — and is understood, masked, classified and answered. It is designed to run on Indian infrastructure, on ordinary CPUs, and the recognition stage has been measured on a cloud machine in Central India. A classifier trained on your categories is built for each deployment. Nothing is kept but an anonymised tally. The first pilot deployment is being built now.
- Languages
- Seven to start — English, Hinglish, Hindi, Urdu, Tamil, Telugu and Marathi.
- Runs
- Designed to run server-side on Indian infrastructure, CPU at runtime. No GPU; region-pinned in India, on the provider you choose.
- Retains
- Nothing but an anonymised tally — no audio, no transcript, no identity.
- Accuracy
- Being measured on a frozen evaluation set and not yet published.
In India, the sovereign problem is spoken.
Most of India's conversations with its institutions are not typed. They are said — into a helpline, a bank's WhatsApp number, a clinic's phone, a field officer's handset — in a mix of languages that changes mid-sentence. A recording of that conversation is the most sensitive artefact the institution holds: a voice, a name, an account number, a fear.
The default way to make that recording useful is to send it to a foreign transcription API. That is a data-export event, whatever the contract says. AgentAnywhere Vaani is built so the audio never leaves Indian infrastructure, and so the pipeline keeps nothing it does not need — the transcript is masked before any model reasons over it, and the only thing left afterwards is a count.
What happens to every voice note.
Six steps, in the order they run. Each is either ours, or labelled as what it is.
Language
The caller or the worker chooses the language; a check confirms it. Hinglish is treated as its own case, never as Hindi with English words in it.
Recognition — interim
Speech becomes text, in a stage designed to run on Indian infrastructure. Today this uses an open-licensed external recogniser as an interim component. Our own recogniser replaces it, language by language, when it beats the interim on held-out audio — a rule, not a date.
Masking — Aavaran
Numbers, emails, ID numbers and recognised names and places are masked before classification.
Classify and route
A classifier trained on your categories, built for each deployment, is designed to assign category, urgency and confidence with constrained decoding: a label from your taxonomy, never free text. Low confidence, urgency or any safety flag goes to a person.
Reply
A reply approved by you — text plus a pre-rendered voice note. Urdu replies are voiced in Hindi for now. The copy is customer-owned; the synthesis is an external open-licensed component, labelled as such.
Tally
What remains is an anonymised tally: counters only, with small numbers suppressed. Sessions hold no audio and no transcript.
What is ours, and what is not — said on the page.
The sovereign claim covers the parts that are actually ours. The running system reports the same thing in its own descriptors, so the page and the software cannot disagree.
Ours
The masking (Aavaran), the classification and routing (a classifier trained on your categories, built for each deployment), the taxonomy, the reply templates, the safety logic, the anonymised tally, the design for deployment on Indian infrastructure, and the frozen evaluation set every number will be measured on.
External, interim, labelled
Speech recognition and speech synthesis currently use open-licensed external models. They are described that way everywhere, including inside the running system. Each is replaced by our own model on a written switchover rule — when ours beats the interim on the held-out audio for that language.
Until a language has been measured on the frozen set, no accuracy figure for it appears anywhere. Hinglish is measured explicitly, never inferred from the Hindi number.
Where AgentAnywhere Vaani goes to work.
The stack is general — a voice note in, a routed answer out — and built to be reused. The first pilot is being built; the other uses below are illustrations of where it could be used.
Helplines and CSR programmes
The first deployment, being built now: a WhatsApp voice-note helpline where callers describe a problem in their own language, are answered with approved copy, and are handed to a caseworker when it matters.
PSU banks and banking correspondents
Voice-note requests from customers and correspondents in the field — masked, categorised and routed to the right desk without a transcript ever being stored.
Contact centres and BPOs
Voice notes and IVR fall-through classified before an agent picks up, with nothing retained that a client's data-residency clause would object to.
Insurers
Claims intimation by voice in the language the claimant actually speaks, with health and financial identifiers masked before triage.
Agriculture and rural services
Advisory requests from a farmer's own phone, in their own language — the intake side of what Tatva Kisan is being built to answer.
Health programmes
Appointment and symptom triage lines where the recording must never leave the programme's infrastructure and identifiers must never reach a model.
What we are not claiming yet.
We do not publish an accuracy figure for any language until it has been measured on the frozen evaluation set, and we do not infer Hinglish from Hindi. We do not claim the speech recognition is ours — it is not yet, and the page says so. We do not run the model on the caller's phone; AgentAnywhere Vaani is server-side, designed for Indian infrastructure. And we do not claim a certification of your deployment: ShepHertz operates a control environment credentialed for SOC 2 and ISO 27001 and independently assessed for HIPAA and GDPR, and our alignment to the DPDP Act is designed-to-support, there to inform your own assessment.
Work with us on AgentAnywhere Vaani.
The pipeline runs end to end with fixtures and with the interim components; the first pilot deployment — a voice-note helpline — is being built, and the evaluation set it will be measured on is frozen before the pipeline is tuned, so the number cannot be built to flatter what is already running.
We work with teams that have a real voice workload in an Indian language. If your callers speak rather than type, and the recording cannot leave the country, talk to us.
FAQ
Frequently asked questions.
- What is AgentAnywhere Vaani?
- AgentAnywhere Vaani is ShepHertz's Indic voice stack: a server-side pipeline designed to take a voice note or call in an Indian language, recognise it on Indian infrastructure, mask numbers, emails, ID numbers and recognised names and places with Aavaran, classify and route it with a classifier trained on your categories, answer with approved text and a pre-rendered voice note, and retain nothing but an anonymised tally. The first pilot deployment is being built.
- Which languages does AgentAnywhere Vaani support?
- Seven to start: English, Hinglish, Hindi, Urdu, Tamil, Telugu and Marathi. Hinglish is treated and measured as its own language rather than inferred from Hindi. Accuracy per language is being measured on a frozen evaluation set and is not yet published.
- Does the audio leave India, and what is retained?
- It is designed not to. Every step is designed to run on Indian infrastructure on ordinary CPUs, region-pinned in India on the provider you choose; the design sends no audio to a foreign API. Sessions hold no audio and no transcript, and numbers, emails, ID numbers and recognised names and places are masked before classification. What remains afterwards is an anonymised tally of counters only.
- Is the speech recognition ShepHertz's own model?
- Not yet, and the page says so. Speech recognition and speech synthesis currently use open-licensed external models as interim components. Our own recogniser replaces the interim language by language when it beats it on held-out audio. The masking, taxonomy, templates, tally and evaluation set are ours, and the classifier is trained on your categories for each deployment.
- Can AgentAnywhere Vaani work with our WhatsApp number or contact centre?
- Yes. The first deployment is a WhatsApp Business voice-note helpline, and the same pipeline accepts voice notes and IVR fall-through from a contact centre. The reply copy and the category taxonomy are yours; low-confidence, urgent or safety-flagged notes are routed to a human you designate.
Bring us a voice workload that cannot leave the country.
A helpline, a WhatsApp number, a claims line — in the language your callers actually speak. We will run it through AgentAnywhere Vaani on Indian infrastructure and show you what is kept: a tally, and nothing else.