Hindi STT for enterprise telephony is a fundamentally different problem from studio transcription. Voice AI in India runs on phone lines, not condenser microphones. Every production call an AI voice agent handles arrives compressed, noisy, accented, and mid-way through a sentence that switches between Hindi and English. Most speech-to-text systems were never trained for that. They were trained on clean, read, 16kHz audio, and their accuracy drops sharply the moment a real customer picks up a real phone.
This is why we built our own model. Hindi STT for enterprise telephony is a different engineering problem from general-purpose transcription, and solving it requires training on the exact kind of audio that enterprise voice agents actually hear. Orato ASR Hindi v1 is a supervised fine-tune of Qwen3-ASR-0.6B, trained on approximately 1,000 hours of Hindi, English, and Hinglish calling audio, and evaluated on public Indian speech benchmarks.
Across appointment booking, conversational IVR, insurance verification, sales qualification, and customer support, accuracy on the first attempt determines whether a voice agent completes the workflow or escalates to a human. A model that mishears a policy number, a vehicle model, or a street name breaks the entire automation chain behind it.
In this article we share the architecture, the training setup, the benchmark results against both the base model and the wider Hindi STT market, and the implementation details for running the model yourself. The weights are published on Hugging Face under Apache-2.0.
Why Standard Hindi STT Fails on Indian Phone Calls
Telephony audio is a hostile input format. Calls are typically compressed to 8kHz or 16kHz PCM, which removes much of the high-frequency detail that speech recognition models rely on to distinguish similar sounds. Add background noise, room reverberation, network packet loss, and a caller speaking from a moving vehicle, and the acoustic signal degrades well beyond what most benchmark datasets represent.
Language adds a second layer of difficulty. Indian callers rarely speak one language for the duration of a call. A single sentence may carry a Hindi verb structure, English product nouns, and a regional pronunciation pattern, all at once. This code-switching, commonly called Hinglish, is where general-purpose multilingual models tend to fail hardest, because they attempt to decode the whole utterance in one language.
Regional accent variation compounds the problem further. The same Hindi sentence spoken in Bihar, Rajasthan, Maharashtra, and Delhi produces meaningfully different acoustic patterns. A model trained mostly on standardised Hindi handles the last case well and the first three poorly.
Finally, enterprise calls are dense with named entities that carry no useful language-model context: policy numbers, order references, vehicle models, drug names, and street names. These are exactly the tokens a business workflow depends on, and exactly the tokens a general model is most likely to get wrong.
Building reliable Hindi STT for enterprise telephony therefore means training on domain audio rather than tuning prompts on top of a general API.
What Is Orato ASR Hindi v1?
Orato ASR Hindi v1 (tryorato/orato-asr-hindi-v1) is a speech-to-text model built for Hindi, English, and Hinglish conversation in enterprise calling environments.
It is a supervised fine-tune of Qwen/Qwen3-ASR-0.6B, an audio-language multi-modal architecture.
The fine-tune trains the decoder and projector layers in full while keeping the audio tower frozen. That choice preserves the acoustic representations the base model already learned and concentrates the training budget on the language and alignment behaviour that our domain actually changes.
The target applications are production voice agents, conversational IVR, appointment booking, sales qualification, and insurance workflows operating over Indian phone networks.
The model card and weights are available on Hugging Face.
Training and Architecture Specifications
| Specification | Configuration |
|---|---|
| Base model | Qwen/Qwen3-ASR-0.6B (audio-language multi-modal framework) |
| Parameter distribution | Approximately 0.8B total parameters, approximately 0.6B trainable |
| Layer freezing strategy | Full supervised fine-tuning on decoder and projector; audio tower frozen |
| Training corpus | Approximately 1,000 hours of Hindi, English, and Hinglish calling audio |
| Dataset composition | Rasa Hindi, Gram Vaani, and MUCS, plus proprietary enterprise calling data |
| Optimization setup | Learning rate 1e-5, cosine schedule, warmup ratio 0.03, bfloat16 precision |
| Gradient hyperparameters | Max gradient norm 1.0, batch size tuned for low-latency loss convergence, 1 epoch |
Training Infrastructure: Azure H100 Nodes and CPU Clusters
Training a multi-modal speech model efficiently requires separating GPU work from data preparation work. If audio decoding, segmentation, and tokenization run on the same nodes as the training loop, the GPUs spend a significant share of wall-clock time waiting for batches.
Fine-tuning ran on a Microsoft Azure instance with 8 NVIDIA H100 GPUs and 96 CPU cores. We used bfloat16 precision and native scaled-dot-product attention, which together kept Tensor Core utilization high and shortened epoch time.
Data preparation ran separately. Distributed audio preprocessing, voice activity detection segmentation and tokenization were all executed across dedicated CPU clusters, so the GPU cluster stayed saturated throughout the run.
Orato holds H100 access through Microsoft for Startups, which made this scale of experiment practical at our stage.
Hindi STT Benchmark Results: Base Qwen3-ASR vs Orato ASR Hindi v1
We evaluated the model on standard held-out Indian speech benchmarks. Word error rate is the metric throughout, and lower is better.
The largest gains came on dialectal and read speech. Lahaja, which covers regional dialects and accents, improved by 25.5% relative. Kathbath, which covers read and clean speech, improved by 24.6% relative.
Gramvaani improved by only 3.6%. That dataset is rural, noisy telephony audio, and it remains the hardest set for every system we tested, including ours. We report it because it is the closest public proxy to the conditions our production workloads actually run in.
| Benchmark Dataset | Base Qwen3-ASR WER | Orato ASR WER | Relative Reduction |
|---|---|---|---|
| Kathbath (read / clean speech) | 15.24% | 11.49% | 24.6% |
| Gramvaani (rural / noisy telephony) | 39.07% | 37.66% | 3.6% |
| Lahaja (regional dialects and accents) | 25.09% | 18.68% | 25.5% |
| FLEURS (multilingual standard) | 19.12% | 16.98% | 11.2% |
How Orato Compares to the Hindi STT Market
We then compared Orato ASR Hindi v1 against major commercial speech-to-text APIs and open-source models using published results from the AI4Bharat Vistaar benchmark suite and the AI4Bharat Lahaja benchmark.
On Lahaja, the dialect and accent benchmark most relevant to Hindi ASR for enterprise telephony, the model records the lowest word error rate in the comparison set.
| System | Lahaja WER |
|---|---|
| Orato ASR Hindi v1 | 18.68 |
| IndicASR M1 (Conformer-L) | 19.40 |
| Google Chirp | 22.30 |
| Azure Speech to Text | 28.60 |
| Whisper Large-v3 | 32.40 |
| MMS (300M) | 34.40 |
On Kathbath, the read and clean speech benchmark, the model outperforms Azure Speech to Text, Google Speech to Text, and NVIDIA Conformer-M, but IndicWhisper remains ahead. IndicWhisper is a Hindi-specialised research model and clean read speech is where it is strongest, so this result is expected rather than surprising.
| System | Kathbath WER |
|---|---|
| IndicWhisper | 10.30 |
| Orato ASR Hindi v1 | 11.49 |
| Azure Speech to Text | 13.60 |
| NVIDIA Conformer-M | 14.00 |
| Google Speech to Text | 14.30 |
Full Hindi STT Benchmark Table
The table below shows every system and benchmark pair for which a published value exists. A dash indicates no published result for that combination.
| System | FLEURS | CommonVoice | IndicTTS | Kathbath | Kathbath-Hard | Gramvaani | Lahaja |
|---|---|---|---|---|---|---|---|
| Orato ASR (ours) | 16.98 | 20.63 | 13.23 | 11.49 | 13.21 | 37.66 | 18.68 |
| Azure STT | 24.30 | 14.60 | 15.2o% | 13.60 | 15.10 | 42.30 | 28.60 |
| Deepgram | 17.14 | 20.04 | 13.65 | — | — | — | — |
| ElevenLabs Scribe | 11.99 | 16.59 | 16.06 | — | — | — | — |
| Google Chirp | — | — | — | — | — | — | 22.30 |
| Google STT | 19.40 | 20.80 | 18.30 | 14.30 | 16.70 | 59.90 | — |
| IndicASR M1 (Conformer-L) | — | — | — | — | — | — | 19.40 |
| IndicWav2Vec | — | — | — | — | 16.20 | 42.10 | — |
| IndicWhisper | 11.40 | 15.00 | 7.60 | 10.30 | 12.00 | 26.80 | — |
| MMS (300M) | — | — | — | — | — | — | 34.40 |
| NVIDIA Conformer-M | — | — | — | 14.00 | 15.60 | 41.30 | — |
| Sarvam Saarika | 16.00 | 18.21 | 15.37 | — | — | — | — |
| Whisper Large-v3 (1.55B) | — | — | — | — | — | — | 32.40 |
Sources and methodology: metrics compiled from the AI4Bharat Vistaar benchmark suite, the AI4Bharat Lahaja benchmark, and third-party published word error rates. Results for other systems are published figures rather than our own re-runs, and text normalization rules vary between sources. Gaps of under roughly one point should be treated as directional rather than decisive.
Running the Model
The checkpoint is loaded through the custom qwen_asr wrapper rather than the standard transformers loader.
import qwen_asr
import torch
# Initialize wrapper and load model weights in bfloat16
wrapper = qwen_asr.Qwen3ASRModel.from_pretrained(
"tryorato/orato-asr-hindi-v1",
dtype=torch.bfloat16,
device_map=None,
attn_implementation="sdpa",
)
wrapper.model = wrapper.model.to("cuda")
# Transcribe a 16kHz audio array
result = wrapper.transcribe(
audio=(wav, 16000),
language="Hindi",
)Integration note: weights must be initialized via qwen_asr.Qwen3ASRModel. Loading the checkpoint directly through transformers.AutoModel bypasses the custom decoding layers and raises a runtime error.
Data Rights, Governance, and Licensing
The training corpus combines open-source research datasets, including Rasa Hindi, Gram Vaani, and MUCS, used under their respective public licenses, with proprietary consent-verified calling audio collected by Orato. PII review for the proprietary calling data, are in progress.
No raw audio and no personally identifiable information is included in the published model repositories. Model weights are released under the Apache-2.0 license for internal and beta evaluation.
For businesses evaluating deployment, this matters as much as accuracy. Enterprise buyers in regulated sectors need to know where training data came from, what leaves their environment, and whether the model can run inside their own infrastructure.
Why Hindi STT Accuracy Matters for Enterprise Voice Deployments
Word error rate is not an abstract research metric in a voice agent. It is the input to every downstream step. A transcription error on a policy number sends the wrong record to the CRM. An error on an appointment time books the wrong slot. An error on an intent phrase routes the caller to the wrong workflow.
Reducing word error rate by roughly a quarter on dialectal speech changes how many calls complete without human intervention, which is the number that determines whether voice automation is worth deploying at all.
Owning the model rather than calling a general API also means we can tune for a specific customer vocabulary, run inside a customer’s own environment where compliance requires it, and control latency in the streaming path instead of inheriting whatever an external provider offers.
Why Choose Orato?
Orato builds AI voice agents for enterprise calling, with the speech models developed in-house rather than licensed from a third party.
That means the automatic speech recognition and text-to-speech layers can be tuned to the language mix, accent range, and vocabulary of each deployment, instead of forcing a general-purpose model to handle audio it was never trained for.
For teams evaluating Hindi ASR for enterprise telephony, we will benchmark Orato ASR against your current provider on your own call recordings, so the comparison reflects your audio conditions rather than a public dataset.
If you are exploring AI calling more broadly, these guides may also be useful:
- WhatsApp AI Calling for Indian SMEs
- AI Calling Compliance in India: What Businesses Need to Know
- How AI Agents Are Replacing BPO Calls in India
- AI Calling for Real Estate Lead Follow-Up in India
Conclusion
General-purpose speech recognition was not built for Indian phone calls. Compressed audio, regional accents, and constant Hindi-English code-switching push word error rates to levels that break automated workflows.
Orato ASR Hindi v1 addresses that by fine-tuning Qwen3-ASR-0.6B on approximately 1,000 hours of domain calling audio, reaching 11.49% word error rate on Kathbath and 18.68% on Lahaja, ahead of Azure Speech to Text and Google Speech-to -Text/Google Chirp on both.
Work continues. Gramvaani remains difficult for every system in this comparison, including ours, and rural noisy telephony is where the next version is focused. We publish the numbers as they are, including the ones that are not flattering, because that is the only way benchmark reporting is useful to anyone evaluating a model.
The weights are on Hugging Face. If you run Hindi or Hinglish call volume and want to test the model on your own audio, get in touch.
Frequently Asked Questions
What is Orato ASR Hindi v1?
Orato ASR Hindi v1 is a speech-to-text model based on Qwen3-ASR-0.6B and fine-tuned for Hindi, English, and Hinglish speech, built specifically for enterprise calling and telephony environments.
What word error rate does Orato ASR achieve?
The model reached 11.49% word error rate on Kathbath, 13.21% on Kathbath-Hard, 18.68% on Lahaja, 16.98% on FLEURS, 20.63% on CommonVoice, 13.23% on IndicTTS, and 37.66% on Gramvaani.
How much audio was used for training?
Approximately 1,000 hours of domain-specific Hindi, English, and Hinglish calling audio, drawn from public research datasets and proprietary consent-verified enterprise call recordings.
Why is Hindi ASR for enterprise telephony harder than standard transcription?
Telephony audio is compressed to 8kHz or 16kHz, carries background noise and reverberation, and contains constant Hindi-English code-switching alongside wide regional accent variation. Each factor raises word error rate for models trained on clean read speech.
How does Orato ASR compare to Azure, Google, and Whisper?
On the Lahaja dialect benchmark, Orato ASR records 18.68% word error rate against 22.30% for Google Chirp, 28.60% for Azure Speech to Text, and 32.40% for Whisper Large-v3. On Kathbath, IndicWhisper remains ahead at 10.30% versus 11.49%.
How do I load the model?
Initialize the checkpoint using qwen_asr.Qwen3ASRModel.from_pretrained("tryorato/orato-asr-hindi-v1"). Loading it through transformers.AutoModel bypasses the required custom decoding layers.
Is the model available for commercial use?
The weights are published under Apache-2.0 for internal and beta evaluation. For production telephony deployments, including on-premise and sovereign hosting, contact the Orato team.