Skip to content

Guide

What is an AI voice agent? How it works on Indian phone calls

An AI voice agent listens, decides and speaks on a phone call. See how speech-to-text, an LLM and text-to-speech fit together, and what changes in India.

By Published ai voice agentsguidesindia
One turn of an AI voice call
  1. Caller speaks

    Audio streams in from the phone line

  2. Speech-to-text

    Turns speech into text as they talk

  3. Language model

    Decides the reply, or uses a tool

  4. Text-to-speech

    Speaks the reply in your chosen voice

  5. Caller hears

    Then the loop repeats, turn by turn

On this page

In short

An AI voice agent is software that holds a phone conversation. It turns the caller's speech into text, uses a large language model to decide the reply, and speaks it in a synthetic voice. In India it also has to follow code-mixed speech, run on Indian carrier numbers and respect TRAI's calling rules.

Key takeaways

  • An AI voice agent holds a phone conversation in ordinary sentences; an IVR only plays menus and waits for key presses.
  • Most agents chain three models: speech-to-text listens, a language model decides, text-to-speech speaks.
  • Turn-taking and memory of the call are what make it feel like a conversation.
  • In India it has to follow code-mixed speech such as Hinglish and run on Indian carrier numbers.
  • AI calls fall under the same TRAI telemarketing rules as other commercial calls; consent, DLT and the number series stay with the business.

Most people's picture of an automated phone call is still a recorded voice saying "press 1 for sales". An AI voice agent is a different thing. It picks up, or dials out, and talks with the person on the other end in ordinary sentences. When the caller asks a question in the middle of a sentence, it answers. When they say "kal call karna", it books a callback for tomorrow.

This guide explains what goes on inside one, the two ways it gets used, the jobs it suits and the jobs it doesn't, and what is different about running one on Indian phone lines.

How does an AI voice agent work on a call?

#

Think of three specialists passing notes very quickly. One listens, one thinks, one speaks. Most voice agents, including CallMitra's default setup, are built exactly this way, as a chain usually called a pipeline.

  1. Speech-to-text (STT) listens. The caller's audio streams in from the phone line, and a speech recognition model turns it into text as they speak. This is the step that has to cope with phone-line audio, background traffic, accents and people switching between Hindi and English mid-sentence.
  2. The language model (LLM) decides. The text goes to a large language model along with the agent's brief: who it is calling for, what it must find out, what it must never promise, and the knowledge notes you gave it. The model writes the next thing to say. It can also decide to use a tool, for example to book a callback, transfer the call or save an answer such as the person's budget.
  3. Text-to-speech (TTS) speaks. The reply is turned into audio in the voice and language you chose and played down the line.

Then the loop repeats, turn after turn, until the call ends.

Two more pieces make it feel like a conversation rather than a walkie-talkie:

  • Turn-taking. The agent has to judge when the caller has finished speaking, and stop talking when the caller interrupts. CallMitra exposes these turn-taking and interruption settings, along with noise reduction for noisy lines.
  • Memory of the call. Everything said so far stays in context, so the agent doesn't ask for the person's name twice.

In CallMitra, the speech recognition step can use Deepgram, OpenAI, Sarvam or Soniox. The language model defaults to OpenAI's gpt-4o-mini and the voice defaults to Sarvam. You can change each one per agent.

The pipeline at a glance

Listen
Speech-to-text turns the caller's audio into text
Decide
A language model reads the brief and writes the reply
Act
Tools book callbacks, transfer calls, save answers
Speak
Text-to-speech plays the reply in your chosen voice

Is a "realtime" voice agent different from a pipeline?

#

Some newer models skip the separate steps and take audio in and send audio out directly. These are often called realtime or speech-to-speech models. CallMitra supports a few of them (Gemini Live, OpenAI Realtime and ElevenLabs Agents) as alternatives to the pipeline.

The trade-off is control. With a pipeline you pick the best speech recognition for your language, the language model that follows your brief best and the voice that sounds right, and you can change one without touching the others. A realtime engine is one box, so you take its language coverage and voices as they come. For Indian languages, check that the engine you pick actually speaks the language your callers use before you commit.

Either way, the business side looks the same: a brief, knowledge notes, a voice, a phone number and rules about when to call.

What is the difference between inbound and outbound voice agents?

#

The same agent technology does two quite different jobs.

Outbound means the agent dials. You give it a list of people, or leads arrive from a form, and it calls each one. Typical jobs are following up new enquiries, qualifying leads, confirming appointments and sending reminders. Outbound calling is where most of the rules sit: calling windows, retries, do-not-call lists and consent. CallMitra's AI telecaller is the outbound side. It runs campaigns inside a calling window you set (10:00 to 17:00 IST by default), retries unanswered numbers and stops calling anyone who asks to be removed.

Inbound means the agent answers. People call your number and the agent picks up, answers questions, takes details and transfers to a person when needed. CallMitra's AI receptionist remembers returning callers, plays an after-hours message when you are closed, and by default handles up to 10 simultaneous inbound calls per line.

Many businesses use both. A real-estate developer might have outbound agents calling ad leads within the calling window and an inbound agent answering the number printed on the hoarding.

Which jobs does an AI voice agent do well?

#

Voice agents are good at calls that are frequent, follow a pattern and have a clear end:

  • First calls to new leads. Confirm the enquiry, ask three or four qualifying questions, and book a visit or a callback.
  • Reminders and confirmations. Appointment, class or site-visit reminders, where the person mostly needs to hear the details and say yes, or ask to reschedule.
  • Short surveys. Feedback after a visit, with answers saved as fields rather than buried in a recording.
  • Front-desk questions. Timings, location, documents needed, fee structure, all from notes you provide.
  • Sorting and routing. Working out what the caller wants and transferring the ones that need a person.

What these have in common is that a good human telecaller could do them from a one-page brief. If you can write the brief, an agent can usually follow it. Our guide to writing an AI calling agent brief walks through one end to end.

Here is the kind of short, structured call an agent handles well:

Vikas Yadav

Call ended · 00:43

Hinglish
  1. Agent: Namaste, main Greenfield Homes ki taraf se AI assistant bol rahi hoon. Aapne kal 2BHK ke liye enquiry ki thi, abhi do minute baat ho sakti hai?

  2. Lead: Haan, bolo. Actually main Saturday ko site dekhna chahta tha.

  3. Agent: Bilkul. Saturday ko morning slot chahiye ya afternoon?

  4. Lead: Morning, around 11.

  5. Agent: Theek hai, Saturday 11 baje site visit note kar liya hai. Hamari team aapko location bhej degi. Aur koi sawaal?

  6. Lead: Nahi, bas itna hi.

Call completed

Saved to lead · transcript attached

Sample call

Outbound follow-up in Hinglish

Language
Hinglish
On the call
Agent, Lead
Length
About 43 seconds

The agent follows the lead's mix of Hindi and English and saves the visit date as a field.

Scroll the phone screen to read the whole call.

Where does an AI voice agent fall short?

#

It is worth being honest about this, because a bad fit damages your brand faster than no automation at all.

  • Long negotiations. Haggling over price, terms or exceptions needs judgement and authority the agent should not have. Tell it what it must never promise, and have it hand over.
  • Upset or distressed callers. An agent can stay polite, but someone who is angry about a refund or worried about a medical issue should reach a person quickly. Set up a transfer for these moments.
  • Answers it was never given. A language model can sound confident while being wrong. Keep prices, fees and policies in its knowledge notes, tell it to say "I will check and call back" when something isn't there, and review the topics it couldn't answer.
  • Very noisy or poor lines. Speech recognition works on what it hears. On a weak mobile signal or in a crowded market, it will mishear more often, just as a person would.
  • Things that need a signature or a decision. Agents collect information and book next steps. Approvals, payment disputes and anything legally binding stay with your team.

The fix in almost every case is the same: a clear brief, a transfer to a person for the hard cases, and someone on your team who listens to a sample of calls every week. Every CallMitra conversation keeps a recording, transcript, summary and the caller's mood, so that review is quick.

What changes when you run a voice agent in India?

#

Most voice-agent guides are written for American English and American phone lines. Three things are different here.

Languages are mixed, not separate

#

Very few Indian callers speak "pure" Hindi or "pure" English on the phone. A lead in Pune may answer in Marathi, ask about the EMI in English and confirm in Hindi. An agent for India needs speech recognition that copes with code-mixing, and a voice that pronounces Indian names and numbers properly.

CallMitra agents speak ten languages: English, Hindi, Hinglish, Telugu, Tamil, Kannada, Malayalam, Marathi, Gujarati and Bengali. You choose the language per agent, and you can add an extra language or let the agent adapt when the caller switches. If your leads are spread across the north, Hinglish calling is usually the safest default; if you serve one state, pick its language.

Phone numbers come from Indian carriers, with KYC

#

The agent needs a phone line. CallMitra connects to numbers you bring from Twilio, Plivo or Vobiz, and each behaves differently in India. Twilio's own India guidelines say outbound calls to India can only be made from international, non-Indian numbers. Plivo and Vobiz rent Indian numbers to India-registered businesses after KYC, and both require call audio to stay within India. Our Twilio vs Plivo vs Vobiz comparison covers setup, limits and transfers for each.

The number series matters too. Under TRAI's rules, promotional calls come from numbers starting with 140, and service or transactional calls from BFSI and government senders come from the 1600 series, with a 1601 series being phased in for other sectors. CallMitra warns when a calling number sits outside these series but doesn't block the call.

TRAI's telemarketing rules apply to AI calls too

#

India regulates commercial calls under TRAI's Telecom Commercial Communications Customer Preference Regulations (TCCCPR). The third amendment, published on 18 September 2026, defines application-to-person (A2P) calls to include calls made with artificial voice, and expects senders to declare such calls and their caller-ID ranges on the DLT platform. Read the regulation and confirm with your telecom operator exactly what you need to register.

In practice, for an AI voice agent this means:

  • Keep promotional calls to sensible hours. CallMitra uses a calling window per campaign, 10:00 to 17:00 IST by default, and warns you if you set a window that starts before 9:00 or ends after 21:00. TRAI's preference time bands treat 21:00 to 10:00 as off by default for promotional calls unless the person has opted in.
  • Honour "stop calling me". When someone asks to be removed, CallMitra adds the number to your workspace do-not-call list and never auto-dials it again.
  • Know what CallMitra does not do. It doesn't check the national DND registry (NCPR) or register you on DLT. Consent records, scrubbing, DLT registration and choosing the right number series stay with your business. Our page on TRAI telemarketing rules goes deeper.

None of this is legal advice. If your calls sit close to the line between service and promotion, ask your telecom operator or a lawyer.

How do you get started with an AI voice agent?

#

Start small and specific:

  1. Pick one call type. New-lead follow-up or appointment reminders are good first jobs, because the outcome is easy to judge.
  2. Write the brief. Who to call, what to find out, what never to promise, how to end. Put prices and policies in knowledge notes.
  3. Choose the language and voice. Match how your callers actually speak.
  4. Test on your own phone. In CallMitra, test calls in the browser or to your own phone aren't billed. Listen for wrong answers and awkward pauses.
  5. Connect a number and a small list. Run a few days of real calls, listen to recordings and tighten the brief.

CallMitra bills only answered call time, with plans from ₹3,999 a month for 400 minutes (prices exclude 18% GST). The demo page lets you hear an agent before you sign up.

Written by

Neeraj Goswami

Founder and CEO, CallMitra AI

Writes and reviews the guides, comparisons and compliance notes on this site and is responsible for their accuracy. Facts about rules and other companies link to their sources.

Frequently asked questions

Short answers to what teams ask before they switch on AI calling.

Is an AI voice agent the same as an IVR?

No. An IVR plays fixed menus and waits for key presses. An AI voice agent understands what the caller says in their own words, answers from a brief and knowledge notes, and can ask follow-up questions.

Can an AI voice agent speak Hindi and Hinglish?

Yes, if its speech recognition and voice support the language. CallMitra agents speak English, Hindi, Hinglish, Telugu, Tamil, Kannada, Malayalam, Marathi, Gujarati and Bengali, and you pick the language per agent.

Does the caller know they are talking to AI?

They should. In CallMitra, AI disclosure is on by default: the agent says it is an AI assistant if the caller seems to think it is a person, and it always answers honestly when asked.

What happens when the AI voice agent cannot answer a question?

A well-set-up agent says it will check, offers a callback or transfers the call to a person. CallMitra also lists the topics an agent could not answer so you can add them to its knowledge.

Do I need my own phone numbers?

For CallMitra, yes. You connect numbers from Twilio, Plivo or Vobiz, because CallMitra does not sell numbers. Each carrier has its own KYC and India calling rules.

Is an AI voice agent allowed under TRAI rules?

Calls by an AI voice agent fall under the same TRAI telemarketing rules as other commercial calls, and the 2026 amendment names artificial voice explicitly. Consent, DLT registration and the right number series are the business's responsibility.

Sources

  1. TRAI — Telecom Commercial Communications Customer Preference Regulations (TCCCPR)www.trai.gov.in
  2. TRAI — TCCCPR 2018 (regulation text)www.trai.gov.in
  3. TRAI — TCCCPR (Third Amendment) Regulations, 18 September 2026www.trai.gov.in

Hear an AI agent call your phone

Build an agent from a job description and test a call before you connect any leads.