How to Add an Avatar to a Deepgram Voice Agent

If you already use Deepgram's Voice Agent API, you can keep Deepgram's complete listen → think → speak loop and use Liforma only for the avatar. Liforma receives the Voice Agent's generated audio, plays it through the Experience and drives realtime facial animation and lip-sync.

Can you add an avatar to a Deepgram Voice Agent?

Yes. Liforma provides connectDeepgramAgent() in @liforma/client/deepgram.

Deepgram continues to handle the voice-agent pipeline over its single realtime connection. Liforma adds the visual output layer.

Why Deepgram Voice Agent + Liforma?

Deepgram's Voice Agent API already combines speech recognition, an LLM integration and speech generation in one realtime agent API. If that system is already working for you, there is no need to unbundle it just to create a visual agent.

Liforma Motion is designed for exactly this case: take the speech produced by the existing voice agent and animate a character from it.

What stays in Deepgram?

• speech input;
• the Voice Agent session;
• your listen / think / speak configuration;
• LLM/provider choice;
• prompt and agent behaviour;
• function calls; and
• generated agent speech.

What does Liforma add?

• an animated character with realtime lip-sync;
• avatar appearance, costumes and hair;
• backdrops and scenes;
• browser-native visual playback;
• Liforma Experience state and orchestration if required; and
• embedding into a website or application.

Minimal Deepgram + Liforma integration

npm install @liforma/client

import { connectDeepgramAgent } from '@liforma/client/deepgram';

const bridge = await connectDeepgramAgent(experience, {
  proxyUrl,
  // agent: { listen, think, speak }
});

// Later:
await bridge.end();

The Liforma helper manages Deepgram's realtime handshake, streams microphone audio and forwards the binary agent audio into Liforma utterances for playback and animation.

See the Deepgram → Liforma implementation guide.

Why does the browser use a proxy?

The Deepgram Voice Agent endpoint expects authenticated WebSocket access. Browsers cannot attach the required Authorization header to an arbitrary WebSocket in the same way a server can.

The recommended Liforma pattern is therefore a same-origin WebSocket proxy. Your server holds the Deepgram credential, and the browser connects to your proxy.

Does Liforma handle the Deepgram handshake?

Yes. The helper handles the required sequence: connect, wait for Welcome, send Settings, wait for SettingsApplied, then begin the realtime audio flow.

It also requests headerless linear PCM output so the generated audio can flow directly into the Liforma utterance.

What happens when the user interrupts?

When Deepgram reports that the user has started speaking, the Liforma bridge cancels the active avatar utterance. The visual character therefore stops speaking with the voice agent rather than finishing stale audio.

Can I keep my own LLM in Deepgram?

Yes. Deepgram supports multiple LLM provider configurations for the Voice Agent API. Liforma does not need to know which intelligence provider produced the answer; it simply consumes the resulting speech.

Can I use Deepgram TTS without the Voice Agent API?

Yes. If you only use Deepgram speech synthesis, you can feed the resulting audio to Liforma directly through the generic speech API. This page focuses on Voice Agent because it is the most natural end-to-end speech-to-speech integration.

Why use Deepgram + Liforma instead of Liforma Live?

Use Deepgram + Liforma when the Deepgram Voice Agent is already part of your architecture or when its speech, telephony, models, provider configuration or agent behaviour are important to your product.

Use Liforma Live when you want the simpler option: one managed STT → intelligence → TTS → animation stack without a second provider integration.

When is this a good fit?

• you already have a Deepgram Voice Agent;
• you want a visual layer without changing the voice-agent architecture;
• you want to preserve your Deepgram model and voice configuration;
• you want a website avatar driven by Deepgram;
• you want the avatar to inherit Deepgram interruption behaviour; or
• you want Deepgram speech inside a Liforma Experience.