How to Add an Avatar to the Gemini Live API

If your conversational app already runs on the Gemini Live API, Liforma can turn Gemini's native audio responses into an animated character without replacing Gemini. Gemini remains the realtime multimodal brain; Liforma supplies the visual character and speech-to-animation layer.

Can you add an avatar to Gemini Live?

Yes. Liforma provides connectGeminiLive() in @liforma/client/google. The helper streams microphone audio to your Gemini Live proxy, receives Gemini's generated PCM audio and drives the Liforma avatar in realtime.

Why Gemini Live is a natural fit for an avatar

Gemini Live is designed for realtime bidirectional interaction using audio and other modalities. That means the conversational system may already be capable of listening, reasoning and producing native spoken responses. The missing layer is often presentation: what does the user see while Gemini speaks?

Liforma can provide that layer without forcing the Gemini interaction back through a separate text-only agent.

What stays in Gemini?

• realtime audio input;
• Gemini reasoning and instructions;
• multimodal context you choose to send;
• tool/function behaviour in your Gemini application;
• native generated audio; and
• your existing Gemini Live conversation architecture.

What does Liforma add?

• the animated character;
• realtime lip-sync / speech-to-animation;
• visual identity, hair, costumes and backdrops;
• browser-native rendering and playback;
• Experience scenes and state; and
• the option to use the same character across other experiences.

Minimal Gemini Live + Liforma integration

npm install @liforma/client

import { connectGeminiLive } from '@liforma/client/google';

const bridge = await connectGeminiLive(experience, {
  proxyUrl
  // captureMic: true // default
});

// Later:
await bridge.end();

The helper sends microphone PCM to Gemini Live and writes Gemini's generated PCM into Liforma utterances. It also consumes Gemini output transcription where available so Liforma can improve alignment between the speech and facial animation.

See the Gemini Live → Liforma integration guide.

Why do I need a WebSocket proxy?

The Gemini Live connection should be terminated by a trusted backend so that Google credentials are not exposed in the browser.

The Liforma helper therefore connects to a same-origin WebSocket proxy in your application. That proxy talks to Gemini Live and forwards the relevant realtime messages.

This is also operationally important: a long-lived WebSocket proxy needs hosting that supports long-running WebSocket connections. A conventional short-lived serverless function is not always the right place for it.

How does interruption work?

Gemini Live can indicate that a model turn has been interrupted. The Liforma bridge maps that event to the active avatar utterance so the visual speech stops with the agent rather than continuing after the conversation has moved on.

Does Liforma replace Gemini's native audio?

No. In this setup Gemini owns the spoken response. Liforma consumes the audio that Gemini generated and animates it.

Can Gemini's output transcription improve lip-sync?

Yes. Liforma can use the accompanying output transcription as additional alignment information. The audio remains authoritative, while the transcript helps the avatar's speech animation follow the spoken content more precisely.

Why use Gemini Live + Liforma instead of a chained STT / LLM / TTS stack?

If your product was built around Gemini Live specifically for realtime native-audio interaction or multimodality, replacing it with a conventional chained pipeline solely to add an avatar would throw away much of that architecture.

Liforma Motion lets Gemini stay intact and treats the avatar as a composable output layer.

What if I only use Google Cloud Text-to-Speech?

That works too. Liforma's Google provider guide also covers one-shot and streamed Google Cloud TTS. For a realtime conversational agent, however, Gemini Live is the more relevant speech-to-speech path.

When is this a good fit?

• you already built a voice agent with Gemini Live;
• you want to keep Google's native audio output;
• your agent also uses Gemini's multimodal capabilities;
• you want to add a face without replacing Gemini;
• you want a visual tutor, assistant, guide or fictional character driven by Gemini; or
• you want Gemini Live as the voice brain inside a larger Liforma Experience.