How to Add an Avatar to a LiveKit Agent

If your voice agent already lives in a LiveKit room, Liforma can turn the agent's remote audio track into an animated character. You can keep LiveKit Agents, rooms, telephony, model plugins and realtime transport exactly where they are. Liforma becomes the visual participant layer.

Can you add an avatar to a LiveKit Agent?

Yes. Liforma provides connectLiveKitAgent() in @liforma/client/livekit. The helper joins the LiveKit room, selects the agent audio track and routes that MediaStreamTrack into the Liforma Experience.

Why LiveKit is different from the other integrations

LiveKit is not one specific speech-to-speech model. It is a realtime media and agent framework. Your LiveKit Agent might use a traditional STT → LLM → TTS pipeline, OpenAI Realtime, Gemini Live or another realtime model.

That makes the Liforma integration especially general: Liforma can sit at the media boundary and animate whatever speech the LiveKit agent ultimately publishes into the room.

What stays in LiveKit?

• the room and realtime media transport;
• your LiveKit Agent process;
• STT / LLM / TTS or realtime-model selection;
• tool logic and workflows;
• telephony if you use it;
• participant lifecycle; and
• the agent's published audio track.

What does Liforma add?

• the visible animated character;
• speech-to-animation and lip-sync;
• character identity and appearance;
• backdrops and scenes;
• browser-native visual rendering; and
• the wider Liforma Experience layer around the LiveKit voice agent.

Minimal LiveKit + Liforma integration

npm install @liforma/client livekit-client

import { connectLiveKitAgent } from '@liforma/client/livekit';

const bridge = await connectLiveKitAgent(experience, {
  url,
  token,
  // enableTranscript: true,
  // shouldBridgeParticipant: (p) => p.identity.startsWith('agent')
});

// Later:
await bridge.end();

The helper joins the room, bridges the selected remote audio track directly into a Liforma utterance, and can consume LiveKit transcription streams for improved alignment.

See the LiveKit → Liforma integration guide.

Mint the LiveKit participant token on your server

The browser needs a participant token appropriate for the room. Generate that token server-side, then pass the LiveKit URL and token to the Liforma bridge.

Do not play the agent track twice

This is the most important implementation detail.

Once Liforma owns playback of the remote agent audio track, do not also attach that same track to a LiveKit <audio> element. Otherwise the user will hear two copies of the agent speech.

How does lip-sync work with a long-lived media track?

LiveKit commonly exposes one long-lived remote audio track rather than a series of separate PCM files. Liforma can create an utterance directly from the MediaStreamTrack.

If your agent publishes text on the standard LiveKit transcription stream, the bridge can forward that text to the utterance as alignment information while the media track remains the authoritative audio source.

Can I use OpenAI Realtime or Gemini Live inside LiveKit?

Yes. LiveKit Agents supports both chained voice pipelines and realtime speech models. If your agent already wraps OpenAI Realtime, Gemini Live or another supported realtime provider inside LiveKit, Liforma does not need a separate provider-specific connection. It can simply animate the LiveKit remote track.

What if ElevenLabs is connected through WebRTC?

This is also the right path when an ElevenLabs configuration delivers the agent speech as a LiveKit remote track rather than as PCM callbacks. Bridge the media track with connectLiveKitAgent().

Can I keep LiveKit telephony?

Yes. Liforma is only consuming the resulting agent media for the visual experience. Your LiveKit room, SIP/telephony setup and backend agent lifecycle can stay intact.

Why use LiveKit + Liforma?

LiveKit is a strong foundation when realtime media infrastructure is already part of your product. Liforma lets you add a visible character without replacing that infrastructure or tying the avatar to one particular model provider.

When is this a good fit?

• your voice agent already joins LiveKit rooms;
• you want to preserve LiveKit transport and telephony;
• you want model-provider flexibility behind the avatar;
• your agent audio already exists as a remote media track;
• you want to add a face to an existing LiveKit application; or
• you want the LiveKit voice agent inside a richer Liforma Experience.