Multi-Character AI: How to Build Conversations With Multiple AI Characters

Multi-character AI is not just several chatbots on the same screen. A convincing multi-character experience needs each character to have a distinct identity, know when to speak, understand who else is present, react to shared events and carry the right state from one scene to the next.

Once you add those requirements, the problem changes. You are no longer building a single conversation with an avatar. You are orchestrating a cast.

What is multi-character AI?

Multi-character AI is an interactive system in which a user can encounter, speak with or observe more than one AI character inside the same experience.

Those characters may appear together or at different times. They may cooperate, disagree, share information, know different facts or react differently to the user's behaviour.

Examples include:

• a panel interview with three interviewers;
• a sales meeting with a buyer and procurement manager;
• a customer-service scenario involving a customer, supervisor and coach;
• a language lesson with a waiter, friend and shopkeeper;
• a mystery in which different witnesses know different facts;
• a role-playing game with a party of AI characters;
• a historical discussion involving several figures with different perspectives; or
• an interactive story in which characters enter and leave as the plot progresses.

Why one AI agent is often not enough

The standard conversational-AI architecture assumes one user and one agent. That makes sense for many assistants: ask a question, receive an answer, repeat.

But real-world situations are frequently social rather than one-to-one.

A candidate may face a hiring manager, technical interviewer and future colleague. A salesperson may need to persuade an enthusiastic champion while answering objections from a skeptical finance director. A manager may mediate between two employees whose accounts of a problem differ.

Representing all of those people as one omniscient AI voice removes much of what makes the situation interesting.

Multiple characters let the experience preserve conflicting goals, incomplete information, different personalities and changing relationships.

The experience should own the world

A useful architecture separates character state from experience state.

A character needs to know things about itself: its identity, role, personality, voice, objectives and perhaps private knowledge.

The experience needs to know things about the world: which scene is active, which characters are present, what the user has done, what shared facts have been discovered, what objectives have been completed and what should happen next.

That separation is central to Liforma's Avatar Experience model. Characters are reusable participants; the experience orchestrates what happens around them.

Shared state is what makes the cast feel like one world

Without shared state, multi-character AI can feel like opening several unrelated chat windows.

Imagine telling Character A that you lost a key, then walking into another room and meeting Character B. If the second character is supposed to have been informed, the experience needs a shared representation of that event.

That does not mean every character should know everything. In fact, selective knowledge is often what makes the experience interesting.

A mystery might track:

• key_missing = true as shared world state;
• Character A knows where it was last seen;
• Character B knows who entered the room;
• Character C is secretly responsible and should not reveal that fact unless confronted.

The experience can decide which state each character is allowed to see.

Characters should not all know the same things

A common failure mode in multi-agent experiences is accidental omniscience. Every AI receives the same giant transcript and therefore behaves as though it heard every private conversation and knows every hidden fact.

That may be convenient technically, but it damages the fiction.

A stronger system separates information into categories such as:

• public world state — facts everybody present can know;
• scene state — events that occurred in the current location;
• character knowledge — facts known only by one character;
• user history — things the user has done or said;
• hidden author state — variables used to control progression that characters should not expose.

This gives creators something closer to dramatic structure than a shared group chat.

Who should speak next?

Turn-taking is one of the hardest parts of multi-character AI.

If every character responds to every user utterance, the experience becomes noisy and unnatural. If the user has to manually select a character before every sentence, it can feel mechanical.

There are several useful strategies.

Explicit addressing

The simplest rule is to respond when the user clearly addresses a character by name or selects them in the interface. This is deterministic and works well when precision matters.

Contextual turn selection

The experience can infer who should answer from the content. If the user asks the finance director about budget, that character is the natural respondent. If the user asks the whole group a question, one or more characters may respond.

Author-controlled turns

In structured training, the author may already know who should speak at a particular point. A manager can enter after an escalation threshold is reached, or a coach can speak only after the scenario is complete.

Character initiative

Some experiences benefit from characters deciding to interject. That can feel much more alive, but it needs restraint. A character should have a reason to speak: disagreement, urgency, new information, an emotional reaction or an objective that has become relevant.

The best systems usually combine these approaches rather than relying on one universal rule.

Multi-character does not mean multi-LLM chaos

It is easy to imagine a system where five autonomous agents continuously talk to one another, consume tokens and independently decide what to do. That is not necessary for most interactive experiences.

The useful abstraction is often much simpler: only invoke the intelligence that is needed for the current turn.

If the user is speaking to the hotel guest, there may be no reason to run the manager and coach at all. They can remain dormant until the experience brings them into the scene.

This makes multi-character experiences both more controllable and more economical.

Scenes make a cast manageable

Not every character should be present all the time.

Scenes let an experience define a smaller active cast and a clear context. Moving to a new scene can change:

• the characters who are present;
• the backdrop or 3D environment;
• the current objective;
• the information available to characters;
• the tone of the interaction; and
• the rules governing what happens next.

That is one reason Liforma supports multi-scene experiences rather than treating the whole session as one indefinitely long conversation.

Example: a multi-character sales simulation

Consider a sales-training experience with three AI characters.

Maria, the internal champion, likes the product and wants the purchase to happen. She is helpful but does not have final authority.

David, the finance director, cares about cost, payback and contractual risk. He starts skeptical and becomes more engaged only when the learner gives precise answers.

Alex, the coach, does not participate in the sales meeting. Alex appears afterwards to review what happened.

The experience might track:

• whether the learner uncovered the commercial objective;
• whether pricing was discussed too early;
• Maria's confidence in the salesperson;
• David's perception of financial risk;
• which objections were handled successfully; and
• whether a clear next step was agreed.

Maria and David can hear the same conversation but react differently because their goals are different. Alex can later receive the relevant transcript and stats without pretending to have been physically present in the meeting.

That is much closer to a real sales situation than collapsing all three roles into a single “sales-training chatbot.”

Example: a mystery with private knowledge

Multi-character design becomes even more interesting in entertainment.

Imagine a mystery with four characters. Each witnessed a different part of an event. One is lying. The player can move between locations and question them in any order.

The experience maintains the objective truth of the world, but each character sees only the subset they should know. When the player discovers a clue, a shared stat records it. That clue might unlock a new question, cause another character to become nervous or open a new scene.

At that point, the AI is not merely generating dialogue. Dialogue is one interface into a stateful interactive system.

Example: panel interview practice

A panel interview demonstrates another advantage: different characters can evaluate different aspects of the same answer.

A hiring manager might focus on leadership. A technical interviewer can probe expertise. A future colleague might care about communication and collaboration.

The user experiences the social pressure of addressing a group rather than repeatedly answering one generic interviewer. The panel can also ask follow-up questions based on what other members heard.

At the end, the experience can combine character-specific observations with shared stats to generate feedback.

Characters need motives, not just personalities

“Friendly,” “serious” and “sarcastic” are useful style cues, but they are not enough to create a convincing cast.

Characters become interesting when they want different things.

For each character, define:

• what they want from the situation;
• what they know;
• what they do not know;
• what they are willing to reveal;
• what changes their attitude;
• what would make them leave, interrupt or escalate; and
• how their objectives relate to the other characters.

Conflict and cooperation then emerge from the structure instead of being simulated by random personality quirks.

Shared stats can coordinate characters without hard-coding dialogue

One of the most useful techniques is to let several characters react to the same small set of experience variables.

Suppose a negotiation tracks trust, budget_confidence and urgency. Different characters can interpret those values differently.

• The champion becomes more supportive as trust rises.
• The finance director pushes harder when budget confidence is low.
• The CEO enters only when urgency becomes high enough.
• The coach later explains which user behaviours moved each variable.

The author is not scripting every possible sentence. They are defining the dynamics of the experience.

Should AI characters talk to each other?

Sometimes.

Character-to-character dialogue can make a scene feel alive, but it should serve the user rather than become an expensive AI conversation the user merely watches.

Useful cases include:

• one character challenging another's claim;
• a manager interrupting an escalating customer interaction;
• two witnesses disagreeing about what happened;
• a panel briefly discussing the user's answer; or
• story characters reacting to an important event.

In many cases the author can constrain these exchanges tightly. The point is not to create an infinite autonomous-agent simulation. It is to make the authored situation feel socially coherent.

How much context should each character receive?

Sending the entire history of the entire experience to every character is simple, but it is rarely optimal.

It increases token usage, makes prompts harder to control and risks leaking information that a character should not know.

A better context package can include:

• the character's identity and instructions;
• relevant recent dialogue;
• the current scene and objective;
• only the shared state the character is allowed to know;
• character-specific memories or facts; and
• the available tools or actions for this moment.

This is similar to how modern game AI systems expose selected world state to characters rather than giving every non-player character a copy of the entire game database.

Current character platforms are moving in this direction too. Convai, for example, documents “Dynamic Context” for exposing selected live game variables and events to AI characters rather than relying only on a static prompt. See Convai's Dynamic Context example.

Multi-character AI is an orchestration problem

Once several characters exist, the most important intelligence may not belong to any character.

The experience itself needs orchestration logic that can answer questions such as:

• Who is currently active?
• Who should respond to this turn?
• Which characters heard what was said?
• Which stats should change?
• Has an objective been completed?
• Should somebody enter or leave?
• Should the scene change?
• Has the experience reached an ending?

That is why multi-character AI fits naturally with an experience or agent-platform model rather than a single prompt attached to a face.

Visual design matters more when there is a cast

With one avatar, the user only needs to recognise one person. With several characters, visual differentiation becomes part of usability.

Distinct silhouettes, hairstyles, clothes, voices and positions can help the user immediately understand who is speaking and what role they play.

This is one reason Liforma's reusable costumes, hairstyles and sets matter. They are not merely cosmetic customisation; they become part of how a cast is designed and understood.

Stylised characters can be especially useful here because creators have more freedom to make characters visually distinct while keeping them inside the same art direction. See Photorealistic vs Stylized AI Avatars.

Multi-character experiences do not have to be expensive

A naïve multi-agent architecture can multiply costs quickly if every character runs continuously. But an authored experience does not need to work that way.

Most of the time, only one character needs to generate a response. Other characters can remain inactive until the orchestration layer decides they should participate.

The same is true across scenes. There is no reason to keep a coach, manager or future character actively connected while the user is talking to somebody else.

This aligns well with Liforma's request-based architecture. Rather than maintaining a persistent voice session for every possible character, the system can invoke the STT → LLM → TTS → STA pipeline when a character actually needs to respond. For the architectural reasoning, see Does Conversational AI Really Need WebRTC?.

How to design your first multi-character experience

Keep the first version small. Three characters are enough to create dynamics that one agent cannot.

For each character, write down:

1. Role: who are they in this situation?
2. Goal: what do they want?
3. Knowledge: what do they know that others may not?
4. Relationship: how do they feel about the user and the other characters?
5. Triggers: what makes them speak, change attitude, enter or leave?

Then define the experience-level pieces:

1. the initial scene and active characters;
2. a small set of shared stats or facts;
3. one meaningful transition to another scene;
4. one consequence that depends on what the user did; and
5. an ending or feedback phase.

That is enough to create something that feels fundamentally different from a one-character chatbot.

When should you use multiple characters?

Use multiple characters when the differences between people are part of the problem the user is trying to navigate.

If you only need a helpful product expert answering questions, one character is probably better. Adding three avatars does not automatically make the experience more sophisticated.

But if the scenario contains different roles, incentives, perspectives, knowledge or relationships, collapsing them into one AI can remove the very thing you are trying to simulate.

Where Liforma fits

Liforma is built so that creators can treat AI characters as a cast rather than as isolated endpoints.

Characters are reusable. They can have different appearances and voices. Experiences can contain multiple characters and multiple scenes. Stats and state can carry what happened forward. Nodes can change the progression of the experience. Feedback can be generated from the events that actually took place.

And because the experience itself is the reusable unit, the author does not need to build a bespoke multi-agent application every time they want to create a panel interview, negotiation, training scenario or interactive story.

That is the larger idea behind multi-character AI: the intelligence is not only in the characters; it is in the experience that coordinates them.