RAG for AI Avatars: Why Your Character Doesn't Always Need a Frontier LLM

Your AI avatar does not always need a frontier LLM to be an expert. In many interactive avatar experiences, the character only needs deep knowledge of a very narrow domain: the help pages for one product, a specific training manual, a company's policies, a single course module, or one topic such as simultaneous equations.

In those situations, a small fast model with high-quality retrieval can often be more useful than a much larger model relying only on what it learned during training.

What is RAG?

Retrieval-Augmented Generation (RAG) means retrieving relevant information from an external knowledge source and placing that information into the model's context before it generates an answer.

Instead of asking the language model to answer entirely from its internal training, the system first asks:

1. What information does this question require?
2. Which parts of our knowledge base are most relevant?
3. What small set of evidence should the model see?
4. Given that evidence, what should the character say?

OpenAI describes RAG as retrieving content to augment the prompt before generation, particularly when a model lacks contextual knowledge, has stale knowledge or needs access to proprietary information. See OpenAI's accuracy-optimisation guide.

Why this matters especially for AI avatars

Most intelligent avatars are not being asked to solve every problem in the world.

They usually have a job.

A website assistant may need to know:

• the company's products;
• the current pricing plans;
• the help centre;
• the returns policy; and
• the page the visitor is currently viewing.

A maths tutor may need to be excellent at:

• simultaneous equations;
• the exact teaching method used by the course;
• common learner mistakes;
• worked examples; and
• the curriculum level being taught.

A training character may only need to know:

• one sales methodology;
• a company's product catalogue;
• the objection-handling playbook;
• the scenario-specific facts; and
• the learner's current state in the exercise.

A frontier LLM may know a little about all of those things. That is not the same as knowing the exact material your experience needs.

Frontier models can still be weak on your narrow domain

Large frontier models are trained to be broad generalists. Their strength is that they can handle an enormous range of tasks and domains.

But the knowledge inside the model may be:

• incomplete;
• out of date;
• too generic;
• different from your organisation's own rules;
• different from the teaching method you want the avatar to follow; or
• simply absent because the source material is private.

For a support assistant, knowing general facts about SaaS billing is less useful than knowing the exact cancellation policy on your website today.

For a tutor, knowing several ways to solve simultaneous equations can actually be less useful if the lesson requires the character to follow one particular method consistently.

A small model with the right context can beat a larger model with the wrong context

This is the key architectural shift.

Suppose Model A is a frontier model with broad world knowledge but no access to your current product documentation.

Model B is much smaller, but every time the visitor asks a question it receives the three most relevant passages from your help centre.

For the question:

“Can I transfer unused credits to another workspace?”

Model size may be much less important than whether the model receives the paragraph containing your actual workspace-credit policy.

Recent research supports this broader direction, while also showing that implementation quality matters. The Alan Turing Institute reported that a 3B model combined with retrieval and lightweight reasoning techniques achieved near-frontier performance on a constrained real-world health task. Other 2026 work has found small models competitive with larger models in specialised industrial RAG settings. See the Turing Institute work on small language models.

That does not mean RAG magically turns every small model into a frontier model. Small models can still struggle to interpret difficult evidence or perform complex reasoning. The point is more specific: for narrow knowledge-grounded tasks, supplying the right evidence can reduce how much general model capability you need.

Why small models are attractive for realtime avatars

Interactive characters are unusually sensitive to latency.

A user finishes speaking and waits. During that pause, the system may need to:

1. finish speech recognition;
2. understand the request;
3. retrieve relevant knowledge;
4. generate the answer;
5. start text-to-speech; and
6. begin avatar animation.

Every extra few hundred milliseconds is noticeable.

A smaller model can offer:

• lower inference latency;
• lower cost per turn;
• higher throughput;
• less GPU memory;
• more predictable capacity; and
• the option to run on cheaper infrastructure.

If RAG supplies the domain knowledge, the small model can spend its capacity on the actual conversation rather than needing to contain an enormous amount of world knowledge in its weights.

RAG changes the economics of expertise

Without retrieval, developers often respond to quality problems by selecting a larger model.

That can work, but it may also mean paying a premium on every turn forever.

RAG creates another option:

make the context better instead of making the model bigger.

For an experience that answers thousands or millions of narrow-domain questions, that distinction can be economically significant.

Example: a website support avatar

Imagine a website with 300 help pages.

A visitor opens a Liforma website assistant and asks:

“Why can't I embed this experience on my client's domain?”

The avatar does not need a model that has read most of the public internet.

It needs the current documentation describing publication settings, allowed origins and embedding rules.

A good RAG pipeline can retrieve those passages and put them in front of a fast model. The model's job becomes much simpler:

understand the question, read the relevant evidence, explain it clearly.

That can be both faster and more reliable than hoping a larger model remembers a product whose behaviour may have changed last week.

Example: a simultaneous-equations tutor

Now consider an AI maths tutor.

The course may require a particular progression:

1. recognise the two equations;
2. choose elimination or substitution;
3. show the transformation step explicitly;
4. identify the common learner mistake; and
5. ask the learner to complete the next step rather than simply revealing the answer.

A frontier model probably knows simultaneous equations. But it may not consistently teach them in the exact style your course requires.

A knowledge pack can contain your worked examples, explanation style, error patterns and pedagogy. Retrieval then gives the model the relevant teaching material for the current learner question.

The expertise comes from the combination of the model and the curated knowledge, not from model size alone.

Liforma Knowledge Packs

Liforma makes this pattern a first-class part of creating intelligent characters.

A Knowledge Pack is a reusable body of domain knowledge that can be attached to an avatar experience. The simplest way to create one is to upload the documents that define what the character should know.

That might be:

• product documentation;
• help-centre articles;
• training manuals;
• course material;
• policies and procedures;
• technical reference documents;
• lesson notes and worked examples; or
• other curated source material.

Once the pack is attached, Liforma can retrieve the relevant material during a conversation and make it available to the character's intelligence layer.

The creator does not need to design a vector database, choose chunk sizes, build a retrieval service or manually assemble prompt context for every turn.

Knowledge Packs are reusable

The knowledge should not have to belong to one character forever.

A company support Knowledge Pack might be shared by several characters:

• a sales assistant on the website;
• a support agent inside the product;
• a training customer used to teach staff; and
• a coach that reviews whether staff gave accurate answers.

Likewise, one maths Knowledge Pack can support a tutor in several different lessons or visual environments.

This matches Liforma's wider philosophy: characters, appearances, sets, knowledge and experiences should be reusable building blocks rather than hard-coded into one application.

Why not just paste all the documents into the system prompt?

For a tiny knowledge base, you sometimes can.

But it stops working well as the source material grows.

Sending everything on every turn means:

• more input tokens;
• higher latency;
• higher cost;
• more irrelevant information competing for the model's attention; and
• a greater chance that the important passage is buried inside noise.

RAG is a filtering mechanism. The goal is not to give the model more context. It is to give it the right context.

Good RAG is much harder than “use a vector database”

This is where many home-grown implementations underestimate the problem.

A naive RAG prototype can be built quickly:

1. split documents into chunks;
2. embed them;
3. store the vectors;
4. embed the user's question;
5. retrieve the nearest chunks; and
6. paste them into the prompt.

That can produce an impressive demo.

It is not the same as building a fast, reliable production retrieval system.

Production RAG has two separate quality problems

A RAG answer can fail in two fundamentally different ways.

1. Retrieval failure: the system did not find the evidence the model needed.
2. Generation failure: the correct evidence was retrieved, but the model misunderstood or ignored it.

OpenAI's RAG guidance makes this distinction explicitly. Too little relevant context leaves the model unable to answer; too much irrelevant context can drown out the useful information and increase hallucination.

What actually has to be optimised?

A serious RAG system may need to make good decisions about:

• document ingestion — extracting useful content from different file formats;
• chunking — where one retrievable unit begins and ends;
• metadata — document type, section, date, permissions and source;
• embeddings — choosing representations suitable for the domain;
• semantic retrieval — finding conceptually relevant passages;
• keyword or hybrid retrieval — handling exact product names, identifiers and rare terms;
• query rewriting — turning a conversational utterance into a better search query;
• reranking — deciding which retrieved candidates deserve prompt space;
• context packing — fitting the useful evidence into a compact model context;
• source freshness — ensuring old documents do not override current ones;
• permissions — ensuring a character only retrieves information the user is allowed to see;
• caching — avoiding unnecessary repeated work;
• latency budgets — making retrieval fast enough for spoken conversation;
• evaluation — testing retrieval separately from answer quality; and
• fallback behaviour — knowing when there is not enough evidence to answer confidently.

None of those problems is conceptually exotic. Together, they represent a substantial engineering project.

Realtime RAG is harder than ordinary document Q&A

Latency matters much more when the answer will be spoken by a character.

A user waiting three seconds for a document chatbot may tolerate the pause. A character standing in front of them silently for three seconds can feel broken.

A realtime avatar RAG pipeline therefore has to optimise both:

• answer quality — retrieve the right evidence; and
• time to first speech — get enough evidence quickly enough that the character can start responding.

That can mean smaller models, smaller context windows, fast retrieval, aggressive caching and streaming the beginning of an answer before the entire response is complete.

Why a managed avatar platform can outperform a hand-built stack

It is easy to build a naive conversational avatar:

microphone → STT → LLM → TTS → avatar.

The difficult part begins when you want the experience to be fast, knowledgeable, stateful, observable and inexpensive in production.

A developer building RAG themselves now has another distributed subsystem to own:

• document processing;
• retrieval infrastructure;
• embedding costs;
• retrieval latency;
• reranking;
• prompt assembly;
• model selection;
• evaluation;
• monitoring; and
• continuous tuning as the knowledge base changes.

A competent team can absolutely build this.

But “we can build it” and “we should build it” are different questions.

The surprising outcome: building it yourself can be slower and more expensive

Home-grown RAG is often justified as a way to save money or retain control. Sometimes that is the right decision.

But an unoptimised implementation can easily erase those savings.

For example:

• poor retrieval may force you to compensate with a much larger LLM;
• oversized context can increase both latency and token cost;
• too many retrieval and reranking stages add round trips;
• inefficient ingestion can duplicate vectors and storage;
• lack of caching repeats expensive work; and
• weak evaluation can mean months of tuning without knowing which component is failing.

A team can spend months building a sophisticated custom pipeline and still end up with something slower or more expensive than a managed system whose retrieval, model selection and realtime path have already been optimised together.

That is not because RAG is impossible to build. It is because production optimisation is where much of the work lives.

Liforma's job is to absorb that complexity

Liforma's philosophy is that the creator should think in terms of:

• what the character should know;
• what sources define that knowledge;
• how the character should behave;
• what state matters in the experience; and
• what the user is trying to accomplish.

The creator should not need to become an expert in retrieval architecture just to make a support character knowledgeable about 50 help pages.

A Knowledge Pack makes the high-level workflow deliberately simple:

1. upload or connect the relevant source material;
2. attach the Knowledge Pack to the experience;
3. let Liforma retrieve the relevant material when the character needs it.

The underlying retrieval system can evolve without every creator rebuilding their experience.

RAG and state solve different problems

This distinction is important.

RAG answers: “What external knowledge is relevant to this question?”

State answers: “What is true in this experience right now?”

A training character might use RAG to retrieve the company's sales methodology while state tracks whether the learner has already discovered the customer's budget.

A website assistant might retrieve a returns policy while state records which product the visitor is currently considering.

Combining the two produces a much stronger character than simply giving an LLM a large prompt. See AI Character Memory vs State.

RAG is also different from long-term memory

A Knowledge Pack contains domain knowledge that many sessions or users may need.

Long-term memory stores relevant information about a particular user, relationship or previous interaction.

For example:

• Knowledge Pack: “The Business plan includes SSO.”
• Memory: “This visitor told us last week that SSO is a requirement.”
• State: “The visitor is currently comparing Business and Enterprise.”

A sophisticated character may use all three.

Should you always use a small model with RAG?

No.

RAG supplies knowledge. It does not automatically supply reasoning ability.

A larger model may still be preferable when the experience requires:

• complex multi-step reasoning;
• ambiguous planning;
• difficult tool orchestration;
• deep creative generation;
• subtle emotional understanding; or
• robust synthesis across many conflicting sources.

The right question is not “small model or frontier model?”

It is:

“What is the smallest, fastest model that reliably performs this task when given the right knowledge?”

You can still bring your own frontier model

Liforma does not force one answer to the model-selection question.

If a small managed model plus a Knowledge Pack is sufficient, Liforma Live gives you the simplest and lowest-friction path.

If your application needs a particular frontier LLM, you can use Liforma Relay and keep your own intelligence layer while still using Liforma's STT, TTS and avatar animation.

If you already use a native speech-to-speech stack such as OpenAI Realtime, Gemini Live or ElevenLabs ElevenAgents, Liforma Motion can remain only the visual layer.

The retrieval architecture and the visual avatar should not force you into one intelligence provider forever.

The best domain expert may be a small model with excellent evidence

For years, improving an AI application's answers usually meant choosing a larger model.

RAG changes that equation.

If an intelligent character has one narrow job, the most important question may not be how many parameters its language model has.

It may be:

Does the character have the right information, at the right moment, in a form the model can use quickly?

When the answer is yes, a small fast model can become an excellent domain expert — and the resulting avatar can respond faster, serve more users and cost much less.

That is the reason we think retrieval belongs inside the avatar platform rather than being another infrastructure project every developer has to build for themselves.