How RAG uses context to generate an answer

What is it?

After retrieval selects relevant chunks, RAG assembles context — the text the LLM reads alongside the user's question. The model generates an answer based on that context. This is how RAG produces grounded responses tied to your documents.

Why does it exist?

An LLM alone generates from its training and the prompt. Without retrieved context, it has no access to your private manuals at answer time. Context is the bridge between "what we found" and "what we say."

How does it work conceptually?

Retrieved chunks (context) + User question ↓ Prompt assembly ↓ LLM (within context window limit) ↓ Generated answer ↓ Optional source attribution

Key terms

• Context — the retrieved text (and sometimes system instructions) included in the prompt
• Context window — maximum tokens the LLM can process in one request (context + question + answer)
• Prompt assembly — formatting chunks and question into the structure the model expects
• Grounding — tying the answer to retrieved facts rather than free speculation
• Source attribution — showing which documents or chunks supported the answer
• Hallucination — the model states something not supported by sources or facts

Example

Retrieved context (from manual):

"At 500 operating hours: replace oil filter, inspect belts, lubricate bearings."

Question: "What maintenance is required after 500 operating hours?"

The LLM reads context + question and writes a clear answer listing those tasks. Sources may cite the manual document.

Why RAG reduces hallucinations but cannot eliminate them

RAG improves factual alignment by supplying real text. However:

• Wrong retrieval — irrelevant chunks mislead the model
• Incomplete retrieval — correct answer exists but was not retrieved
• Model behavior — LLMs may still over-generalize or ignore context
• Conflicting sources — multiple chunks may disagree

Engineering focuses on retrieval quality, chunking, and evaluation — topics in advanced levels of this learning path.

Where does it fit in a RAG pipeline?

Context assembly and generation are the final stages:

Retrieve → Assemble context → LLM generate → Answer (+ sources)Question ↓ Embedding ↓ Vector Search ↓ Relevant Chunks ──→ Context ↓ LLM ↓ Answer

How this relates to Noulek

Noulek Chat follows this pattern:

1. User message triggers retrieval from indexed Knowledge Base.
2. Retrieved chunks are included in the LLM request.
3. The model generates a conversational response.
4. When enabled, sources show document names or citations from retrieved content.

Configuration requirements:

• LLM connected (Settings → LLM)
• Vector DB connected for retrieval
• Documents in Ready status

Errors at generation (NX_LLM_CALL_FAILED) or retrieval (NX_RETRIEVAL_FAILED) prevent a proper grounded answer. See troubleshooting articles for LLM, Vector DB, and no results.

Noulek Search stops after retrieval and displays matches — it does not always invoke full LLM generation for every interaction. Chat is the primary conversational interface using context + generation.

For the product workflow, see Use Noulek Chat.