Amazon AWS AIP-C01: RAG

Retrieval-augmented generation (RAG) is one of the most important application patterns in AWS Certified Generative AI Developer – Professional because it connects foundation models with external knowledge. Instead of relying only on what a model learned during pretraining, a RAG application retrieves relevant information at request time and adds that evidence to the model context.

For Amazon AWS AIP-C01, RAG should be understood as an end-to-end system: documents or data are prepared, indexed, retrieved, authorized, inserted into prompts, evaluated, observed, and maintained over time. Weakness at any stage can make a strong model produce a poor answer.

RAG separates knowledge retrieval from generation

The basic flow is retrieval first, generation second. A user asks a question, the application searches an external knowledge source, relevant passages are added to context, and the foundation model generates a response using that evidence.

RAG architecture is valuable when knowledge is private, changes frequently, or is too large to place permanently inside a prompt.

This separation also makes the system easier to update: documents can change without retraining the foundation model.

This separation also creates two evaluation surfaces. Retrieval can be wrong even when generation is excellent, and generation can ignore or distort correct retrieved evidence.

Build diagnostics that expose both stages. Store retrieved chunk identifiers and scores alongside model output where privacy allows so developers can see which layer failed.

RAG can also improve transparency when citations are connected accurately to the claims being generated.

Ingestion and chunking determine what can be retrieved

Documents must be parsed into retrievable units. Chunk size, overlap, headings, metadata, tables, and document structure all influence what information remains together.

Chunks that are too large can dilute similarity and consume context. Chunks that are too small can lose the surrounding explanation needed to answer correctly.

Use the document type and question pattern to guide chunking rather than adopting one fixed size for every source.

Preserve metadata such as source, date, document owner, tenant, security label, and section. Metadata can improve filtering, freshness, authorization, and later investigation.

Tables, code, PDFs, and scanned documents may require different parsing strategies. A generic text splitter can destroy structure that carries meaning.

Reprocess documents when the parser, chunking rule, or embedding model changes materially.

Track document version and source timestamp so the application can prefer current material and explain where an answer came from.

For frequently changing sources, define how quickly updates must enter the index and how stale chunks are retired.

Embeddings create semantic retrieval

Embedding models convert content into vectors that represent semantic relationships. Query embeddings can then be compared with stored vectors to identify related chunks.

Embedding choice should fit language and content. Code, legal text, multilingual documents, and general prose may not perform equally with the same embedding model.

Metadata filters can narrow retrieval by tenant, product, date, document type, access group, or other structured attributes before or after vector similarity.

Store the embedding-model version with the index because switching models can make old and new vectors incompatible or inconsistent.

Test multilingual and domain-specific questions explicitly if the application serves them. Average benchmark performance may hide poor retrieval for one important user group.

Use metadata or hybrid retrieval when exact identifiers, product codes, or legal clauses are not captured well by semantic similarity alone.

Vector stores are part of the operating system

The vector store must support the scale, latency, filtering, durability, and operational needs of the application. AWS can support RAG through several data technologies depending on the architecture.

Do not treat the vector database as a passive index. It needs backup, update, deletion, monitoring, access control, and lifecycle planning.

When source documents are removed or permissions change, the retrieval layer must reflect that change instead of continuing to expose stale or unauthorized chunks.

Plan tenant isolation against a caller who can change a tenant identifier in a retrieval request. A shared vector index can be efficient, but metadata filters and namespaces are not a substitute for authenticating the caller and deriving their permitted scope in trusted application code. Enforce document authorization before retrieved content reaches the model, and test permission revocation and caches. With Bedrock Managed Knowledge Bases, ACL-aware retrieval filters against a supplied identity; it does not authenticate that identity, and real-time permission verification depends on the connector. S3 and custom ACL sources do not offer that real-time verification.

Monitor index growth and query latency as documents scale. Retrieval that is excellent at ten thousand chunks may behave differently at tens of millions.

Deletion needs verification because privacy or retention rules may require removing derived embeddings as well as original documents.

Choose operational metrics such as index size, ingestion lag, query latency, retrieval error rate, and unauthorized-access attempts. Retrieval infrastructure needs the same observability discipline as any other production datastore.

Plan rebuild strategy. If embeddings or chunking change, the team should know how long a reindex takes and how the application behaves during migration.

Retrieval quality should be evaluated separately

A model cannot produce a grounded answer if the necessary evidence was never retrieved. Evaluate retrieval with known questions and known relevant documents before blaming the generator.

Useful measures can include recall of relevant chunks, precision, ranking quality, and whether the retrieved context contains enough information to answer.

Reranking or hybrid search that combines semantic and keyword signals can improve results for some domains.

Create a labeled evaluation set with query, expected relevant source, and acceptable alternatives. This turns retrieval tuning into measurable work.

Analyze failures by cause: missing ingestion, poor chunking, embedding mismatch, weak query, metadata filter, ranking issue, or stale source.

Use reranking only where it creates measurable quality improvement relative to extra latency and cost.

Track failures where the correct document was retrieved but ranked too low to enter the context window. That is different from the document being absent entirely.

Evaluate difficult query types such as abbreviations, product codes, multi-part questions, and vague natural language if users are likely to ask them.

Use user feedback carefully because users may rate the final answer poorly even when retrieval was correct and generation was the real problem.

Track whether poor answers cluster around particular document types, business units, or languages. Retrieval quality can be uneven even when the overall average looks acceptable.

Segmented evaluation often reveals that one parser, source, or metadata pattern needs different treatment.

Prompt construction turns retrieved evidence into model context

The application needs clear instructions that distinguish trusted application policy, the user request, and retrieved evidence.

Prompts can tell the model to answer only from provided sources, cite evidence, disclose uncertainty, or refuse when support is insufficient.

The prompt design should be evaluated with the retrieval system because changing context format can alter answer quality even when the documents are identical.

Delimit retrieved evidence clearly so the model can distinguish source content from application instructions.

Prompt the model to disclose when the provided context is insufficient. A refusal or request for clarification is often safer than a confident answer built from weak evidence.

Use structured citation formats if downstream software needs to verify or display sources reliably.

Control context length deliberately. Too many retrieved chunks can crowd out instructions or increase cost without improving answer quality.

Use citations only when the application can map them reliably to the exact retrieved source; fabricated or weak citations reduce trust.

Authorization must survive the retrieval layer

A user should not gain access to a document merely because the vector search found it semantically relevant. Identity and document authorization must be enforced before protected content enters model context.

Multi-tenant systems need especially strong boundaries around indexes, metadata filters, roles, and retrieval queries.

Logs should show enough about the retrieval decision to investigate mistakes without exposing sensitive document content unnecessarily.

Authorize before generation, not after the model has already seen protected content. Output filtering cannot reliably undo a data exposure that entered context.

Use user or workload identity all the way through retrieval where practical so access decisions reflect the requesting principal.

Audit denied and suspicious retrieval attempts as security signals.

Security tests should include a user who is allowed to see one document but not another semantically similar document.

Verify that cache, conversation history, or generated citations do not leak content from a previous authorized session into a later unauthorized one.

Tenant isolation should be tested under both normal retrieval and error paths.

Indirect prompt injection is a RAG-specific security risk

Retrieved documents are untrusted input. A malicious or compromised document can contain instructions intended to override application policy or manipulate tool use.

The security of GenAI and agentic systems therefore requires instruction hierarchy, content isolation, tool boundaries, validation, and runtime monitoring.

Do not assume a document is safe because it came from an internal repository.

Content can include hidden or misleading instructions intended for the model. Treat retrieved text as data, not authority.

Use tool and action controls outside the model so injected content cannot simply instruct the application to take privileged action.

Evaluate security with malicious documents as well as malicious user prompts.

Test documents containing hidden instructions, misleading role labels, and requests to reveal system prompts or invoke tools. Security evaluation should reflect the actual content sources the application will ingest.

Keep action authorization outside the model so retrieval content cannot grant itself new permissions.

RAG is not always better than fine-tuning or normal search

Use RAG when the problem is external knowledge that needs to stay current or private. Use prompt engineering when instructions are the main weakness. The fine-tuning vs RAG decision changes when repeated specialized behavior needs to be learned and the lifecycle cost is justified.

Some tasks may not need generation at all. If users only need exact document retrieval, search can be simpler, cheaper, and more reliable.

AIP-C01 questions often reward this trade-off thinking rather than a “RAG everywhere” mindset.

RAG adds moving parts: ingestion, index, retrieval, prompt context, authorization, evaluation, and freshness. That complexity should be justified by the knowledge requirement.

Fine-tuning may improve behavior but does not create a dynamic knowledge store. Normal search may be better when users need exact documents instead of synthesized answers.

Choose the smallest architecture that meets quality, freshness, security, and cost requirements.

Monitor ingestion lag, retrieval quality, answer quality, latency, token usage, cost, authorization failures, safety events, and user feedback.

Version prompt templates, chunking strategy, embedding model, reranker, model, and evaluation set where changes can affect behavior.

AIP-C01 treats RAG as production engineering. The strongest preparation is to build a small RAG system, break retrieval or permissions deliberately, and explain which evidence reveals the failure.

Include operational ownership in the decision. RAG requires someone to maintain ingestion, index freshness, retrieval evaluation, permissions, and source lifecycle.

A simpler architecture may outperform RAG organizationally if the team cannot operate those dependencies reliably.

Architecture quality includes maintainability, not only benchmark answer quality.

Factor user experience into the decision. A synthesized answer may be useful for discovery, while regulated or expert users may prefer direct source documents so they can interpret the evidence themselves.

Revisit the choice when source freshness, user need, or operating capacity changes.

Reevaluate the pattern when the knowledge source, user need, or operating team changes; architecture that was appropriate at launch may become unnecessarily complex later.

Choose deliberately and keep the trade-off documented.

Prefer the smallest maintainable system that meets the requirement.