RAG and Grounding Pipelines for Microsoft AI-103

Microsoft AI-103 treats retrieval and grounding as core engineering skills rather than optional add-ons. Microsoft’s AI-103 skills guide expects candidates to ingest and index content, configure semantic, hybrid, and vector retrieval, enrich documents and media, build RAG ingestion flows, and connect the results to applications and agent tools. The general architecture in embeddings, vector databases, and RAG is useful background. For AI-103, focus on how Microsoft Foundry and Azure AI Search turn source content into grounded evidence that models and agents can use.

Start with source authority and access rules

Before indexing, identify which repositories are authoritative, which versions are current, and who may access the content. Retrieval quality is not enough if the index mixes obsolete policy with current policy or exposes restricted documents to the wrong user. Keep source IDs, timestamps, ownership, and access metadata through the pipeline. Those attributes support filtering, deletion, freshness, and provenance later.

Ingestion should preserve structure across source types

AI-103 explicitly includes documents, images, audio, and video in retrieval and grounding pipelines. Different media need different extraction or representation steps before they can be indexed. Do not convert everything into unstructured text if important layout, visual, or temporal information would be lost. Content Understanding and other Foundry tools can produce structured representations for downstream search and reasoning.

Use OCR and layout analysis when documents depend on structure. Scanned documents need OCR before text retrieval can work. Complex forms, tables, and multi-column layouts may need additional structure extraction so related fields remain connected. Validate extraction before blaming the search layer. If a heading or table row is reconstructed incorrectly, no ranking method can retrieve the intended meaning reliably.

Choose semantic, vector, and hybrid retrieval deliberately

Keyword (lexical) search can match exact terms, product codes, and names, while vector search identifies conceptually similar content. A hybrid query combines keyword and vector results, typically using rank fusion; semantic reranking can then refine eligible results by meaning. These are different retrieval stages, not interchangeable names for one method. Evaluate representative queries that include precise identifiers as well as paraphrased requests. Prefer the configuration that consistently returns authorized, authoritative evidence near the top, rather than choosing a technique because it has the newest label.

Enrichment should improve searchability, not add noise. Built-in or custom skills can extract entities, tags, layout signals, captions, or other fields that help search. Each enrichment step should have a reason: filtering, ranking, routing, source attribution, or downstream reasoning. Unnecessary enrichment increases processing time and can create misleading metadata. Test whether the added field improves retrieval or operations before making it a permanent pipeline stage.

Separate indexing quality from retrieval quality

An index can be healthy operationally while still producing weak search results. Monitor ingestion failures, document counts, vector generation, and index freshness separately from relevance metrics. AI-103 includes both data-ingestion quality and index-health monitoring. Operational health tells you whether the pipeline is running; relevance evaluation tells you whether it is useful.

Assemble grounded context with source evidence

A RAG flow should select enough evidence to answer the request without filling the model context with unrelated material. Preserve source boundaries and useful metadata. Long context is not a replacement for retrieval discipline. Irrelevant passages can distract the model or introduce contradictory statements even when everything fits inside the context window.

Grounded answers should retain source evidence. When the interface supports it, keep citations or source identifiers close to the claims they support. This lets users and downstream systems verify the answer and makes troubleshooting easier when two sources disagree. Grounding quality is partly a product-design question: the application should make insufficient evidence visible rather than forcing the model to produce an answer every time.

Connect retrieval directly to agent tools when appropriate. Foundry agents can use search as a tool inside a larger workflow. The agent may decide when retrieval is needed, then combine the evidence with another tool or action. Keep retrieval permissions aligned with the agent’s identity. A tool should not turn a broadly capable agent into a bypass around document access controls.

Evaluate RAG by stage

Use the approach in AI evaluation fundamentals to measure source extraction, retrieval recall, ranking, grounding, and final answer quality separately. When the response is wrong, you should know which stage failed. Include queries where no relevant source exists. A system that handles unsupported questions safely is stronger than one that scores well only when the answer is present.

Plan freshness, deletion, and production monitoring before launch

Source freshness needs a deletion plan, not just a refresh schedule. Azure Storage indexers can detect changed blobs, but deleted source items need a configured detection strategy to prevent orphaned search documents. Configure the policy before the first indexing run and test soft-delete timing; push-indexed documents instead need explicit delete operations. Verify removal by querying the index, not merely checking that the source is gone. Keep stable source IDs and a plan for historical versions that must remain accessible.

Monitor production grounding quality. Track queries with poor results, missing sources, unexpected rankings, or answers that users correct. Those cases can reveal corpus gaps, chunking problems, stale content, or weak ranking. Feed verified failures back into the evaluation suite. The retrieval system should improve from production evidence rather than relying only on a launch benchmark.

Refine queries, chunks, and source quality

Keep query transformation visible. User questions do not always map directly to the best search query. A grounding pipeline may normalize terms, expand synonyms, extract filters, or issue several searches. Record those transformations so retrieval failures can be diagnosed instead of appearing as mysterious model behavior. Query rewriting should be evaluated against known questions. More sophisticated rewriting is not automatically better if it drifts away from the user’s real intent.

Use source-specific chunking when content types differ. One chunking strategy rarely fits manuals, policy documents, tables, transcripts, and images equally well. Use structure-aware approaches where the source provides useful boundaries and preserve media-specific context when converting content into searchable representations. The goal is consistent retrieval quality, not a single global chunk size.

Deduplicate equivalent content before ranking. Enterprise repositories often contain copied or syndicated material. If several near-identical versions occupy the top results, the model receives less diverse evidence and may treat repetition as stronger support than it really is. Use canonical-source rules, version metadata, or deduplication to keep retrieval results informative.

Enforce the caller’s document permissions during retrieval, not as an instruction to the model. With query-time document-level access controls, permission metadata must be indexed and the caller’s authorized identity passed to search; test that denied documents never enter the result set. In preview targeted vector queries, a filter override can replace a global filter, including security trimming, unless those constraints are repeated. Test hybrid and vector paths with both authorized and unauthorized users before giving results to an agent.

A security filter is not an identity system. Microsoft’s Azure AI Search trimming pattern compares indexed user or group strings; it does not authenticate their owners. The application must derive caller identity and permitted groups from a verified session, enforce the resulting filter in every retrieval or fallback path, and never trust a group identifier supplied in question text. Native token-enforced document ACL mechanisms are separate preview features, not proof that ordinary identity-string filters enforce authorization by themselves. Revoke a test user’s group membership, rerun hybrid and vector searches, and confirm that disallowed passages never enter the model context.

Test evidence quality and fallback behavior

Measure citation correctness separately. A response can contain accurate claims but attach them to the wrong source. Test whether citations actually support the statement they accompany and whether the cited version is authoritative. Citation evaluation is especially important in regulated or policy-heavy applications where source traceability is part of the product promise.

A retrieval fallback must preserve authorization. If vector retrieval degrades, a lexical search or narrower repository is acceptable only when it enforces the same user permissions, source freshness, and citation contract. If no eligible evidence remains, stop or disclose the limitation rather than fabricate a grounded response. Log the selected path and whether adequate evidence returned. Test the fallback with denied-user queries and deleted-policy documents as well as ordinary service outages; a supposedly resilient retrieval path must not become a privacy bypass.

Monitor unanswered questions as a corpus signal. Repeated questions with no supporting evidence may reveal a real knowledge gap rather than a search defect. Track them and route them to content owners. RAG can only ground answers in information the organization actually maintains. This closes the loop between user demand and knowledge-base improvement.

Preserve provenance across retrieval paths

Use different retrieval paths for different source classes. A product catalog, long policy document, image archive, and meeting transcript may benefit from different indexing and retrieval strategies. Keep the user-facing grounding contract consistent while allowing source-specific ingestion behind it. This improves relevance without forcing every content type through one lowest-common-denominator pipeline.

Preserve a raw-source path for audit. Enriched or chunked representations are useful for search, but important workflows should still be able to trace a result to the original source. Store stable source identifiers and, where appropriate, offsets, page numbers, timestamps, or region metadata. This makes citations, debugging, and source corrections much easier.

Use separate evaluation sets for retrieval and grounded response quality. A retrieval benchmark should answer whether the right evidence appears, while a grounded-response benchmark should answer whether the model used that evidence correctly. Keeping those sets separate makes it easier to diagnose whether a change in search, chunking, ranking, or prompting actually improved the system. Use a smaller set of high-value end-to-end scenarios as a final integration check after both component tests pass.

A good RAG pipeline is an information system first

The model is only the final reasoning layer. Source governance, ingestion, extraction, indexing, retrieval, filtering, and evaluation determine which evidence the model sees. If you can trace a document from source to chunk to index to retrieved evidence to grounded answer, you are reasoning at the level AI-103 expects.

When an agent workflow uses retrieval, retain the query, source identifiers, returned passages and decision made from them. That trail allows production monitoring to distinguish stale or unauthorized evidence from a model-reasoning error. Use the resulting incidents to update separate retrieval and grounded-response evaluations, then retest the full request path before deploying a change.

  • img