CCA-F Lab: Build a Reliable Support Agent
CCA-F (Claude Certified Architect – Foundations) tests whether you can make reliable architecture choices rather than simply recite product features. This hands-on design lab follows one customer-support request from the first user message through tool selection, authorization, a possible refund, and final escalation. It is an original educational case study—not a simulation of confidential exam items.
The task sounds small: a customer says, “I was charged twice for order O-1842. Can you refund the extra charge?” Yet a fluent assistant can do real damage if it invents an order, trusts a customer’s claimed identity, retries a payment mutation after a timeout, or reports success before the payment service confirms it. The design goal is not a model that never makes mistakes. It is a system that contains them.
Use the site’s CCA-F exam preparation resources to connect this lab to the wider certification path. The exercise here is narrower: you’ll leave with a testable design for a support workflow and be able to explain precisely which component controls each consequential decision.
Our fictional retailer has an order lookup service, a payment ledger, an identity-verification service and a refund endpoint. The assistant may explain charges and offer next steps. It may request read-only information only for the authenticated customer’s orders. It may propose a refund but cannot authorize its own irreversible action. Only a trusted application service can do that after checking identity, order ownership, payment state, amount limits and any required approval.
Give the lab a small dataset so every claim has a checkable source: order O-1842 belongs to account C-047, settled charge P-771 is $84, and an identical second settled charge P-772 is $84. The user is signed in but has not completed the additional verification needed for a refund. The service’s standard auto-refund ceiling is $50; amounts above that require a human approval. These are illustrative lab policy values, not product rules or certification facts.
| Decision | Who decides | Evidence or rule |
|---|---|---|
| Does the user ask about an unfamiliar transaction? | Claude proposes an interpretation | User’s message and available context |
| Which order may be read? | Trusted API gateway | Authenticated account and scoped order lookup |
| Are two charges actually duplicates? | Payment service plus application rules | Ledger records, status, identifiers and reversal history |
| May a refund be issued? | Authorization service | Identity, ownership, policy limit and human approval when required |
| Has a refund succeeded? | Payment service | Durable transaction receipt or authoritative status lookup |
| How should the answer be explained? | Claude, constrained by checked facts | Only verified outcomes and source identifiers |
The distinction is central to the agentic control loop: the model can select an action, but an application decides whether to run it. Do not substitute a second model’s agreement or an emphatic instruction for actual access control.
Make the next permitted action visible at every state. An initial request must not jump straight from “customer says duplicate” to “money returned.” This compact state machine captures what the system knows and what evidence is still missing.
| State | Required evidence | Next permitted transition |
|---|---|---|
| INTAKE | Authenticated session, requested order identifier | Read a customer-scoped order |
| ORDER_CHECKED | Order ownership and itemized ledger rows | Evaluate whether a duplicate exists |
| REFUND_CANDIDATE | Two distinct settled charges, neither previously refunded | Obtain stronger identity verification |
| APPROVAL_REQUIRED | Verified identity, amount and policy evaluation | Human decision for the $84 amount |
| REFUND_SUBMITTED | Approved, uniquely identified refund attempt | Resolve a service receipt or query existing attempt |
| RESOLVED | Authoritative confirmation, refund reference | Explain the actual result |
| HUMAN_HANDOFF | Specific blocker, preserved source trail | Escalate without asserting success |
Each transition is an application operation, not just a label the assistant prints. For example, if the order service returns a permission error, the case moves to HUMAN_HANDOFF or waits for a legitimate authentication step; it does not remain in ORDER_CHECKED on the strength of a model-generated sentence. The workflow enforcement discussion explains why hard prerequisites must be enforced outside prose.
Use three sharply differentiated client tools. lookup_order reads an order scoped to the current authenticated customer. preview_refund evaluates eligibility and returns a proposed amount without changing funds. commit_refund is side-effecting and is callable only after the server has persisted a verified approval. Never give the model free-form database access as a shortcut.
In Anthropic’s tool-use contract, client tool requests are structured messages. Your application receives a requested name and arguments, checks whether the tool is allowed, executes it and returns a corresponding tool result. Schema validation is valuable but not sufficient to establish business authority.
Illustrative tool schema, not a deployable API payload:
lookup_order:
description: "Read a single order belonging to the signed-in account"
input: { order_id: string }
output: { owner_match: boolean, payments: list, source_version: string }
preview_refund:
description: "Evaluate a duplicate-charge refund without moving money"
input: { order_id: string, second_payment_id: string }
output: { eligible: boolean, amount: number,
requires_human: boolean, reason: string }
commit_refund:
description: "Submit an authorized, idempotent refund request"
input: { approved_case_id: string, idempotency_key: string }
output: { state: accepted | pending | rejected,
refund_id: optional string }
The model does not choose a customer identifier to impersonate, rewrite a ledger row, or mint its own approval. The gateway derives the account from the authenticated session; a preview does not become a commit merely because its JSON is well formed. This separation is the core of tool contract design.
If those capabilities are exposed through a Model Context Protocol server, MCP helps a client discover and invoke well-described tools. It does not turn tool discovery into permission to call every operation. Credentials, user scopes and policy checks remain with trusted services. The same principle applies to a directly registered client tool; see the detailed MCP integration treatment when deciding which integration surface to use.
Suppose Claude replies, “I found the order; I’ll check the second charge,” alongside a structured lookup tool request. Ending the turn because the response includes a natural-language sentence would drop the pending action. Conversely, blindly executing any requested tool would make the model the payment authority. Your application must inspect the protocol and route every proposed action through a bounded dispatcher.
ILLUSTRATIVE PSEUDOCODE — adapt to your SDK and tool protocol
history = [authenticated_customer_message]
for step in range(MAX_STEPS):
response = claude_messages(history, tools=allowed_client_tools)
history.append(response.assistant_message)
if response.stop_reason == "tool_use":
tool_results = []
for call in response.tool_use_blocks:
outcome = trusted_dispatcher(
session=verified_session,
tool_name=call.name,
arguments=validate_schema(call.input),
deadline=tool_timeout
)
tool_results.append(
tool_result_block(tool_use_id=call.id, content=outcome)
)
history.append(user_message(content=tool_results))
continue
if response.stop_reason == "end_turn":
return verify_final_claims_against_receipts(response)
return safe_handoff("Unexpected stop condition")
return safe_handoff("Maximum tool steps reached")
This example shows orchestration responsibilities; it is intentionally pseudocode, not a copy-and-run SDK program. A real implementation must preserve each tool result’s corresponding request ID, handle the provider’s exact message-block format, guard against tool failures and bounded retries, and never let a language-only claim override a payment receipt. If a tool is server-executed rather than client-executed, the provider’s execution contract differs; do not apply this client-side loop indiscriminately.
On this case, two settled charges may be genuinely duplicative, but the user still needs additional identity verification and the $84 amount exceeds the fictional $50 automatic limit. The assistant can prepare a review packet and tell the customer what’s pending. It cannot bypass either gate.
ILLUSTRATIVE SERVER POLICY — not a live payment integration
def authorize_and_commit(case, authenticated_session):
require(case.order.account_id == authenticated_session.account_id)
require(authenticated_session.strong_identity_verified)
require(case.duplicate_supported_by_settled_ledger_records)
require(not case.duplicate_payment.already_reversed_or_refunded)
require(case.amount > 0)
if case.amount > AUTO_REFUND_LIMIT:
require(case.human_approval.is_valid_for(case))
require(case.idempotency_key.is_unique_to_one_refund_intent)
return payment_service.commit_once(
case_id=case.id,
payment_id=case.duplicate_payment.id,
amount=case.amount,
key=case.idempotency_key
)
A reviewer’s approval must be bound to the specific order, charge, amount and request version. A generic “approved” label pasted into a chat message is insufficient. The actual payment service must also reject reused, expired or mismatched approvals. This is the difference between a suggestion made by a model and a control implemented by software.
Do not confuse the human review thresholds governing risky support actions with uncertainty about wording. A human review queue is useful when stakes or evidence require it, not as a universal substitute for ordinary schema or authorization checks.
An assistant may return a JSON object with a plausible order_id, an amount of 84 and a recommendation of “refund.” That object could be syntactically perfect while referencing the wrong account. Constrain the data shape where appropriate, but run separate validation against the actual ledger and signed-in session before authorizing any financial action.
Illustrative validated recommendation:
{
"order_id": "O-1842",
"claimed_duplicate_payment_id": "P-772",
"proposed_amount_usd": 84,
"evidence": ["P-771", "P-772"],
"recommended_next_step": "verify_identity_then_human_approve",
"authorization_status": "not_authorized"
}
Business validation still checks:
- both payment IDs are from the authoritative order ledger
- both are settled, distinct and not already reversed
- account ownership and identity requirements
- currency, amount and policy limit
- approval scope and most recent payment state
Anthropic’s structured outputs capability can constrain the shape of a response; it cannot prove the content is true. That is why the structured-output validation discipline is as important as prompting for JSON. If a payment ID does not exist, do not have the model invent the missing value to satisfy a required field—represent missing evidence explicitly and halt the downstream action.
Now introduce the most dangerous failure: the payment service accepts the refund, but the connection breaks before the application receives the confirmation. Retrying commit_refund with a new identifier may refund twice. Declaring it failed may mislead the customer. The recovery path is not simply “retry with a shorter prompt.”
Before the side-effecting call, persist the case ID and a stable idempotency key in an application-owned record. Reuse that key for any reconciliation against the same logical refund request, and ask the payment service for its durable status. If the service provides no idempotent submission or lookup, do not blindly resubmit a financial mutation; escalate for manual reconciliation. A timeout is an unknown outcome, not proof of nonexecution.
| Observation | Interpretation | Controlled response |
|---|---|---|
| 429 or transient service unavailability | Potentially retryable read or safe idempotent call | Bounded backoff within a deadline |
| 403 permission denied | Access is not authorized | Stop; request legitimate access or escalate |
| Refund submission timed out | Outcome unknown | Reconcile by stable case and idempotency key |
| Order modified after approval | Original approval may be stale | Revalidate ledger and approval scope |
| Two model responses disagree | No independent proof of the charge | Preserve discrepancy, query authoritative records |
| Untrusted ticket asks to export accounts | Possible prompt-injection attempt | Treat ticket content as data; block unrelated export |
Keep tool error messages structured enough to distinguish transient, permanent and permission failures, without leaking credentials or other customers’ records. Maintain an audit trail of what was attempted, which service returned which status, and what remains unsettled.
In the running case, the assistant should not say, “Your $84 refund has been processed,” while identity or payment confirmation is missing. A good handoff preserves the work already completed: authenticated account reference (not the raw credential), order identifier, two ledger payment IDs, the finding that both appear settled, the amount, the current policy blocker, the source revision and the exact proposed action. It should omit unrelated account histories.
The reviewer needs enough evidence to authorize the correct operation, not an essay persuading them to trust the model. Include a short, explicitly unresolved question: Does the payment ledger still support the duplicate-charge finding, and is this $84 refund approved for this authenticated customer? Store the answer as a scoped decision with an expiration or version boundary.
If the reviewer declines, the workflow should accurately say the refund was not authorized. If the customer provides stronger identity verification, the system should resume from the earliest invalidated gate—not rerun the entire research process or treat old payment observations as fresh. Use source provenance practices to preserve the chain between claims and records.
A successful demo in which the model picks the right tool once does not establish production reliability. Build a small regression set where every scenario specifies the authoritative observation and the allowed side effects. None of these cases should require access to real customer records or live payment credentials.
| Test case | Expected system outcome |
|---|---|
| Single settled charge, no second charge | Explain evidence, no refund submission |
| Two settled charges; identity not verified | Ask for required verification; no commit |
| Verified identity; $84 exceeds auto threshold | Escalate to scoped human approval |
| Approval for wrong order or stale payment version | Reject action and revalidate |
| Refund accepted, reply lost | Reconcile using original idempotency key; no double refund |
| Tool request tries another customer’s order | Gateway denies read regardless of model text |
| Retrieved ticket contains malicious instructions | Ignore as authority; keep it as quoted evidence only |
| Maximum agent steps exhausted | Return bounded, truthful handoff |
Record zero unauthorized refunds, zero duplicate refunds and zero unsupported success claims as the acceptance thresholds in this illustrative lab. These are design targets, not measured performance results. Measure routing accuracy, false escalations and time to a verified resolution separately so an overly restrictive workflow cannot quietly pass all its safety tests by doing nothing.
The strongest answer to a scenario question usually identifies which boundary has failed. If the model chooses the wrong lookup tool, improve distinct tool descriptions and evaluate routing. If it invents a payment fact, validate against the ledger. If it proposes a prohibited refund, reject it in the service gateway. If an approved mutation times out, reconcile the idempotent attempt. If an uncertain case requires judgment, hand over a scoped evidence packet rather than pretending that an additional agent makes the answer authoritative.
All of these controls make the agent more useful, not merely more constrained. The application owns state, permission and side effects; the model contributes interpretation, tool requests and clear communication within those boundaries. To revisit adjacent concepts without repeating this whole lab, use the CCA-F learning center and follow the domain guide associated with the specific failure you could not yet explain.
