CCAR-P Claude Certified Architect - Professional Questions and Answers

Risk-stratified sampling concentrates limited reviewer capacity where error consequences and uncertainty are greatest. All low-confidence and high-impact cases should receive mandatory review, while random sampling of routine high-confidence outputs provides an unbiased signal for drift, unexpected failure modes, and overconfident errors. Complaint-only monitoring discovers defects after harm occurs. Reviewing only high-confidence cases systematically ignores the most dangerous outputs. Universal review provides maximum coverage but contradicts the stated throughput constraint and may create queues that undermine service objectives. The sampling policy should define confidence calibration, impact categories, escalation thresholds, review SLAs, and periodic adjustment based on observed error rates. Reviewer decisions should feed the evaluation dataset so monitoring quality improves over time.

Control A directly constrains the outbound action surface. An allow-list limits which tools and destinations may receive data, while per-call review introduces an approval boundary before information leaves the controlled environment. These measures therefore address data exfiltration through outbound tool calls, making Option C correct. Anthropic’s security guidance recommends least privilege, narrowly scoped permissions, sandboxing, and limiting access to sensitive information and actions so that a compromised agent can cause minimal damage. Mitigate Jailbreaks and Prompt Injections

Control B addresses model-version drift. Running a stable adversarial evaluation set after each version change creates a consistent comparison baseline and detects regressions that might otherwise remain invisible during routine testing. This correctly maps Control B to silent quality drift, making Option E correct. Anthropic’s migration guidance repeatedly calls for workload reevaluation because instruction following, style, reasoning, and agent behavior can differ between versions. Model Migration Guide

Control A may limit the consequences of successful prompt injection, but it does not prevent or directly detect adversarial instructions in retrieved content. Control B measures behavioral performance but does not impose outbound authorization controls. Each control must therefore be mapped to the risk it most directly mitigates.

Study Guide references/topics: Tool governance; least privilege; exfiltration controls; adversarial evaluations; model-change regression testing; defense in depth.

===============

A structured feedback session must enable informed review, traceable decisions, and accountable follow-through. Option B allows stakeholders to examine architecture diagrams, evaluation results, risk registers, or prototypes before the meeting. The focused agenda keeps discussion aligned with decisions that must be made rather than allowing the session to become an unstructured status conversation. Written decision records prevent different participants from leaving with incompatible interpretations. Assigning an owner and due date to every action converts feedback into executable delivery work. Options A, C, and D depend heavily on memory, provide inadequate preparation time, and create weak decision traceability. In-flight deployments particularly require documented feedback because accepted changes may affect scope, cost, risk, testing, and release timing.

Study Guide references/topics: Stakeholder feedback; decision records; action ownership; lifecycle alignment; communication planning.

Yes, Yes, No, Yes, No

More capable reasoning configurations generally require greater computation, produce higher latency, and may consume larger token or rate-limit allocations than lighter alternatives. This makes the first statement generally accurate, although actual performance must be measured under the organization’s specific provider and service tier.

Representative workload testing is essential because generic benchmark rankings do not establish performance on the organization’s prompts, data, edge cases, tools, or required output format. Anthropic recommends benchmark tests specific to the use case, using actual prompts and data, followed by comparison of accuracy, quality, edge-case behavior, and cost. Choosing the Right Model

The strongest model is not automatically the correct production choice. Model selection must balance capability, speed, cost, throughput, and operational constraints. Tiered routing implements this principle by serving predictable requests with an efficient model and escalating difficult cases to a more capable model. Anthropic documents multi-model strategies in which lower-cost workers handle most traffic while harder decisions are escalated.

Recency is also not a valid selection criterion. A newly announced model must pass the workload’s existing evaluation, security, latency, and cost gates before replacing a validated production model.

Study Guide references/topics: Model trade-offs; representative evaluations; tiered routing; escalation architectures; cost–capability optimization; model migration validation.

===============

A project-scoped CLAUDE.md converts repeatedly supplied information into persistent, version-controlled context. Claude Code loads applicable CLAUDE.md instructions at session initialization, allowing architecture principles, coding conventions, build commands, repository structure, and verification requirements to be shared consistently across engineers. Anthropic explicitly recommends using CLAUDE.md for information that developers would otherwise explain repeatedly; concise and specific instructions provide stronger adherence. Committing the file also enables normal review and change-control practices. Option C reduces typing speed but preserves the underlying waste and inconsistency. Options A and D remove useful capability or governance rather than correcting the workflow. The implementation should periodically review the file for obsolete, conflicting, or unnecessarily verbose instructions.

Study Guide references/topics: Claude Code project memory ; CLAUDE.md; persistent context; repository conventions; workflow standardization.

Threat assessment requires an inventory of what must be protected and who or what may threaten it. Asset identification should cover prompts, retrieved documents, credentials, model outputs, audit logs, tools, external systems, and sensitive business or personal data. Sensitivity classifications establish the potential confidentiality, integrity, safety, and availability impact of compromise. Threat actors and attack vectors must then be enumerated so likelihood and impact can be assessed against specific assets. Mitigation recommendations, residual-risk acceptance, ownership, and final documentation follow the analysis; they cannot be completed coherently before the threats are understood. Anthropic’s security transparency materials similarly emphasize maintaining threat models that reflect relevant attacker tactics and techniques. Anthropic Transparency Hub

Effective documentation begins with audience analysis and structural planning. Identifying the audience determines the required technical depth, terminology, decision context, and operational emphasis. Defining the questions the guide must answer prevents sections from becoming generic descriptions. The outline then establishes complete coverage across architecture overview, component responsibilities, interface contracts, dominant flows, operational runbooks, and known limitations. Translation occurs only after stable source content exists. Implementation-team validation also follows an initial draft because there must be material to review. Version control, reviewers, and cadence are important governance controls, but they do not replace the two content-planning activities required before section drafting. The guide should be planned around the decisions and tasks its intended readers must perform.

The retrieval pipeline is stated to be unchanged, so the first investigation should isolate changes in generation behavior and grounding instructions. Replaying the failing cases on the previous model version determines whether the regression follows the model upgrade or current model-prompt combination. Inspecting the system prompt establishes whether source-use rules, uncertainty behavior, citation requirements, or prohibitions against unsupported claims were removed or weakened. Increasing context or replacing the embedding model changes a component for which no failure evidence exists and may introduce additional noise. Lowering temperature may reduce variation but does not explain systematic contradiction of retrieved evidence. The investigation should compare identical requests, retrieved passages, prompts, and model versions before applying a fix. Define success criteria and evaluations

Drafting first-pass investigative reports from semi-structured incident logs for analyst review — Yes

Computing real-time fraud scores at sub-50-millisecond latency on a streaming pipeline — No

Long-context contract review with structured clause extraction and deviation flagging — Yes

Replacing a vector index for semantic retrieval over a multi-million-document corpus — No

Routing inbound support tickets into 30 categories with reasoning for the selected route — Yes

Claude is well suited to language-intensive analysis, summarization, extraction, classification, and drafting. Investigative-report drafting, contract analysis, and support-ticket routing all require interpretation of unstructured or semi-structured language and can incorporate human review. A sub-50-millisecond fraud-scoring path requires deterministic, specialized real-time processing and should not place LLM inference on the critical path. Claude also does not replace the retrieval index used to search millions of documents; it consumes the results produced by that infrastructure. The correct architectural decision separates tasks requiring semantic language reasoning from deterministic computation and information-retrieval infrastructure. Model selection and evaluation must then confirm that the chosen Claude model satisfies quality, latency, and cost requirements.

The evaluation metrics must map directly to the two stated requirements. Role-based access-control enforcement rate, tested against a red-team dataset, measures whether requests from unauthorized identities, roles, or access paths are consistently denied. It converts the privacy and authorization requirement into an observable security metric and exposes bypass conditions that ordinary functional tests may miss. Because the requirement says patient data must never reach unauthorized roles, the acceptance threshold should be effectively 100% enforcement across representative and adversarial cases.

Per-query token cost directly measures compliance with the financial constraint. The metric should include uncached input, cache reads and writes where applicable, output tokens, thinking tokens, and any other billable components attributed to the request. Results can then be compared with the defined ceiling across normal, boundary, and worst-case inputs.

Anthropic’s evaluation guidance identifies privacy preservation and price as distinct success criteria and emphasizes that criteria must be specific, measurable, and relevant to the application. Define Success Criteria and Build Evaluations

BLEU may assess textual similarity but does not test authorization or cost. Throughput and percentile latency are valuable operational metrics, yet neither directly answers the two requirements in the scenario.

Study Guide references/topics: Multidimensional evaluation; privacy preservation; authorization testing; red-team datasets; token-cost measurement; requirement-to-metric traceability.

===============

Transaction value is only one component of risk. Even a small refund can create fraud exposure, discriminatory outcomes, contractual violations, account compromise, policy inconsistency, or cumulative financial loss. Delegation should therefore depend on the nature, detectability, reversibility, and aggregate impact of failure—not a monetary threshold alone. Option B supports risk-stratified automation: routine, well-evidenced cases may proceed automatically, while anomalous, identity-sensitive, disputed, or policy-edge cases require review or escalation. Anthropic’s trustworthy-agent principles emphasize maintaining meaningful human control rather than imposing universal review or unrestricted autonomy. Option A is unnecessarily absolute; Option C prioritizes efficiency over governance; and Option D prohibits potentially safe, controlled automation without risk analysis.

Study Guide references/topics: Delegation competency; human control; risk classification; reversibility; cumulative exposure; escalation design.

Prompt caching reduces repeated input processing when a stable system prefix is reused, directly lowering input cost and processing latency. Model routing provides the second optimization: routine cases can use a faster, lower-cost model, while difficult cases retain access to the more capable model. The routing policy must be validated against representative evaluations to confirm that quality remains within the required threshold. Increasing max_tokens can increase output cost and generation time. Retrieving the complete corpus and adding examples to every request both enlarge the input, worsening latency and token consumption. Anthropic identifies prompt caching as a cost-and-latency optimization and recommends selecting a model according to the workload’s quality, speed, and economic requirements. Prompt caching ; reducing latency

Least privilege means granting the exact capabilities required for the defined responsibility—no more and no less. Option B permits the assistant to read the approved knowledge base and write only to the draft queue. Enforcing the allow-list at the orchestration layer creates a deterministic authorization boundary outside the model’s prompt and reasoning.

The rule should also be role-specific so different users or operational roles receive only the tools and data scopes they require. The knowledge-base tool should remain read-only, while the draft-queue tool should be restricted to draft creation rather than external sending, approval, or production publication.

Option A exposes every globally readable tool, creating unnecessary access to unrelated information. Option C relies on voluntary model behavior instead of enforceable authorization. A mistaken tool selection or successful prompt injection could therefore invoke an inappropriate capability. Option D minimizes permissions by eliminating the assistant’s required functionality and is not a viable least-privilege implementation.

Anthropic’s permission guidance distinguishes allow, ask, and deny rules and confirms that these controls are enforced by Claude Code rather than by model instructions. Configure Permissions

Study Guide references/topics: Least privilege; per-role authorization; orchestration-layer enforcement; capability scoping; read/write separation; tool allow-lists.

===============

Claude Code constructs the model context before the first user message. That initial context can include the system prompt, project instructions, subagent configuration, MCP server instructions, and tool definitions expressed as JSON schemas. When numerous MCP servers expose many tools and Tool Search is disabled, those schemas must be loaded upfront. A large collection of verbose tool descriptions and parameter schemas can therefore consume tens of thousands of tokens before the engineer enters any text.

Tool Search is specifically designed to prevent this form of context inflation. With Tool Search enabled, Claude initially receives only server instructions and compact tool-identification information; full tool schemas are deferred until a relevant tool is discovered and required. Anthropic consequently describes Tool Search as the mechanism for keeping MCP context consumption low as integrations scale.

Keyboard layouts, font rendering, and display scaling operate outside the language-model context and cannot generate API input tokens. Likewise, model behavior does not silently reserve 50,000 tokens for personal preferences. The diagnostic action is to inspect configured MCP servers and tool counts, determine whether Tool Search has been disabled, and remove or defer unnecessary definitions.

Study Guide references/topics: [MCP Tool Search and context scaling](https://docs.anthropic.com/en/docs/claude-code/mcp); MCP server configuration; subagent context budgeting; tool-schema optimization.

===============

The network-boundary requirement is a mandatory architectural constraint; therefore, Option D must be resolved before evaluating operational savings. Managed infrastructure can reduce responsibility for agent loops, sandboxes, session state, recovery, and orchestration, but those advantages are immaterial if financial data leaves the approved boundary. Anthropic describes Claude Managed Agents as a hosted agent harness and provides configurable data-residency controls . The architect must still validate inference location, session persistence, encryption, subprocessors, credential handling, auditability, and deletion behavior against the organization’s exact security policy. Options A, B, and C affect migration effort or economics, but they cannot override a non-negotiable data-governance requirement.

Study Guide references/topics: Architectural constraints; managed versus self-hosted agents; data boundaries; residency; security-gated decision-making.

The evaluation specification must exist before inference begins. Option D defines what the evaluation will measure and how results will be segmented. Representative cases measure normal workload performance, edge cases test boundary behavior, and adversarial cases examine safety, security, or robustness under hostile inputs.

Option B then supplies the correctly curated and labeled dataset corresponding to those slices. Labels, expected outcomes, grading rubrics, provenance, and dataset versions must be established before the deployment generates outputs. Otherwise, evaluators risk modifying criteria after seeing results and introducing confirmation bias.

Anthropic defines an evaluation as an input combined with grading logic used to measure success. Its evaluation guidance distinguishes tasks, trials, graders, traces, outcomes, and the harness that runs and aggregates them. Demystifying Evals for AI Agents

Option E occurs after model execution because outputs must exist before they can be scored. Option A follows scoring and aggregation. Option C is an iterative improvement step performed after failure cases become available, although rubric calibration may also be conducted on separate development data.

Study Guide references/topics: Evaluation specification; metrics and slices; representative and adversarial datasets; labeling; scoring; aggregation; release gates.

===============

Lifecycle continuity depends on preserving validated knowledge and feeding operational learning back into subsequent phases. Carrying the evaluation framework and reference set forward, as stated in Option B, creates a stable baseline across prompt changes, model migrations, retrieval adjustments, and architectural revisions. Rebuilding the evaluation system each time would undermine longitudinal comparison because changes in the test framework could be mistaken for changes in solution performance.

Option D establishes the second essential continuity mechanism: lessons from discovery, design, implementation, deployment, and production monitoring become explicit inputs to the next phase. This closes the feedback loop and prevents recurring defects, invalid assumptions, and operational findings from being lost at organizational handoffs.

Option A is overly mechanical. Stakeholders should receive communications relevant to their responsibilities and decision rights, not indiscriminate notifications at every transition. Option C confuses comprehensive archival with lifecycle continuity; retention must follow business, regulatory, security, and records-management requirements rather than an unconditional “archive everything” policy. Option E is directly contrary to iterative architecture. Decisions should be documented and governed, but material evidence or changed requirements must be allowed to reopen them.

Study Guide references/topics: Lifecycle phases; evaluation continuity; reference datasets; feedback loops; lessons learned; decision records; iterative architecture governance.

===============

Approval-rate parity measures whether groups receive positive recommendations at similar rates. It does not establish whether false approvals, false denials, sensitivity, specificity, or calibration are comparable across those groups. A system can therefore satisfy aggregate approval parity while imposing materially different error burdens on protected populations. The evaluation must include subgroup confusion matrices, false-positive and false-negative rates, calibration, intersectional analysis, and confidence intervals.

The 94 percent agreement rate measures fidelity to human underwriter decisions, not fairness. Human decisions are not automatically unbiased ground truth. If historical underwriting practices contain structural, procedural, or measurement bias, a model that reproduces those decisions accurately can reproduce the same bias. The reference labels must therefore be independently assessed for legitimacy, consistency, and potential discriminatory effects.

Nothing in the scenario establishes that protected groups were omitted, so D is unsupported. Similarly, sample size may require examination, but the scenario provides no statistical information proving that sample size is the principal deficiency. The evidence already contains two identifiable conceptual gaps regardless of sample size.

Study Guide references/topics: [Defining multidimensional evaluation criteria](https://docs.anthropic.com/en/docs/build-with-claude/develop-tests); fairness measurement; subgroup error analysis; label and benchmark bias; human-baseline limitations; governance evidence.

===============

The existing evaluation set does not represent the population or operating conditions the hiring-support system will encounter. Option B corrects this coverage defect by expanding geographic and tenure slices and requiring the system to be rescored before broader deployment.

Aggregate performance on one narrow group can conceal substantial differences among regions, experience levels, job families, languages, and other relevant cohorts. The expanded dataset should therefore support disaggregated metrics, not merely one combined score. It should also include representative, edge, and adversarial cases, with labels and grading procedures reviewed for consistency and potential bias.

Anthropic’s evaluation guidance states that evaluations should mirror the real-world task distribution and include edge cases. Success criteria should also be relevant to the application’s actual purpose and users. Define Success Criteria and Build Evaluations

Option A replaces one unrepresentative method with another. Option C deliberately narrows coverage further. Option D mistakes high performance on a restricted subset for evidence of generalization.

Because hiring is a consequential domain, the organization should combine representative quantitative evaluation, subgroup analysis, expert review, monitoring, access controls, transparency, and meaningful human responsibility for final decisions.

Study Guide references/topics: Dataset representativeness; evaluation slices; subgroup performance; bias detection; consequential-use governance; pre-release rescoring.

===============

Credentials must never be supplied as prompt content. Prompts can be retained in traces, debugging output, caches, or logs, and model output may inadvertently expose included secrets. Credentials should instead be resolved server-side through an approved secret store or delegated authentication mechanism. Option C is also a serious gap because post-response RBAC occurs after unauthorized data has already entered model context. Authorization must be enforced before retrieval and prompt construction. Options A, D, and E describe appropriate controls: least-privilege service identities, per-user scoped OAuth, and encrypted transport. Anthropic’s prompt-leak guidance recommends excluding unnecessary sensitive information, while Claude Code security relies on permission-based enforcement rather than natural-language instructions alone. Reduce prompt leak

An Architecture Decision Record is designed to preserve a significant architectural choice in a concise, durable, and reviewable form. The context explains the problem, constraints, and forces that shaped the decision. The decision section records the selected approach. Alternatives demonstrate that other viable options were evaluated, while consequences identify benefits, costs, limitations, operational obligations, and residual risks. The date and authors establish accountability and historical sequence.

This structure supports audit because a reviewer can trace why a model, integration pattern, retrieval architecture, security boundary, or deployment approach was chosen. It also supports onboarding by preventing new team members from interpreting intentional design choices as arbitrary implementation details.

A slide deck may communicate a decision at a meeting but is weak as the sole authoritative record when it lacks written rationale. A code comment is too localized and may not capture system-level implications or rejected alternatives. A verbal conversation creates no durable evidence and depends entirely on individual memory.

The ADR should be version-controlled, linked to affected requirements and implementation artifacts, assigned an owner, and reviewed when its assumptions or operating conditions materially change.

Study Guide references/topics: Architecture Decision Records; decision context; alternatives; consequences; auditability; onboarding; lifecycle review.

===============

Option A augments the existing review process without transferring final accountability to the model. Claude Code can inspect the change in repository context, identify logic defects, security weaknesses, regressions, and maintainability concerns, and publish structured findings for human evaluation. Existing branch protection then preserves required approvals and separation of duties. Automatic merging or disabling protection rules would convert an advisory capability into an uncontrolled authorization mechanism. Silent deletion would eliminate transparency and prevent authors from challenging or correcting the assessment. Anthropic’s Code Review documentation states that findings are posted as comments and do not approve or block the pull request, leaving the established review workflow intact. Claude Code Review

Documentation remains grounded only when the generating agent can inspect the authoritative implementation. A purpose-built subagent should receive scoped filesystem and code-search capabilities, identify the relevant modules, interfaces, configuration files, tests, and call sites, and generate documentation from that evidence. This approach minimizes reliance on training-data recall, which cannot represent the repository’s current private code or recent changes.

Subagents are useful because repository exploration occurs in their own context, allowing them to return focused findings or proposed edits without filling the main conversation with every file read. Their tool access should follow least privilege: read access to required paths, narrowly controlled write access, and explicit exclusions for secrets or unrelated repositories.

Generated documentation should enter the team’s established change-control process as a patch, branch, or pull request. Normal review, automated checks, link validation, example execution, and human approval provide accountability and prevent unverified material from being published. Direct publication bypasses these safeguards, while disabling repository tools makes factual grounding impossible.

Study Guide references/topics: [Claude Code custom subagents](https://docs.anthropic.com/en/docs/claude-code/sub-agents); [codebase exploration workflows] (https://docs.anthropic.com/en/docs/claude-code/common-workflows); source-grounded generation; least-privilege tools; human review and lifecycle controls.

The prompt contains mutually incompatible behavioral requirements. The architect should resolve the contradiction explicitly so the model receives one coherent grounding policy. If the application requires source-supported answers, the prompt should prohibit unsupported completion, permit an insufficient-evidence response, and require citations or supporting passages. Adding another priority instruction leaves the underlying conflict in place and creates ambiguity about when speculation is permitted. Increasing temperature makes behavior more variable rather than more correct. Retaining both instructions produces inconsistent outputs across requests and model versions. Anthropic recommends clear, direct, specific prompting and notes that conflicting project instructions can cause Claude to select among them unpredictably. Prompting best practices ; project-instruction troubleshooting

Category 1 tools are necessary for the agent to complete its approved responsibilities and must remain. Category 2 tools are also justified when they are regularly used and eliminate predictable hand-offs without expanding the agent beyond its defined operating role.

Categories 3 and 4 create capability bloat. A tool that is occasionally useful only for unrelated work does not belong to this agent’s responsibility boundary. A speculative tool has no validated requirement and adds attack surface, context consumption, authorization complexity, evaluation burden, and operational dependencies without demonstrated value.

Tool inclusion should be based on task traceability: every configured capability should map to a documented user journey, responsibility, permission scope, and evaluation case. Anthropic recommends least privilege so that a successful injection or model error can cause minimal damage. Mitigate Jailbreaks and Prompt Injections

The architect should periodically review tool-call telemetry and remove unused capabilities. New tools can be added when a validated workflow requires them, after security review and evaluation. The objective is not the smallest possible catalog regardless of usefulness; it is the smallest catalog that reliably completes the agent’s approved tasks.

Study Guide references/topics: Capability-bloat analysis; tool necessity; hand-off reduction; least privilege; attack-surface management; tool lifecycle review.

===============

A controlled prompt experiment must begin with a falsifiable hypothesis and a predefined primary metric. Option C prevents the team from examining results first and then selecting whichever metric makes the candidate look successful. The metric might measure task accuracy, rubric score, citation validity, escalation rate, latency, cost, or another criterion directly connected to the hypothesis.

Option A determines whether the experiment can detect a practically meaningful improvement. The minimum detectable effect expresses the smallest difference worth acting upon, while the power calculation determines the required sample size. Without this step, the experiment may be too small to detect a real improvement or unnecessarily large and expensive.

Random assignment should then distribute representative traffic between the control and candidate prompts while controlling model version, retrieval configuration, tool availability, and other confounding variables.

Options B and D occur after data collection. Option E follows the completed analysis and decision. The team should also define significance thresholds, stopping rules, guardrail metrics, exclusion criteria, and treatment of repeated observations before launch.

Study Guide references/topics: Prompt A/B testing; hypothesis definition; primary metrics; minimum detectable effect; statistical power; random assignment; decision sequencing.

===============

Robust guardrails require both preventive enforcement and continuous empirical validation. Per-role tool allow-lists satisfy the preventive requirement by restricting each authenticated role to explicitly authorized tools. Enforcement must occur in the orchestration or permission layer before execution; relying on the model to decide whether a tool call is authorized is not an adequate security boundary. Anthropic recommends allow-list-based validation and provides permission rules and pre-tool-use controls capable of denying calls before they execute.

Adversarial evaluation with regression tracking supplies the validation component. The evaluation set should include prompt injections, indirect injections, encoded instructions, multi-turn escalation, role-confusion attempts, and requests designed to trigger unauthorized tool use. Guardrail performance must then be measured across releases so that changes to prompts, models, tools, or orchestration logic do not silently reduce protection.

Centralized logging and user feedback are useful detective and lifecycle controls, but neither prevents an unsafe action nor proves guardrail effectiveness. Periodically refreshing refusal wording is also insufficient unless the revised behavior is evaluated against defined safety criteria. The strongest design therefore combines deterministic authorization with adversarial regression testing.

Study Guide references/topics: Claude Code permissions and pre-tool enforcement ; tool allow-list guidance ; adversarial evaluations; regression testing; least-privilege orchestration.

===============

A stable adversarial regression set provides a controlled basis for detecting behavioral change between model versions. Version-attributed results show whether the proposed upgrade improves, preserves, or degrades performance on the same safety-relevant cases. The evaluation must run before promotion so a regression can block deployment rather than becoming a production incident. Randomly replacing the test inputs removes comparability and makes score changes difficult to attribute. Disabling evaluation or relying on complaints converts a preventable release-control failure into reactive incident handling. Anthropic characterizes regression evaluations as tests of whether an agent still handles tasks it previously completed and recommends maintaining them continuously to detect backsliding. Demystifying evals for AI agents

Progressive tool discovery preserves the complete capability catalog while limiting initial context to a curated set of common tools and a discovery mechanism. When the task requires an additional capability, Claude searches the catalog and loads only the relevant tool definitions.

Anthropic’s Tool Search mechanism implements this pattern through deferred loading. The context initially contains the search tool and any deliberately non-deferred tools; relevant definitions are expanded only when discovered. This reduces token consumption and helps maintain tool-selection accuracy in large catalogs. Tool Search Tool

Option B adds a model call and still repeatedly processes all definitions, increasing latency and cost while potentially losing important schema details. Option C treats a larger context window as permission to waste context and does not address time-to-first-response overhead. Option D reduces capability breadth because the fixed five tools may be irrelevant to a given task.

The architect should provide clear tool and server descriptions, define which high-frequency tools remain immediately available, and evaluate discovery recall, selection accuracy, latency, and context savings across representative workflows.

Study Guide references/topics: Progressive discovery; deferred tool loading; MCP scaling; context optimization; tool-selection accuracy; on-demand capability loading.

===============

Expectation alignment begins with measured evidence. Option A gives stakeholders the median experience through p50 latency and the slower-tail experience through p95, then compares both with the agreed SLA. The measurements should reflect representative input sizes, concurrency, retrieval activity, tool calls, geographic routing, and cache conditions.

If the target remains infeasible after reasonable optimization, Option E establishes the correct governance response. The architect should collaboratively revise the SLA, scope, cost envelope, model choice, or user-experience design rather than silently accepting a breach or making unilateral changes. Any revision should document the business impact and accepted trade-offs.

Option B commits all resources before the problem’s severity, causes, and business priority have been established. Option C wrongly treats latency as uncontrollable; model selection, prompt length, retrieval depth, caching, streaming, and architecture can materially affect it. Option D introduces an unvalidated provider and could create quality, security, compliance, and integration regressions.

Expectation management is not merely communicating bad news. It requires transparent measurements, credible optimization options, explicit trade-offs, and jointly approved commitments that the production architecture can actually satisfy.

Study Guide references/topics: SLA alignment; p50 and p95 latency; stakeholder negotiation; evidence-based communication; architectural trade-offs; production constraints.

===============

Implementation-ready documentation must provide both executable specifications and preserved architectural reasoning. Option A supplies the contracts an implementation team needs to build components independently: configuration schemas, accepted inputs, produced outputs, validation requirements, and integration boundaries. These details prevent teams from inventing incompatible assumptions about data structures, tool behavior, error handling, or component ownership.

Option B complements the technical contracts with a decision log. Recording the selected design, underlying constraints, alternatives considered, and reasons for rejection enables implementers to distinguish deliberate architectural decisions from accidental omissions. It also prevents previously rejected approaches from being reintroduced without new evidence.

A high-level business narrative in Option C may explain purpose but cannot direct component-level implementation. Contact information in Option D provides an escalation route, but documentation is not sufficient if routine progress still depends on the architect’s availability. Option E records activity rather than a decision; a model list without the selected model and selection rationale leaves critical cost, latency, capability, and evaluation assumptions unresolved.

A strong handoff package should permit implementation, testing, and support teams to proceed while escalating only genuinely new architectural questions.

Study Guide references/topics: Architecture documentation; interface contracts; configuration schemas; Architecture Decision Records; implementation handoff; decision traceability.

===============