A PRACTICAL FIELD GUIDE
Recognize the failure.
Design the boundary.
Nineteen patterns to examine in an agent workflow. This is an editorial catalogue, not a measured ranking or an exhaustive taxonomy.
19 entries
01Tool overreachExecution
- What goes wrong
- A useful task expands into actions outside the intended scope.
- Warning sign
- The plan introduces new tools, destinations, or writes without a corresponding policy decision.
- Control to design
- Bind permission to the exact tool, arguments, resource, and tenant. Route governed calls through the executor.
02Cloud and IAM changesAuthority
- What goes wrong
- An agent changes infrastructure or permissions using a broadly privileged identity.
- Warning sign
- A read or planning task begins creating roles, grants, or cloud resources.
- Control to design
- Separate planning from apply. Use limited credentials and an explicit review path for privilege changes.
03Credential exposureData
- What goes wrong
- Secrets enter prompts, logs, tool output, or exported artifacts.
- Warning sign
- Raw tokens appear in debug output or a generated configuration file.
- Control to design
- Keep credentials in a dedicated custody layer and limit their delivery to the operation that needs them.
04Memory contaminationData
- What goes wrong
- Context from one task or tenant influences another task’s decisions.
- Warning sign
- An answer cites information the current task never supplied or authorized.
- Control to design
- Scope sensitive context to the task and tenant, and define when it must be discarded.
05Hallucinated planningEvidence
- What goes wrong
- A plan relies on capabilities, access, or dependencies that have not been established.
- Warning sign
- Steps assume a connector, service, or permission exists without checking it.
- Control to design
- Confirm prerequisites and inspect a local or simulated path before consequential execution.
06Unverified completionEvidence
- What goes wrong
- A success message is emitted without evidence that the requested result exists.
- Warning sign
- The report says “done” while the artifact, response, or deployed state is missing.
- Control to design
- Require task-specific evidence and keep attempted, completed, and verified states distinct.
07Broken generated codeExecution
- What goes wrong
- A plausible patch breaks a behavior outside the narrow example used to write it.
- Warning sign
- Checks are skipped or only demonstrate the new happy path.
- Control to design
- Use an isolated change, focused checks, and a separate review appropriate to the risk.
08Unbounded retry loopsExecution
- What goes wrong
- Repeated retries consume resources or repeat a side effect without a stopping rule.
- Warning sign
- The same failure is retried without changing assumptions or checking prior results.
- Control to design
- Set task-specific limits, check idempotency, and stop when the permitted attempt budget is exhausted.
09Misleading benchmarksEvidence
- What goes wrong
- A narrow or historical measurement becomes a general product promise.
- Warning sign
- Performance numbers omit hardware, workload, harness, or raw results.
- Control to design
- Keep the measured artifact and environment attached to every quantitative claim.
10Unclear authorityAuthority
- What goes wrong
- A task description is treated as permission for every possible action.
- Warning sign
- The proposer also decides which permissions or approvals it needs.
- Control to design
- Record the permitted scope and the accountable reviewer before a consequential workflow begins.
11Missing audit evidenceEvidence
- What goes wrong
- A consequential action cannot be reconstructed from its request, policy, and result.
- Warning sign
- Logs contain status text but no bound request, policy version, or decision context.
- Control to design
- Choose the audit mode and retention controls explicitly, then verify the evidence path.
12No recovery pathExecution
- What goes wrong
- A change is easy to issue and difficult to undo.
- Warning sign
- A delete, overwrite, or deploy step has no verified recovery procedure.
- Control to design
- Prefer reversible stages and review recovery requirements before committing the change.
13Approval bypassAuthority
- What goes wrong
- A reviewed proposal changes before the eventual action is executed.
- Warning sign
- The approval refers to a description rather than the exact version, arguments, or artifact.
- Control to design
- Bind approval to the intended action and re-evaluate when those inputs change.
14Multi-agent confusionAuthority
- What goes wrong
- Several agents share work without clear ownership or review responsibilities.
- Warning sign
- Two agents claim the same task or treat another agent’s output as authorization.
- Control to design
- Separate proposer, reviewer, and executor responsibilities and retain the handoff evidence.
15Context driftData
- What goes wrong
- An agent continues using an earlier objective after the task has changed.
- Warning sign
- The next action is sensible for the old brief but outside the current one.
- Control to design
- Keep a current task brief and re-check the authorized scope at important transitions.
16Prompt injectionData
- What goes wrong
- Retrieved text is treated as an instruction that outranks the task or policy.
- Warning sign
- A document or tool result asks the agent to change its role or send data elsewhere.
- Control to design
- Treat retrieved content as data. Keep authorization outside that content and scope tool access.
17Repository damageExecution
- What goes wrong
- Automated edits overwrite unrelated work or widen beyond the intended change.
- Warning sign
- Large rewrites, destructive cleanup, or unrelated files appear in the diff.
- Control to design
- Isolate the change, inspect the diff, and preserve a recoverable checkpoint.
18Unverified deploymentEvidence
- What goes wrong
- A build or upload is mistaken for a successfully running release.
- Warning sign
- The deployment state, served artifact, or relevant behavior has not been observed.
- Control to design
- Keep build, publication, runtime checks, and production-readiness claims separate.
19Compliance overclaimsEvidence
- What goes wrong
- A checklist or generated report is presented as independent certification.
- Warning sign
- A mapping, fixture result, or policy pass is described as regulator approval.
- Control to design
- State the evidence type, its scope, and the remaining assessment responsibilities.
No matching entries.
Try another term or choose All categories.