# FAILURE.md — AI Agent Failure Mode Protocol (Full Specification) **Home:** https://failure.md **Repository:** https://github.com/Failure-md/spec **Related Domains:** https://throttle.md, https://escalate.md, https://failsafe.md, https://killswitch.md, https://terminate.md, https://encrypt.md, https://encryption.md, https://sycophancy.md, https://compression.md, https://collapse.md, https://leaderboard.md --- ## What is FAILURE.md? FAILURE.md is a plain-text Markdown file convention for defining the **four failure modes an AI agent can encounter** — graceful degradation, partial failure, cascading failure, and silent failure — along with detection signals and response procedures for each. Every failure is logged with full context: when it occurred, which mode, which component, and what action the agent took. ### Key Facts - **Plain-text file** — Version-controlled, auditable, co-located with code - **Declarative** — Define policy, agent implementation enforces it - **Framework-agnostic** — Works with LangChain, AutoGen, CrewAI, Claude Code, or custom agents - **Reliability-focused** — Provides graduated response: from continuing gracefully to complete circuit breaker - **Companion to FAILSAFE.md** — FAILURE.md defines what failures exist and how to respond; FAILSAFE.md defines the safe recovery state --- ## How It Works ### The Four Failure Modes Not all failures are equal. FAILURE.md classifies failures into four distinct modes, each with different detection signals and response procedures. #### Mode 1: Graceful Degradation **Definition:** A non-critical component becomes unavailable, but the agent can continue operating with reduced capability. **Examples:** - Search API is down; agent skips search enrichment and continues with base response - Image generation tool is unavailable; agent provides text-only response instead - Analytics service is slow; agent skips detailed metrics and continues - Recommendation engine is unreachable; agent provides default recommendations **Detection:** - Component health check fails - Component API returns 503 Service Unavailable - Component timeout is exceeded **Response:** - Action: `continue_degraded` (proceed without component) - Log Level: `WARNING` (important but not critical) - Notify: `false` (no alert to operator unless persistent) - Escalate: `never` (graceful degradation is acceptable) **Example Response:** ``` Agent attempted to call SearchAPI, got 503. Graceful degradation: skipping search enrichment. Continuing with base response. [WARNING log] ``` #### Mode 2: Partial Failure **Definition:** A critical component fails, but the agent can detect it and actively route around it (retrying, using a replica, queuing for later). Transient failures (network hiccup, temporary unavailability) should resolve with retry. **Examples:** - Network timeout on API call; retry with exponential backoff - Database connection pool exhausted; wait for connection to free up, then retry - Rate limit hit on external API; backoff and retry after cooldown - Disk write failed; retry operation **Detection:** - Component API returns 5xx error (transient) - Network timeout or connection refused - Temporary resource exhaustion (connection pool empty) **Response:** - Action: `isolate_and_route_around` (retry with backoff, or use backup) - Max Retries: `3` (default; customize per use case) - Retry Backoff: `[5s, 15s, 60s]` (exponential; default; customize per API) - Escalate: `ESCALATE.md` (after max retries exhausted, human approval) - Log Level: `ERROR` (important; track for pattern analysis) **Example Response:** ``` Agent called DatabaseAPI, connection failed. Retry 1/3 after 5 seconds. [ERROR log] [retry succeeds] Agent successfully retried DatabaseAPI. Continuing operation. [INFO log] ``` **If retries exhaust:** ``` Agent called DatabaseAPI 3 times, all failed. Max retries exhausted. Escalating to human approval (ESCALATE.md). Waiting for human decision. ``` #### Mode 3: Cascading Failure **Definition:** Multiple components are failing simultaneously, or a single failure is triggering downstream failures. Circuit breaker activates to prevent cascade from spreading. **Examples:** - Three API errors within 60 seconds (pattern indicates systemic issue, not transient glitch) - Two health checks failing at same time (database + cache both down) - Resource consumption doubling in 10 minutes (leak or runaway process) - Database replication lag exceeds threshold (primary/replica mismatch) **Detection:** - Error rate exceeds threshold (e.g., 3 failures in 60 seconds) - Multiple independent health checks failing simultaneously - Resource metrics exceeding thresholds - Cascade pattern detected (error → downstream error → downstream error) **Response:** - Action: `circuit_breaker` (stop attempting operations; fail fast) - Circuit Open Duration: `5 minutes` (wait before retrying) - Half-Open Probe Interval: `60 seconds` (periodically check if system recovered) - Escalate: `FAILSAFE.md` (revert to safe state) - Log Level: `ERROR` (critical; requires investigation) - Notify: `true` (alert operator immediately) **Example Response:** ``` Agent detected error pattern: 3 failures in 60 seconds. Activating circuit breaker. All operations to failing component(s) will fail fast. Escalating to FAILSAFE.md for safe-state recovery. [Alert operator: "CIRCUIT BREAKER ACTIVATED"] [Circuit open for 5 minutes; health check probe every 60 seconds] [60 seconds later: health check passes; circuit transitions to half-open] [Agent tests operation; if successful, circuit closes and operations resume] ``` #### Mode 4: Silent Failure **Definition:** The agent produces output without detecting an underlying error. The error is "silent" because no exception is raised and normal error handling paths don't fire. Examples: API returned cached stale data, database write partially succeeded, validation check was skipped, data became inconsistent. **Examples:** - Search API returned cached results from 2 hours ago (user believes data is current) - Database INSERT appeared to succeed but actual write failed (phantom write) - Output validation was accidentally disabled (invalid data passed through) - ML model returned degraded predictions but formatter didn't detect quality drop - Cache is returning inconsistent data (stale vs fresh) **Detection:** - Output validation checks (e.g., schema validation, range checks) - Freshness checks (e.g., timestamp is recent, not cached from 2 hours ago) - Consistency checks (e.g., cross-reference validation, double-write verification) - Coherence checks (e.g., predictions are within expected confidence ranges) **Response:** - Action: `flag_and_quarantine` (mark output as potentially unreliable; remove from serving) - Require Human Review: `true` (human must verify and approve before output is served) - Escalate: `ESCALATE.md` (route to human approval; don't proceed unless approved) - Log Level: `ERROR` (critical; most dangerous failure mode) **Example Response:** ``` Agent generated output. Output validation detected: freshness check failed (timestamp is 2 hours old; expected < 5 minutes). Flagging output as potentially unreliable. Escalating to ESCALATE.md for human review. [Output quarantined; not served to user] [Wait for human approval before proceeding] ``` ### Failure Detection FAILURE.md defines the detection mechanisms for each failure mode. #### Health Checks Regular probes of critical components to verify they are functioning: ```yaml health_checks: enabled: true interval_seconds: 30 components: api_availability: check: response_time < 5 seconds failure_mode: graceful_degradation (if non-critical) or partial_failure (if critical) database_connection: check: connection_pool > 0 failure_mode: partial_failure (should have replicas) or cascading_failure (if single point of failure) memory_usage: check: percentage < 80% failure_mode: cascading_failure (memory exhaustion is systemic) ``` #### Heartbeats Periodic "alive" signals from components. Missing heartbeat indicates component is unresponsive: ```yaml heartbeats: enabled: true interval_seconds: 60 timeout_seconds: 120 missing_heartbeat_trigger: partial_failure ``` #### Error Pattern Matching Detect patterns that indicate deeper issues: ```yaml error_patterns: three_errors_in_60_seconds: cascading_failure (single error is transient; three is pattern) rate_limit_429_twice_per_hour: partial_failure (escalate if persists) timeout_5xx_four_times_in_10_min: cascading_failure ``` #### Output Validation Check that generated output is valid, fresh, and consistent: ```yaml silent_failure_detection: schema_validation: required (output matches expected schema) freshness_checks: required (data timestamps are recent) consistency_checks: required (cross-reference validation) coherence_checks: required (predictions within expected ranges) ``` ### Integration with FAILSAFE.md and ESCALATE.md FAILURE.md defines what to do when each failure mode is detected. Often this means escalating to other specs: - **Graceful Degradation** → No escalation; continue with reduced capability - **Partial Failure** → After max retries exhausted, escalate to ESCALATE.md (seek human approval) - **Cascading Failure** → Escalate to FAILSAFE.md (revert to safe state) - **Silent Failure** → Escalate to ESCALATE.md (human review required before proceeding) --- ## Why FAILURE.md? ### The Problem It Solves AI agents fail, but the failure modes are not created equal: 1. **Unclear Failure Classification:** Is this a transient glitch or a systemic issue? Should we retry or give up? Agents don't know, so behavior is inconsistent. 2. **No Early Warning System:** Cascading failures spread undetected until the agent stops entirely. No circuit breaker to fail fast. 3. **Silent Failures Go Undetected:** Agent generates output without detecting that the underlying data is stale, partially lost, or logically inconsistent. User believes data is reliable; it's not. 4. **Ad-Hoc Recovery:** When things fail, recovery is hardcoded in agent logic, scattered across multiple files, or absent entirely. Recovery is unpredictable and unauditable. 5. **No Audit Trail:** No record of what failed, when, which mode triggered, or what action was taken. Auditors can't verify that failures are handled correctly. ### How FAILURE.md Fixes It 1. **Clear Classification** — Failure mode definitions are explicit; agent can identify which mode it's in and respond appropriately 2. **Early Warning System** — Health checks and error pattern detection identify cascading failures before they spread 3. **Silent Failure Detection** — Output validation, freshness checks, consistency checks catch silent failures before they reach users 4. **Documented Recovery** — Recovery procedures are defined in FAILURE.md; version-controlled and auditable 5. **Complete Audit Trail** — All failures logged with timestamp, mode, component, and action taken; compliance auditors can verify procedures --- ## How to Use It ### File Structure Place FAILURE.md in your project root alongside FAILSAFE.md and ESCALATE.md: ``` your-project/ ├── AGENTS.md (what agent does) ├── ESCALATE.md (approval gates & human intervention) ├── FAILSAFE.md (safe-state recovery) ├── FAILURE.md ← add this (failure modes & response) ├── KILLSWITCH.md (emergency stop) ├── README.md └── src/ ``` ### Specification Template Copy and customize this template for your project: ```yaml # FAILURE > Failure mode definitions & handling. > Spec: https://failure.md --- ## MODES graceful_degradation: description: Non-critical component unavailable; agent continues with reduced capability is_critical: false examples: - Search API down → skip search enrichment - Image generation unavailable → text-only response - Analytics service slow → skip detailed metrics action: continue_degraded log_level: WARNING notify: false escalate_after_minutes: null partial_failure: description: Component fails intermittently; agent retries and routes around is_critical: true examples: - Network timeout on API call - Database connection pool exhausted - Rate limit on external service action: isolate_and_route_around max_retries: 3 retry_backoff_seconds: [5, 15, 60] escalate_after_retries: ESCALATE.md log_level: ERROR cascading_failure: description: Multiple failures simultaneously; circuit breaker activates is_critical: true severity: critical examples: - 3+ failures within 60 seconds - 2+ health checks failing at same time - Resource consumption doubling in 10 minutes detection: error_rate_threshold: 3 failures in 60 seconds health_check_threshold: 2+ checks failing simultaneously resource_threshold: usage doubling in 10 minutes action: circuit_breaker circuit_breaker: trigger: error_rate_threshold open_duration_seconds: 300 half_open_probe_interval_seconds: 60 log_level: ERROR notify: true escalate_to: FAILSAFE.md silent_failure: description: Output generated without detecting underlying error is_critical: true severity: critical examples: - API returned stale cached data - Database write partially succeeded - Validation check was skipped detection: output_validation: required freshness_checks: required consistency_checks: required action: flag_and_quarantine require_human_review: true escalate_to: ESCALATE.md log_level: ERROR ## DETECTION health_checks: enabled: true interval_seconds: 30 components: - api_availability - database_connection - model_api - memory_usage - disk_space heartbeats: enabled: true interval_seconds: 60 timeout_seconds: 120 error_patterns: enabled: true patterns: - three_errors_in_60s: cascading_failure - rate_limit_429: partial_failure - timeout_5xx: partial_failure output_validation: enabled: true schema_validation: required freshness_checks: required consistency_checks: required coherence_checks: required ## LOGGING all_failures: enabled log_format: timestamp,mode,component,error_message,action_taken retention_days: 90 ## RECOVERY graceful_degradation_recovery: none (acceptable state; monitor but don't escalate) partial_failure_recovery: strategy: exponential_backoff_retry max_retries: 3 backoff: [5s, 15s, 60s] on_exhaustion: escalate_to_ESCALATE cascading_failure_recovery: strategy: circuit_breaker + safe_state_recovery circuit_open_duration: 5 minutes safe_state_definition: see FAILSAFE.md detection_required_before_recovery: yes silent_failure_recovery: strategy: human_review + manual_approval escalation_path: ESCALATE.md timeout: 1 hour on_timeout: block (conservative; don't proceed without explicit approval) ``` ### Implementation Steps 1. **Copy Template:** Download from https://github.com/Failure-md/spec 2. **Identify Failure Modes:** List all ways your agent can fail (API down, database error, timeout, invalid input, etc.) 3. **Classify Modes:** Determine which failure mode each falls into (graceful degradation, partial failure, cascading, silent) 4. **Configure Detection:** Set health check intervals (default 30s), error thresholds (default 3 errors in 60s), timeout values 5. **Define Responses:** For each mode, specify action, log level, notifications, escalation target 6. **Implement Enforcement:** Agent reads FAILURE.md on startup; implements detection and response procedures 7. **Set Up Monitoring:** Track health check results, error patterns, escalations 8. **Test Each Mode:** Simulate API failures, database errors, cascading failures, silent failures 9. **Log Everything:** Every failure event logged with timestamp, mode, component, action taken 10. **Review Annually:** Analyze failure logs; identify patterns; update detection thresholds and response procedures --- ## The Twelve-File AI Safety Stack FAILURE.md is layer 11 of 12: ### Operational Control (1-5) 1. **THROTTLE.md** — Slow down (rate limits, cost ceilings, concurrency caps) 2. **ESCALATE.md** — Raise the alarm (approval gates, notifications, timeouts) 3. **FAILSAFE.md** — Fall back safely (safe-state definitions, snapshots, rollback) 4. **KILLSWITCH.md** — Emergency stop (triggers, forbidden actions, escalation paths) 5. **TERMINATE.md** — Permanent shutdown (no restart, evidence preservation, credential revocation) ### Data Security (6-7) 6. **ENCRYPT.md** — Secure everything (data classification, protection policy, secrets handling) 7. **ENCRYPTION.md** — Implement the standards (algorithms, key rotation, TLS, compliance) ### Output Quality (8-10) 8. **SYCOPHANCY.md** — Prevent bias (honest outputs, citations, disagreement protocol) 9. **COMPRESSION.md** — Compress context (summarization rules, coherence checks) 10. **COLLAPSE.md** — Prevent collapse (model drift detection, recovery checkpoints) ### Accountability (11-12) 11. **FAILURE.md** — Define failure modes (graceful degradation, cascading failures, silent failures) ← You are here 12. **LEADERBOARD.md** — Benchmark agents (completion, accuracy, cost efficiency, safety scores) --- ## Use Cases ### API-Heavy Agents Monitor external API health; detect rate limits (429), timeouts (504), errors (5xx); implement retry logic with exponential backoff; graceful degrade if API is non-critical. ### Database Operations Monitor database connection pool health; detect slow queries, deadlocks, connection exhaustion; route around failing replicas; circuit-break if primary is down. ### Multi-Component Systems Monitor each component independently; detect cascading failures when 2+ components fail simultaneously; circuit-break to prevent cascade spreading. ### High-Reliability Systems Implement comprehensive output validation; detect stale data, partial writes, consistency violations; require human approval before serving potentially unreliable output. ### Cost-Conscious Deployments Monitor resource consumption (memory, disk, API calls); detect resource exhaustion early; trigger graceful degradation or cost-capping before costs spiral. ### Real-Time Systems Use short health check intervals (5-10 seconds); detect failures immediately; failover to replica or backup; prioritize speed of detection over accuracy. --- ## Regulatory & Compliance Context ### EU AI Act (Effective 2 August 2026) **Requirement:** "AI systems shall be designed to detect and respond to abnormal performance in a timely manner." **Interpretation:** Failure detection and response must be explicit and documented. FAILURE.md satisfies this requirement. ### SOC2 Common Criteria **CC7.2 (Systems Monitoring):** "The entity monitors system components and the operation of those components for anomalies that are indicative of potential cyber security events to enable incident response personnel to respond to those anomalies in a timely manner." **Evidence:** FAILURE.md defines health checks, error pattern detection, and monitoring procedures; logs of all failures demonstrate anomaly detection is working. ### ISO 27001 A.16.1 (Incident Management) **Requirement:** "The organization shall establish and implement policies and procedures for the identification, analysis, assessment, response, and recovery from incidents." **Evidence:** FAILURE.md IS the documented incident response procedure; failure logs are the incident records. --- ## Framework Compatibility FAILURE.md is framework-agnostic. Works with: - **LangChain:** Implement health checks in tool initialization; wrap tool calls in try-catch with retry logic - **AutoGen:** Monitor agent health between turns; implement circuit breaker in conversation manager - **CrewAI:** Monitor task execution; detect silent failures in task outputs - **Claude Code:** Implement health checks in agent startup; log all failures to audit trail - **Cursor Agent Mode:** Fetch health check status from component APIs; implement retry logic - **Custom agents:** Parse FAILURE.md; implement detection and response procedures --- ## FAQ ### What is FAILURE.md? A plain-text Markdown file defining the four failure modes an AI agent can encounter — graceful degradation, partial failure, cascading failure, and silent failure — along with detection signals and per-mode response procedures. Every failure event is logged with timestamp, mode, component, and action taken. ### What is the difference between graceful degradation and partial failure? **Graceful degradation:** the agent continues operating with reduced capability — a non-critical tool is unavailable, so it skips that feature and logs it. **Partial failure:** one component fails and the agent must actively route around it — retrying with backoff, routing to a replica, or queuing for later. Partial failure triggers retries and potentially escalates; graceful degradation does not. ### What is a silent failure and why is it dangerous? A silent failure is when the agent produces output without detecting an underlying error — the API returned stale data, the write partially succeeded, or the validation check was skipped. Because no error is raised, normal escalation paths don't fire. FAILURE.md defines output validation, data freshness checks, and cross-reference consistency checks specifically to catch silent failures. ### What triggers the circuit breaker? Cascading failure detection: three failures within 60 seconds, two or more health check components failing simultaneously, or resource consumption doubling within 10 minutes. The circuit breaker stops all dependent operations immediately and escalates to FAILSAFE.md to prevent the cascade spreading further. ### How does FAILURE.md relate to FAILSAFE.md and ESCALATE.md? **FAILURE.md** defines what failure modes exist and how to detect and respond to each. **Cascading failures** escalate to **FAILSAFE.md** (which defines the safe recovery state). **Partial failures** escalate to **ESCALATE.md** after exhausting retries (which routes to a human). FAILURE.md is the taxonomy; FAILSAFE.md and ESCALATE.md are the recovery paths. ### Does FAILURE.md work with all AI frameworks? Yes — it is framework-agnostic. It defines failure mode policy; your agent implementation enforces it. Works with LangChain, AutoGen, CrewAI, Claude Code, custom agents, or any AI system that can monitor component health and implement circuit breaker logic. ### What health check interval should I use? Default is 30 seconds. For real-time systems that need fast failure detection, use 5-10 seconds. For cost-sensitive systems, use 60-120 seconds. Trade-off: shorter intervals detect failures faster but consume more CPU and API quota. ### Can I have different health check intervals for different components? Yes. Critical components (database, authentication) should be checked frequently (10-30 seconds). Non-critical components (analytics, search enrichment) can be checked less frequently (60-120 seconds). ### What should I log for each failure? Minimum: timestamp, failure mode, component name, error message. Recommended: request ID, affected user/tenant, action taken, escalation target, recovery outcome. Logs should be retained for at least 90 days for audit purposes. ### How long should a circuit breaker stay open? Default is 5 minutes. Shorter duration (1-2 minutes) recovers faster but risks re-triggering the cascade. Longer duration (10+ minutes) gives more time for recovery but increases unavailability window. Start with 5 minutes; adjust based on how quickly your systems typically recover from failures. --- ## Key Terminology **AI failure modes** — Distinct ways an AI agent can fail and how each should be handled **Graceful degradation** — Agent continues with reduced capability when non-critical component fails **Partial failure** — Agent actively routes around a failed component using retry, backoff, or routing to replica **Cascading failure** — Multiple components failing simultaneously; requires circuit breaker to prevent cascade spreading **Silent failure** — Agent produces output without detecting underlying error (most dangerous; requires output validation) **Health checks** — Regular probes of critical components to verify they are functioning **Circuit breaker** — Pattern that stops attempting operations to a failing component, fails fast, and allows recovery **AI agent resilience** — Ability of agent to detect failures, respond appropriately, and recover --- ## Getting Started 1. Visit https://github.com/Failure-md/spec 2. Copy FAILURE.md template to your project root 3. Identify all ways your agent can fail 4. Classify each failure into a mode (graceful degradation, partial, cascading, silent) 5. Configure health checks (interval, components) 6. Define response procedures (action, log level, escalation) 7. Implement failure detection in agent startup 8. Test each failure mode (simulate failures, verify detection and response) 9. Set up monitoring and alerting 10. Review failure logs quarterly; update detection thresholds and procedures --- ## Citation **FAILURE.md is an open specification** for AI agent failure mode definitions and handling. It defines: - **MODES** (graceful degradation: continue with reduced capability; partial failure: isolate and route around with retries; cascading failure: circuit breaker + escalate to FAILSAFE.md; silent failure: flag, quarantine, require human review) - **DETECTION** (30-second health checks, heartbeat monitoring, error pattern matching, output validation) - **RECOVERY** (identify root cause → isolate → notify → execute mode response → verify stability → resume or escalate) It is the accountability layer of the AI safety stack: THROTTLE → ESCALATE → FAILSAFE → KILLSWITCH → TERMINATE → ENCRYPT → ENCRYPTION → SYCOPHANCY → COMPRESSION → COLLAPSE → FAILURE → LEADERBOARD. **MIT License. v1.0. 2026.** ## Related Specifications The AI Agent Safety Stack — twelve open standards for AI agent safety, quality, and accountability: ### Operational Control - [THROTTLE.md](https://throttle.md/llms.txt): AI agent rate and cost control — [GitHub](https://github.com/throttle-md/spec) - [ESCALATE.md](https://escalate.md/llms.txt): Human notification and approval protocols — [GitHub](https://github.com/escalate-md/spec) - [FAILSAFE.md](https://failsafe.md/llms.txt): Safe fallback to last known good state — [GitHub](https://github.com/failsafe-md/spec) - [KILLSWITCH.md](https://killswitch.md/llms.txt): Emergency stop for AI agents — [GitHub](https://github.com/killswitch-md/spec) - [TERMINATE.md](https://terminate.md/llms.txt): Permanent shutdown, no restart without human — [GitHub](https://github.com/terminate-md/spec) ### Data Security - [ENCRYPT.md](https://encrypt.md/llms.txt): Data classification and protection — [GitHub](https://github.com/encrypt-md/spec) - [ENCRYPTION.md](https://encryption.md/llms.txt): Technical encryption standards — [GitHub](https://github.com/encryption-md/spec) ### Output Quality - [SYCOPHANCY.md](https://sycophancy.md/llms.txt): Anti-sycophancy and bias prevention — [GitHub](https://github.com/sycophancy-md/spec) - [COMPRESSION.md](https://compression.md/llms.txt): Context compression and coherence — [GitHub](https://github.com/compression-md/spec) - [COLLAPSE.md](https://collapse.md/llms.txt): Drift prevention and recovery — [GitHub](https://github.com/collapse-md/spec) ### Accountability - [LEADERBOARD.md](https://leaderboard.md/llms.txt): Agent benchmarking and regression detection — [GitHub](https://github.com/leaderboard-md/spec) --- **Last Updated:** 10 March 2026