Operating agentic systems: the on-call surface no one warned you about

A new category of incident

Running a microservices system, you learn to expect a specific set of incident types: service down, latency spike, database exhaustion, queue backup. The runbooks are well-worn. The dashboards are tuned. On-call engineers know the playbook.

Agentic systems introduce a new category of incident that none of those runbooks cover. Not a service being down — a service running too enthusiastically. Not a timeout — an infinite retry loop that's technically succeeding, expensively. Not a crash — an agent confidently completing the wrong task and leaving the system in a state that looks healthy on every existing dashboard.

We've spent a year evolving runbooks for agentic systems across a dozen production deployments. Here is the operational surface no one warned you about.

The five failure modes

Drift. An agent's behaviour changes without any code change. The upstream model received an update, a dependency shifted, or the input distribution changed enough to cross a decision boundary. Drift is invisible to traditional monitoring because the system is still running, still completing tasks, still returning 200s. You discover it when a user reports wrong behaviour or when you run your eval suite.

Runaway loops. An agent gets stuck in a retry loop or a self-reinforcing reasoning pattern. In a well-instrumented system this shows up as cost spike and latency blowout. In a poorly instrumented system it shows up as a large bill at the end of the month.

Tool-use failures. An agent calls a tool with malformed arguments, interprets a tool error as a signal to retry with variations, and works through every permutation before finally giving up or — worse — picking one at random and proceeding. The external tool may have side effects on each call.

Context poisoning. A bad value gets written into the agent's persistent context — a wrong assumption, a hallucinated fact, a misread tool result — and influences all subsequent decisions in the session. The agent isn't malfunctioning. It's reasoning correctly from corrupted premises.

Partial completion. An agent completes 80% of a multi-step task, fails on step 8 of 10, and leaves the system in a partially mutated state. The database has some of the intended writes. The downstream system received some of the intended events. The task is neither done nor cleanly failed.

Instrumenting for operability

The first line of defence against all five failure modes is instrumentation designed specifically for agentic behaviour, not repurposed from service monitoring.

Every agent invocation should emit a structured trace event with: agent name, task ID, turn number within the session, input (sanitised of PII), output, tool calls made (name, arguments, response), tokens consumed, latency, and a session correlation ID that links all events in a single task execution.

Beyond per-turn events, emit session-level rollups: total turns, total tokens, total cost, success or failure state, and — critically — a classification of the failure type if the session ended in failure.

These events feed two audiences. Real-time: your on-call dashboard, watching for cost anomalies, latency outliers, and error rate spikes. Asynchronous: your eval pipeline, which replays failed sessions to identify systematic patterns.

The on-call runbook

The on-call runbook for an agentic system needs sections that don't exist in a services runbook.

How to terminate a running session. Not kill a process — gracefully terminate an in-progress agent session, checkpoint its state, and leave the system in a clean known state. This needs to be a one-command operation available to any on-call engineer.

How to replay a session from a checkpoint. When a session fails at step 8 of 10, you need to be able to resume from the last good checkpoint rather than re-running from the start. This requires that your state store supports point-in-time checkpoints and that your orchestrator can resume from an arbitrary checkpoint ID.

How to inspect active sessions. A dashboard showing every currently running session, its current step, its cost so far, its elapsed time, and any error signals. This is the agentic equivalent of a services health dashboard and is the most common thing on-call engineers reach for.

How to roll back an agent's side effects. For agents that write to databases or trigger external actions, you need a compensation strategy: either a rollback operation or a compensating transaction that undoes the side effects of a failed session.

Cost as a canary

Cost per session is one of the most reliable early warning signals for agentic system anomalies. A session that normally costs $0.04 and suddenly costs $0.40 has almost certainly hit a runaway pattern, a retrieval failure that's generating massively expanded context, or an unusual input that's driving an unexpectedly long reasoning chain.

Set cost alerts at 3× and 10× the p95 session cost for each agent type. The 3× alert is informational — investigate before the next deploy. The 10× alert is a pager — investigate now and consider automatic session termination.

This canary works because LLM costs are a direct function of computation, and anomalous computation almost always means something unexpected is happening.

What good operations looks like

The teams that operate agentic systems reliably share a few practices: they built their observability stack before they went to production, not after. They have session termination and checkpoint-resume as first-class engineering features, not afterthoughts. They run their eval suite on a schedule, not just before deploys. And they treat every production failure as a new eval case.

The hardest adjustment for teams coming from services operations is accepting that agentic failures are often silent and gradual rather than loud and immediate. The posture shifts from "alert when something breaks" to "continuously verify that the system is still doing what you think it's doing." That's a different kind of on-call, but it's manageable once you've built for it.