The Reliability Multiplication Law

The verdict: agent reliability compounds. A task made of n dependent steps, each succeeding with probability p, succeeds end-to-end with probability p^n. That is not a model of agents. That is arithmetic, and it governs every agent ever deployed. It explains ≈ why the demo says ready and the deployment says no. It explains why 0.99^30 74%, why 0.98^50 ≈ 36%, and why a recovery layer that halves the failure rate still loses to a long enough chain. Almost nobody prices agents this way. The headline metrics measure single attempts at short tasks. Production requires long chains at 95 to 99 percent. The distance between those two numbers is the story of the agent economy, and it is measured in exponents.

Method: state the law, then attack its assumptions honestly, because the assumptions cut in both directions. Then price every escape, retries, recovery, verification, human oversight, and show that each escape changes the base of the exponent, never its existence. Then the strongest case against the law, answered with the current record. Then the observation that would kill it. Every number carries its denominator. Where a number is an assumption rather than a measurement, it is labeled.

Scope and date: the record runs to late August 2026. METR, τ-bench, and Google and NVIDIA robotics figures are cited by name, with what each does not prove.

The law, stated plainly

A step is one action whose failure spoils the whole task. Steps are dependent when later steps inherit earlier outputs. For n dependent steps at per-step success probability p, end-to-end success is p^n. ESTABLISHED: this is the multiplication rule of probability, not a conjecture. The empirical questions are only what p is, what n is, and how far reality departs from independence. Those questions get full treatment below. The arithmetic first.

The law is not a capability ceiling. It does not say agents cannot do hard things. It says the probability of doing a long thing is the product of the probabilities of doing its parts, and products shrink. Occasional success supports exploration, and exploration is what demos measure. Dependable completion supports delegation, and delegation is what payrolls require. The economy runs on the second. End-to-end success out of 100 identical runs:

Read the 0.98 row aloud. A step that succeeds 98 times in 100 is excellent by any benchmark standard. Five such steps: 90 of 100 tasks complete. Fifty such steps: 36 of 100. Two hundred: fewer than 2 of 100. Nothing about the step changed. Only the chain lengthened. The decay is not a property of the model. It is a property of the exponent.

The reverse direction is the one procurement never runs. Fix the end-to-end target and solve for the required per-step reliability. For 95 percent success over a 50-step chain, each step must succeed 99.90 percent of the time: 0.999^50 ≈ 0.951. For 95 percent over 200 steps, each step needs 99.974 percent. For 99 percent over a month of autonomous work, roughly a thousand dependent steps, each step needs 99.999 percent, five nines: ≈ 0.99999^1000 0.990. Five nines is the reliability of aviation-grade components. It is being demanded here of a language model's next action.

The last box is why the law stays invisible. A chain that dies at step 47 of 50 does not look like multiplication. It looks like one unlucky event. Postmortems search for the broken step. The broken step is innocent. The exponent killed the run, and the exponent was set on the day the task was defined.

The steps are not independent, and that cuts both ways

The naive product assumes independence. Real agents violate the assumption in both directions, and precision matters more than drama here. The exact statement of the law uses conditional probabilities: end-to-end success equals the product over k of the probability that step k succeeds given that every previous step succeeded. Independence is the special case where each conditional equals the marginal p. Two regimes depart from that case, in opposite directions.

Regime one: persistent environments make the product pessimistic. Suppose tasks come in two fixed kinds. Ninety of 100 tasks are easy for this agent, with per-step success 0.999. Ten of 100 are hard, with per-step success 0.90. The marginal per-step success averages to ≈ 0.9891, and the naive product gives 0.9891^50 57.8 percent. The true end-to-end success ≈ is a weighted sum, not a power of the average: 0.9 × 0.999^50 + 0.1 × 0.90^50 0.9 × 0.951 + ≈ 0.1 × 0.005 85.7 percent. The mixture beats the naive product by 28 points out of 100. This is a general result: p^n is convex in p, so averaging over a heterogeneous task mix can only raise end-to-end success above the product of the average. Agents deployed on work they are good at outperform their blended benchmark. The law does not forbid cherry-picking the deployment. It prices everything else.

Regime two: contamination makes the product optimistic. Agents carry shared state: a context window, a memory store, a half-edited file, a partially executed plan. A failed step does not vanish. It writes its residue into everything downstream.

The conditional framing exposes the trick. The product of marginals averages over histories the deployment never survives. Conditioned on a clean history, a step may succeed at 0.99. Conditioned on a contaminated history, the same step succeeds at far less, and contamination, once present, persists. An agent that does not halt on error walks its residue to the end of the chain.

The contamination regime is not hypothetical. Trace-level rubric analysis of τ-bench failures (REPORTED, 2026) found that airline-domain runs die by gradual trajectory degradation. The sharpest signal is state inconsistency: the agent's running picture of the world drifts from the database it is editing. Flagged runs succeeded 37.5 percent of the time, against 73.5 percent of clean runs, the strongest failure signal in the rubric set. One wrong belief, entered early, taxes every later step. That is the mechanism of regime two, measured in a production-like harness rather than asserted in a think piece.

The author's own agent-reliability model, built for exactly this estimate, carries the warning in its assumptions: steps are modeled as independent before recovery, and real failures may be correlated. Its confidence label is MEDIUM. That is the honest state of the science. Independence is a modeling convenience. Dependence is the deployment.

Dependence also poisons the standard fix, which is the next section's subject. Retry logic assumes each new attempt is a fresh draw. If the failure's cause lives in shared state, a misread goal, a wrong fact in memory, the retry is not a fresh draw. It is the same loaded die, rolled again with confidence.

Recovery changes the base, never the exponent

The standard engineering answer is recovery: detect the failure, retry the step, escalate if needed. Recovery works. It also loses, and the arithmetic of its losing is the most instructive calculation in this article.

Take the 50-step chain at 0.98 per step. Grant one retry per failed step, and grant that the retry recovers 60 percent of failures. The recovered per-step success is 0.98 + 0.02 × 0.6 = 0.992. The failure rate per step fell from 2 in 100 to 0.8 in 100, a 2.5-fold improvement. ≈ Now compound: 0.992^50 67 percent. The recovery layer converted 36 percent into 67 percent. It did not convert 67 percent into deployment. A system that completes two of three assigned tasks is not a worker. It is supervised gambling with better odds.

Push the recovery harder and the pattern holds. Three retries per step at the same 60 − percent recovery each: per-step success becomes 1 0.02 × 0.4^3 = 0.99872, and 0.99872^50 ≈ 93.8 percent. Better. Still short of the 95-to-99 band that delegation requires, and each retry costs latency, tokens, and money. The author's reliability model lands in the same place from calibrated inputs: raw success 0.3642 for the base chain, effective success 0.7457 with recovery modeled in. The model's own low estimate is the chilling number: an effective success of 1.3 × 10^-23, which is what 0.9^500 produces. At 90 percent per step over a 500-step workflow, expect one success per 10^23 runs, one in a hundred billion trillion. No retry budget rescues a weak base at a long exponent. That is the law's sharpest edge: recovery multiplies the base toward one, and the exponent collects whatever distance remains.

Recovery has a hidden prerequisite that rarely gets priced: detection. Every retry begins with knowing the step failed. A detector with false negatives passes failures through to contaminate the chain. A detector with false positives taxes every clean step with review cost and delay. The detector's own error rate enters the product as one more per-step term. In domains where failure announces itself, code that does not compile, a grasp that drops the object, detection is cheap and recovery is real. In domains where failure is quiet, a plausible wrong figure, a polite wrong email, detection is the bottleneck, and the recovery term is worth less than its sticker value.

The human-in-the-loop is one more recovery term, with its own price. Grant that a human reviewer catches 90 percent of the failures the retries missed: per-step success becomes 1 − 0.02 × 0.4 × 0.1 = 0.9992, and 0.9992^50 ≈ 96.1 percent. That number finally enters the delegation band. Now count the human. Failed steps reaching the reviewer: 50 × 0.02 × 0.4 = 0.4 interventions per task, before false alarms. The human reviews, decides, and signs. The human's own accuracy is another per-step probability entering the same product. And the human's review speed now caps the agent's throughput, which was the entire economic point of the agent. Human oversight does not break the law. It renames the bottleneck and sends the invoice to a salary.

The demo-deployment inversion

Benchmarks measure single attempts at bounded tasks under generous conditions. Production requires long chains completed at 95 to 99 percent, every day, with no resets. The gap between those two measurement regimes is not a nuance. It is an inversion: the number that moves markets is the wrong number, and the right number is shorter by a factor of five or more.

METR's time-horizon study is the cleanest instrument here. Its 50-percent-success horizon, the task length at which a model completes half its attempts, doubled roughly every seven months from 2019 through 2025 (arXiv:2503.14499, ESTABLISHED as an evaluation trend). The same paper reports the 80 percent horizon, and it runs roughly five times shorter than the 50 percent figure. The 99 percent horizons are shorter still. The January 2026 update, Time Horizon 1.1, expanded the suite to 228 tasks and kept the separate reliability levels, because a horizon without its reliability level is incomplete. By May 2026 the frontier 50-percent measurement exceeded 16 hours, which is the ceiling of the current suite; METR itself warns that estimates beyond it are unreliable. Every part of that paragraph is good news for capability. None of it is a deployment number. The headline is the 50 percent horizon. Deployment lives at 99.

The consistency record is worse than the capability record, and consistency is the deployment property. On τ-bench, a tool-use benchmark of 115 retail and 50 airline tasks, GPT-4o succeeded on under half of tasks in a single attempt (arXiv:2406.12045, ESTABLISHED). Its pass^8, the probability of succeeding on all 8 of 8 repeated attempts of the same task, fell below 25 percent in the retail domain. An agent that performs at 60 percent once performs near 25 percent eight times running. Unattended operation is the second number, always. A deployment is pass^k with large k. The leaderboard reports pass@1.

The inversion completes itself with time budget. In METR's RE-Bench evaluations, agents beat human experts at a two-hour budget. At an eight-hour budget, human returns to additional time overtook the best agent. Speed is not depth. The short task flatters the machine in exactly the regime where the exponent has no room to work.

Field data carries the same inversion. A May 2026 matched event study of more than 100,000 GitHub developers estimated cumulative commit increases of 40, 140, and 180 percent for autocomplete, interactive agents, and autonomous agents respectively. Read past the headline: the autonomous-agent estimate attenuated to 50 percent for completed projects and 30 percent for releases, and four app marketplaces showed no increase in total usage. Acceleration of steps is not completion of chains. Commits are p. Releases are p^n.

benchmark: one attempt, one task, a headline number procurement signs best conditions production: fifty the exponent steps daily, no arrives resets

Where the law binds hardest, and where it barely applies

The law binds in proportion to three properties: chain length, irreversibility, and the absence of cheap verification. Robotics holds all three. A humanoid body has roughly 30 powered joints, and the author's hardware arithmetic applies the law to the machine itself: if each of 30 joints is 99 percent reliable over a working period, whole-body reliability is ≈ 0.99^30 74 percent. For a 99 percent body, each joint needs roughly 99.966 percent reliability, before counting cameras, hands, computers, or software. The task level sits on top of the hardware level, and the two exponents stack.

The measured per-task numbers in robotics sit far below the delegation band. Google reports Gemini Robotics 2 task success ranging from 32 to 92 percent across five multi-finger manipulation tasks, and 74.2 to 89.6 percent across three gripper task categories, jagged, even within one model release (REPORTED, medium confidence). NVIDIA's GR00T N1.5 reached 83 percent on a real-robot fruit-selection task and 84.2 percent after training on novel-object placement. Take the kind reading, 0.83 per step, across a modest 20-step ≈ manipulation chain: 0.83^20 2.4 percent end-to-end. That is why a five-minute video means almost nothing, and why the valuable metric is productive hours between human interventions.

Irreversibility is the multiplier nobody prices. A step is irreversible when no retry exists: the cut is made, the pour is spilled, the email is sent, the payment settles, the record is deleted. Irreversible steps deny recovery its only mechanism. Every irreversible step in a chain holds the base at its raw value while the exponent collects in full.

Multi-day software work sits in the same vise. A month of autonomous economic work is roughly a thousand dependent steps, and the thousand-step chain at five-nines per step is the delegation threshold computed above. Current measured reality, from METR's March 2025 suite: near-perfect success on tasks under four human-minutes, and success under 10 percent beyond about four human-hours. The exponent is the distance between those two facts.

The deployment record shows the one escape that works, and its price. Figure 02 worked about 1,250 hours at BMW, handled over 90,000 components, and assisted production of more than 30,000 X3 vehicles. That is serious deployment. It is also one repeated sheet-metal task, in an engineered cell, with additional partitions and factory infrastructure. The escape is environment engineering: shorten the chain, fix the objects, remove the surprises, station humans at the exceptions. The law permits this escape. It charges for it in scope. A robot that is dependable inside one engineered cell has proven p for that cell. It has proven nothing about the house next door.

The law binds least where engineering can attack all three properties. Parallel, verifiable, retryable work escapes by construction. Batch-processing ten thousand documents with a cheap checker converts each item into a chain of length one. A failure is a reroll, not a chain break. With retries at 60 percent recovery, per-item success reaches 0.99872 after three attempts, and throughput absorbs the cost. Compilation, unit tests, type checkers, and solvers are verification that collapses serial risk into parallel cost. The design lesson falls out of the math: decompose every long chain into short, checked, reversible segments, and put human judgment only at the irreversible joints. Teams that price the law build exactly this. Teams that price the headline build demos.

The strongest case against the law

The steelman deserves its full force, because parts of it are true. First argument: p improves faster than n grows. METR's doubling is measured and real, and recent windows run faster than the seven-month headline, the author's own paper records roughly 4.3 months post-2023 and roughly 3 months post-2024, with UK AISI's May 2026 cyber tracking finding a comparable 4.7-month doubling. Per-step failure is halving on a clock of months. The base approaches one faster than economically valuable chains lengthen. A companion claim travels with this one: n need not grow at all, because tasks can be decomposed into fixed-size pieces that humans reassemble. Second argument: verification collapses chains. Compilers and test suites convert long sequences into short checked units, shrinking the effective n to the few unverifiable steps. Third argument: models self-correct. Reflection loops catch errors without any human, raising the recovery term for free.

Now the record. Against the first argument: the doubling is measured at 50 percent success. The 80 percent horizon runs five times shorter, and no published series tracks a 99 percent horizon at all. The claim that p outruns n is an extrapolation past the instrument, and METR flags its own estimates above 16 hours as unreliable with the current suite. The trend is anchored exactly where the instrument works. Beyond the instrument, the trend is hope wearing a graph. The decomposition companion claim is real but bounded: reassembly is itself a chain, and the human doing the reassembly is the human-in-the-loop term again, with its cost and its own per-step error rate. Against the second: verifiers exist only where truth is cheap to check. Code compiles. Proofs verify. No checker exists for whether the email was appropriate, the weld sound, the diagnosis wise, or the screw right for this material. The 60-point spread in Gemini Robotics 2's task rates within a single release is what a domain without verifiers looks like. Against the third: self-correction without external ground truth is a correlated re-draw. The model that misread the goal grades its own retry. τ-bench's pass^8 below 25 percent against a pass@1 near 60 percent is the measured shape of that correlation: the same agent, the same task, the same blind spots, eight times.

CUT: the claim that a smarter model escapes the law. Capability raises p. The law is indifferent to how p was earned. Whatever the capability trajectory, deployment requires the product over the chain, and the product is unforgiving by construction. The case against the law is really a bet that p reaches five nines before n reaches a thousand. That bet may win. It is currently unsupported by any measurement at the reliability levels where deployment lives.

What would change the verdict

The law itself is arithmetic and cannot be falsified. What can die is the claim that the law binds real deployments, and the kill condition is specific, because consistency at long horizon is measurable in principle and unmeasured in practice.

Kill condition, three parts. First, a published 99-percent-success horizon on a versioned task suite, rising on the same clock as the 50 percent horizon. Second, audited production telemetry from deployments nobody selling the systems controls: at least 95 percent end-to-end success across chains of one hundred or more dependent steps, over at least ten thousand runs, with under one human intervention per thousand tasks. Third, pass^k converging to pass@1 at long horizons, consistency closing on capability rather than lagging it by factors. Any one of the three would weaken the law's grip. All three would break it.

None exists as of August 2026. The published record holds the opposite shape. Horizons are quoted at 50 percent. Consistency collapses by half or more within eight repetitions. Robotics task rates spread across 60 points within a single release. And the one deployment metric that matters, productive hours between human interventions, remains absent from every launch announcement. Until the telemetry arrives, price agents the way the arithmetic dictates. The demo shows p. Deployment pays p^n.