Circuit breaker pattern: how it works and when it should trip

A circuit breaker is a wrapper around calls to a dependency that stops sending requests once that dependency keeps failing, and returns an error immediately instead. After a cooldown it lets a single test request through; if that succeeds, normal traffic resumes. The circuit breaker pattern protects your service from hanging on a dead dependency, and protects the dependency from being flooded while it tries to recover.

The name comes from electrical wiring. When too much current flows, the breaker trips and cuts the circuit before something burns. In software, "too much current" is a run of timeouts or errors, and the thing that burns is your thread pool, your connection pool, or the patience of your users.

What problem does a circuit breaker solve?

When a downstream service slows to a crawl, the calls to it do not fail quickly. They wait for a timeout. A timeout is the maximum time a caller will wait for a response before giving up. If that limit is 5 seconds and the dependency is effectively down, every request that touches it now takes 5 seconds and then fails anyway.

That waiting is expensive. Each pending call holds something: a worker, a socket, a database connection, memory for the request. Under steady traffic, those held resources pile up until the calling service runs out of them. At that point it starts failing requests that have nothing to do with the broken dependency. This is a cascading failure: one unhealthy service drags down the services that call it, and then the services that call those.

Retries make it worse. If every caller retries a failing dependency, the load on it multiplies right when it has the least capacity. Exponential backoff with jitter spreads those retries out, which handles short blips well. It does not help much when the outage lasts minutes, because every request still pays for at least one full timeout.

A circuit breaker handles the sustained case. Once it has seen enough failures, it stops trying for a while. Calls fail in microseconds instead of seconds, resources are freed, and the dependency gets a quiet period to recover.

How does a circuit breaker work?

A breaker is a small state machine with three states.

Closed is normal operation. The name is confusing at first, but it follows the electrical meaning: a closed circuit lets current flow.

Open means the breaker has tripped. Every call fails fast with an error the caller can handle, such as returning cached data or a default.

Half-open is the test phase. After the cooldown, the breaker does not know whether the dependency has recovered, so it lets a limited number of requests through. If they succeed, the breaker closes. If they fail, it opens again and restarts the cooldown.

Here is a minimal version in JavaScript. It counts consecutive failures, allows one probe in half-open, and takes a now function so you can test it without real clocks.

class CircuitBreaker {
  constructor(fn, { failureThreshold = 5, cooldownMs = 10_000, now = Date.now } = {}) {
    this.fn = fn;
    this.failureThreshold = failureThreshold;
    this.cooldownMs = cooldownMs;
    this.now = now;
    this.state = "closed";
    this.failures = 0;
    this.openedAt = 0;
  }

  async call(...args) {
    if (this.state === "open") {
      if (this.now() - this.openedAt < this.cooldownMs) {
        throw new Error("circuit open");      // fail fast, no network call
      }
      this.state = "half-open";                 // cooldown over: allow one probe
    } else if (this.state === "half-open") {
      throw new Error("circuit open");          // a probe is already in flight
    }

    try {
      const result = await this.fn(...args);
      this.state = "closed";
      this.failures = 0;
      return result;
    } catch (err) {
      this.failures++;
      if (this.state === "half-open" || this.failures >= this.failureThreshold) {
        this.state = "open";
        this.openedAt = this.now();
      }
      throw err;
    }
  }
}

With a threshold of 3 and a 1 second cooldown, five calls against a dead dependency produce timeout, timeout, timeout, circuit open, circuit open. Only three requests ever reach the dependency. A probe 1.5 seconds later still fails, so the breaker reopens. Once the dependency is healthy again, the next probe after the cooldown succeeds, the breaker closes, and calls flow normally.

When should a circuit breaker trip?

Counting consecutive failures is the simplest rule, and it works for low-traffic calls. At higher volume, most implementations look at a rolling window instead: the failure rate over the last N calls or the last N seconds. A common shape is "open if at least 50% of calls failed in the last 10 seconds, and there were at least 20 calls". That minimum volume matters. Without it, two failures out of two calls at 3 a.m. would trip the breaker.

Decide carefully what counts as a failure:

• Count: timeouts, connection errors, 5xx responses, and 429 Too Many Requests.
• Do not count: 400, 404, validation errors, and other client mistakes. The dependency is working; the request was wrong.
• Consider counting slow calls. A response that takes 4.9 seconds against a 5 second timeout is technically a success, but it is a sign of trouble. Some libraries let you treat calls above a latency threshold as failures.

Keep the cooldown short enough that recovery is noticed quickly, often 5 to 30 seconds. Add a little jitter to it if many instances share the same settings, so they do not all probe at the same instant.

Where should the breaker live?

Put one breaker per dependency, and sometimes per endpoint of that dependency. If a payment provider's refund endpoint is broken but charges work, a single shared breaker would block both.

Each instance of your service usually keeps its own breaker state in memory. That is fine. Instances see roughly the same failures, so they trip at roughly the same time, and there is no shared store to become a new point of failure. Shared state across instances is possible but rarely worth the extra moving part.

You can implement breakers in three places:

• In application code, with a library such as resilience4j for Java, Polly for .NET, or opossum for Node.js.
• In a service mesh or proxy, such as Envoy or Istio, where outlier detection removes failing hosts from the load balancing pool.
• In an API gateway, for calls that leave your network.

Application-level breakers know the most about what a failure means. Proxy-level breakers need no code changes but see only HTTP status codes and timing.

What should happen when the circuit is open?

A breaker that just throws an error has done half the job. The other half is the fallback: what you return instead. Good options depend on the call:

• Serve a cached or slightly stale value.
• Return a sensible default, such as an empty recommendations list.
• Hide an optional part of the page rather than failing the whole page.
• Queue the work for later, if it does not need to happen now.

Also make the state visible. Emit a metric or log line every time a breaker opens or closes. An open breaker is an outage signal, and it is often the clearest one you have.

Key takeaways

• A circuit breaker stops calling a failing dependency, fails fast, and probes for recovery after a cooldown.
• It prevents cascading failures by freeing resources that would otherwise wait on timeouts.
• Retries with backoff handle short blips; circuit breakers handle sustained outages. Use both.
• Trip on a failure rate over a rolling window with a minimum call count, and only count real dependency failures.
• Pair every breaker with a fallback and a metric for state changes.

FAQ

What is the difference between a circuit breaker and a retry?

A retry tries the same call again, hoping the next attempt succeeds. A circuit breaker decides not to make the call at all while the dependency looks unhealthy. Retries sit inside the breaker, so a tripped breaker also stops the retries.

Does a circuit breaker replace timeouts?

No. A breaker relies on timeouts to detect slow failures in the first place. Without a timeout, a hung call never counts as a failure and the breaker never trips.

Should every service call have a circuit breaker?

Not necessarily. Calls to remote dependencies that can fail independently, such as other services, third-party APIs and databases, benefit most. Calls inside the same process do not need one.