Service levels¶

The full status page is at https://doc.cheap/status. It is the page to open when something looks wrong. It carries the current state of each part of the service and ninety days of history, day by day. It also carries recognition processing times, past incidents and anything scheduled. The two tiles at the bottom of this page are a short extract of it.

The JSON behind it¶

The same numbers the page draws are served as JSON by the API. The routes need no key, no session and no cookie, and they carry nothing about anybody. They answer every origin (access-control-allow-origin: *), so a dashboard of your own may read them straight from a browser.

An Atom feed at https://doc.cheap/status/feed.xml carries one entry per incident, with its impact, its state and every update newest first. That feed is the subscription channel. Point a feed reader at it and you hear about an incident without anybody holding your email address, and with nothing to unsubscribe from.

component is one of api, recognition, dashboard, docs, topups (default api); days defaults to 90 and is capped at 400.

Caching, so polling is cheap:

• summary.json — cache-control: public, max-age=30, stale-while-revalidate=300 and a strong ETag. The document is rebuilt once a minute, so a poller that sends If-None-Match gets 304 Not Modified most of the time.
• history.json — public, max-age=300, also with an ETag.
• incidents.json — public, max-age=60.

Every document starts with a schema number: branch on it before parsing the rest, because it changes when the shape does. One address may make 60 requests a minute across these routes; past that they answer 429. If the document cannot be rebuilt, the last good copy is served with "stale": true rather than an error. A reader then sees figures that are explicitly out of date, instead of a page that will not load.

What is measured¶

Five components, each judged by its own evidence.

A non-core component is hidden while it is healthy. A green row for a path most visitors never take spends attention for nothing.

The canary¶

Availability of recognition is measured by a synthetic recognition, once a minute, not by a port check.

A drawn specimen is posted to the service's own POST /v1/scans over loopback, with retain_hours: 0, under a deadline of the engine's own timeout plus five seconds. It goes through the routing, the maintenance gate, the authentication guard, the rate limiter and the engine — the same path a customer's call takes.

Only a scan proves a scan works. Three consecutive failures mark recognition down.

The canary carries a credential of its own with its own rate bucket, and it records nothing while a maintenance window is open.

The document it sends is synthetic, and under a real engine it is not recognized. The figures prove the path is alive; they do not prove a customer's document reads correctly. Planned maintenance is announced before it starts and is not counted as downtime. The page lists what is scheduled for the next 30 days, and anything in progress now.

The two latency figures, and what each one is¶

They are not the same measurement, and neither is a substitute for the other.

recognition_ms is a percentile: the raw durations are kept and the percentile is taken over them. The figure beside its samples and window_hours says how many runs it is a percentile of.

engine_typical_slow_minute_ms is not a percentile, and it is never called one. Each minute's own p95 is sampled, and the median of those samples is published. It answers what a slow minute looks like. A percentile over a whole day cannot answer that: a day's p95 is dominated by whichever minute was worst.

The page labels it "a typical slow minute" for that reason.

Incidents¶

An incident is opened automatically after five minutes of a major outage on a component, with a fixed sentence that names nothing about the cause. A second component that goes down joins that incident rather than starting another one. Fifteen minutes after everything it names is working again it moves to monitoring, and it is resolved by a person. The vocabulary is the one other status pages use — investigating, identified, monitoring, resolved, with an impact of none, minor, major or critical.

The headline at the top of the page is derived from the components, never set by hand, and it treats them differently on purpose. An outage of the API, recognition, the dashboard or the documentation is the product not working, and the banner says so. An outage of crypto top-ups is named in the banner and capped at "degraded," while its own row still reports the outage in full. A banner that shouts about a path most readers never take is a banner people learn to ignore.

How uptime is counted¶

Atlassian Statuspage's formula, carried out unchanged, so the figure means the same thing as on the other status pages you read:

counted  = total minutes − maintenance minutes
downtime = major-outage minutes + 0.3 × partial-outage minutes
uptime   = (1 − downtime ÷ counted) × 100

Three consequences worth knowing before you compare numbers:

• Degraded is not downtime. A service that was slow was serving. Counting it as downtime would make "uptime" mean "was never less than perfect."
• Maintenance leaves the denominator. Announced work is removed from the counted minutes, not moved into the up column: it neither helps nor hurts the figure.
• A partial-outage minute is worth 0.3 of a lost one. Some callers got through, so the minute was not lost; some did not, so it was not whole.

Over a longer period the sums are carried, not the ratios. The 30- and 90-day figures are minute-weighted, never the average of the daily percentages. A day with four minutes of traffic does not get the same say as a day with fourteen hundred.

The windows and the error budget¶

Three windows are published, and each answers a different question.

The availability target is published beside the figures, and so is the error budget. That is the number of downtime minutes the target allows over 30 days, and the share of them the period has spent. A target of 99.9% over 30 days is about 43 minutes.

Where a deployment publishes a speed target, the page also reports the share of recognitions that came in under it. With no target set, the durations are reported and nothing is claimed about them.

Daily rows are kept for 400 days, so a 90-day window always has a full year behind it.

The per-process snapshot¶

GET https://api.doc.cheap/status.json is a different and much smaller thing. It carries one API process's request latency — p50, p95 and p99 over a rolling window of recent requests. It also reports how long that process has been up since its last restart. It is useful for looking at a single instance, and it says nothing about the service over time. For that, read the status page above.

If something looks wrong¶

Retry first: most 500s are transient. The code in the body says which failure it was, and errors has a page for each.

For anything persistent, quote the request_id from the failing response, and the event_id where the body carries one. Both identify the exact request in the logs.