The Cloudflare Signals You Need Wired Up Before Something Breaks

Observability at the edge has a deadline, and the deadline is the start of the incident. Retention you did not configure, thresholds you did not choose and alert routes you did not test cannot be added retroactively. The work that decides how an outage goes is done weeks earlier, when nothing is on fire and nobody feels like doing it.

Three jobs that get confused with each other

Teams say "we have observability" when they mean one of three different things. It helps to separate them, because each one is built from a different part of the platform and each one fails in its own way.

Detection is a person finding out that something is wrong, soon enough to matter. It is built from alerts and from checks that run continuously. It fails silently: you only learn it was missing when a customer tells you first.

Diagnosis is narrowing the problem down while it is still happening. Is it one origin or all of them? One path? One country? One customer? It is built from analytics you can slice quickly and from live log access. It fails when the dashboard can only show you a total.

Reconstruction is the timeline you write afterward: what changed, who was affected, for how long. It is built from retained raw logs. It fails when the logs were never exported, or were exported without the fields you now need.

Most accounts are reasonably good at the third, partial on the second and thin on the first. That ordering is the reverse of what the business cares about.

What the defaults give you, and where they stop

Turn on a zone and you get dashboards for traffic, cache, security events and performance without doing anything. This is real value. It is also bounded in ways that only matter under pressure:

• Aggregated analytics are often sampled. They are excellent for shape and trend, and less reliable for answering "exactly which 40 requests failed for this customer".
• History is limited by plan. How far back you can look in the dashboard depends on what you pay for. If you need to compare today with the same weekday last quarter, check that the window actually reaches that far.
• Raw request logs are not kept for you by default. To hold per-request records for later, you have to push them somewhere or enable a product that stores them. This is the single most common surprise in a postmortem.
• Nothing pages anyone by default. A dashboard is a place to look after you already suspect a problem.

None of that is a criticism of the platform. A default cannot know which of your endpoints carries revenue, or what "normal" means for your traffic on a Monday morning.

Decide log retention while you still can

Log export is the one piece you cannot improvise. Cloudflare's log push mechanism delivers batches of log records to a destination you control, such as object storage or a SIEM. Set it up in calm conditions and make four decisions deliberately.

Which datasets. HTTP requests is the obvious one. Firewall or security events, DNS logs, audit logs of account changes and, if you run code at the edge, Workers trace events all answer different questions. The account audit log deserves special mention: a large share of incidents begin with a configuration change, and "who changed which rule at 14:02" is often the fastest route to a fix.

Which fields. Export jobs let you choose fields, and the defaults are not always enough. For request logs, make sure you have the edge response status and the origin response status as separate fields, the cache status, the origin response time, the security action taken and the rule that took it, and the Ray ID.

How long. Pick a retention period on purpose. Thirty days covers most operational questions. Security investigations and compliance obligations often need much longer. Storage is cheap compared with not having the data.

What must not be in there. Request logs can contain query strings, headers and client IPs. If your URLs carry tokens, email addresses or health information, your log bucket has just become a sensitive data store. Fix the URLs, drop the fields, or apply the same access controls you apply to the primary database.

One practical habit pays for itself the first time you use it: log the CF-Ray request header at your origin. That one identifier lets you join an edge log line to the application log line for the same request, which turns "the edge says 502, the app says nothing" into an answer.

The short list of signals worth interrupting someone for

An alert is a claim that a human should stop what they are doing. Very few signals justify that. The table below is a starting set, not a complete one.

Notice two things. First, every "page" row is about user-visible failure, not about resource usage. Second, the rows are scoped to a path or a pool. An alert on the zone-wide error rate is nearly useless on a busy site: a login endpoint can be completely down while the overall number barely moves, because static assets keep succeeding.

The two failures that never trip a simple threshold

Legitimate traffic being blocked. When a WAF rule, a rate limit or a bot rule stops a real customer, nothing looks broken. The edge returns a 403 or a challenge, exactly as designed. Your origin sees less traffic and looks healthier. No error budget burns. The only trace is a security event and, eventually, a support ticket or a quiet drop in conversions. To catch it, watch the ratio of mitigated requests to total requests per critical path, and alert on a change in that ratio, not on its absolute value. Also review the top blocking rules weekly. A rule that suddenly accounts for most of your blocks either caught an attack or broke something, and you want to know which.

Slow drift. A latency that grows three percent a week never crosses a line drawn at "twice normal". Static thresholds catch cliffs, not slopes. Drift needs a comparison against a baseline: this week's p95 against the same window four weeks ago, reviewed on a schedule. This does not need a pager. It needs a recurring report with a named owner, which is a less exciting artifact and a more reliable one.

Tuning: an ignored alert is worse than a missing one

The opposite failure is alerting on everything. Within a few weeks the channel is muted, and the one alert that matters arrives into silence.

A few rules keep the signal credible:

1. Two tiers only. A page means act now. A ticket means act this week. Anything that is neither should be a dashboard, not a notification.
2. Require duration. A threshold that must hold for several minutes filters out the blips that resolve on their own.
3. Set a floor on volume. A 50 percent error rate across four requests at 3 a.m. is noise. Require a minimum request count.
4. Every alert names an action. If the on-call person cannot say what they would do on receiving it, delete it or rewrite it.
5. Review after every page. Was it real? Was it early enough? Could it have been a ticket? Adjust the threshold the same week.

A healthy rotation receives a small number of pages, and each one is taken seriously. If people have stopped reacting quickly, the problem is the alert set, not the people.

Route alerts to where the response happens. Cloudflare notifications can be sent by email and, depending on plan, to webhooks and incident tools. Email to a shared inbox is not an on-call system. Test the route end to end: trigger a test notification and confirm a phone actually rings.

Edge code has its own blind spots

If you run Workers, part of your application now lives where your traditional APM cannot see. Treat it like any other service:

• Emit structured logs (JSON, with a request identifier and the route) instead of free text, so they can be filtered later.
• Record custom metrics for the things only you understand, such as "fallback path used" or "upstream retried". An analytics datastore at the edge is well suited to this kind of high-cardinality counter.
• Watch CPU time and subrequest counts against platform limits. A Worker that fails because it hit a limit looks different from one that threw an exception.
• Tie metrics to deployments. Knowing that the error rate changed at the moment a new version went out shortens diagnosis more than any other single fact.

Do not watch Cloudflare only from inside Cloudflare

Everything above depends on the edge reporting on itself. Add at least one view from outside: a synthetic check, run by a separate provider from a few regions, that performs a real transaction against your public hostname. It covers the cases where the edge is fine but your DNS is wrong, where a rule blocks a specific region, or where the platform itself is degraded. Subscribe to the provider's status notifications as well, and send them to the same place as your own alerts.

Run a second check directly against the origin, bypassing the proxy. During an incident, the first diagnostic question is whether the origin is healthy. Having that answer already on a screen saves ten confused minutes.

A pre-incident checklist

Use this as a working review. Every "no" is a task to schedule now.

• Request, security and audit logs are exported to storage we control, with a stated retention period.
• Exported fields include edge status, origin status, cache status, origin time, rule ID and Ray ID.
• The origin logs the Ray ID.
• We have listed our critical paths, and each has an error-rate alert with a duration and a volume floor.
• Health checks run against every origin from more than one region.
• We track mitigated-versus-total requests on critical paths.
• Someone reviews latency and cache trends on a schedule.
• Alerts reach a paging tool, and we have tested that route.
• An external synthetic check runs against the public hostname and against the origin.
• Log storage has been reviewed for sensitive data.
• The runbook says where to look first for each alert.

Rehearse it once

The cheapest test is a drill. Pick a quiet afternoon. In a staging zone, or carefully in production, break something small: point a health check at a bad port, deploy a rule that blocks a test path, return 500 from a canary route. Then time three things: how long until an alert fires, how long until a person acknowledges it, and how long until that person can say which component is at fault.

Then replay your last real incident. Take its timeline, find the first moment the data showed a problem, and ask what alert would have fired then. The gap between that moment and the moment you actually found out is the number this work is meant to shrink.

If your main origin started failing right now, who would find out first: your team or a customer?

If you are not sure, that is the gap worth closing, and it is cheaper to close before the next incident than after it. Trivo implements Cloudflare logging, analytics and alerting for teams with small on-call rotations, sized to what people can act on. Bring your last incident timeline and we will show you where an alert would have fired.