Guides
Access logs and access keys
Traffic analytics, access log sampling, search, and export, analytics storage modes, and creating and revoking access keys.
Concepts
| Term | Definition |
|---|---|
| Analytics | Complete per-minute counts from nodes: requests, traffic, cache hits, status codes, and approximate top URLs and IPs. |
| Access logs | Individual requests recorded at a sample rate. Bounded diagnostic data, not a lossless audit record. |
| Sample rate | The share of requests recorded; a percentage in the UI, an integer in 1/10,000 units (0–10000) in the API. |
| Analytics mode | The value of EDGEWEIR_ANALYTICS: lite (PostgreSQL, default) or clickhouse. |
| Access key | A key (prefix ewk_) that calls /api/v1 as the console account; created and revoked in Personal settings. |
View analytics
| Location | Scope |
|---|---|
| Overview | All sites and nodes, including Top sites and Top nodes |
| Sites → Analytics tab | One site |
- Open a page from the table and select a range: Last hour, Last 6 hours, Last 24 hours, Last 7 days, or Last 30 days.
- Read Total requests, Data transferred, Cache hit ratio, Peak bandwidth, 4xx rate, 5xx rate, Status codes, Top URLs (approximate), and Top IPs (approximate); click a metric for details.
- Click Refresh to reload.
- To act on an entry, click ⋯ at the end of its row: a top IP offers Ban IP (with the site filled in on a site's Analytics tab, with the Global scope on the overview); a top URL on a site's Analytics tab offers Purge URL, which expands the path to each of the site's domains that is not a wildcard and lists the URLs before it submits the purge.
Analytics data
| Item | Behavior |
|---|---|
| Retention | Minute detail for 7 days, hourly rollups for 90 days, daily rollups for 365 days; in clickhouse mode the console charts and alerts still use the PostgreSQL rollups |
| Late data | Late minute data triggers a recompute of its hour and day; data older than 7 days is discarded |
| Rollups | A background job rolls up incrementally every minute; UTC boundaries are fixed; long-range charts read completed hourly rollups plus detail not yet rolled up |
| Top URLs / IPs | Bounded Space-Saving estimates computed on the node and labeled "approximate": each worker tracks at most 128 site buckets per minute with up to 32 candidates each; the top 50 per bucket are kept; infrequent items can be missed; not for billing |
| URL content | No query strings or headers; paths over 512 bytes or containing ? or control characters are not counted; paths themselves can still contain business identifiers |
| Counter ceiling | A cumulative counter per node, site, and time bucket stops at 9,007,199,254,740,991 (Number.MAX_SAFE_INTEGER) |
| Deletion | Deleting a node keeps the sites' historical analytics and logs; deleting a site deletes its analytics and its logs in PostgreSQL |
Reporting and deduplication
| Item | Behavior |
|---|---|
| Batches | Nodes write analytics batches with a monotonic sequence number to traffic-spool.json (0600) in the state directory, then report over the node channel (mTLS) |
| Deduplication | A lost receipt, a node restart, or a failed local receipt write resends the same sequence; the console updates the counts and the node's cursor in one transaction, so a repeated sequence is not counted twice; a node that loses its local sequence file resynchronizes from the console cursor |
| Offline queue | Up to 10,000 buckets and 32 MiB; overflow is dropped and logged |
| In-memory counts | Counts not yet moved to disk expire after 2 hours; a crash before the first persist loses in-memory counts |
| Semantics | Deduplication of retried batches, not billing-grade per-request exactly-once |
| Old nodes | Reports without a sequence number are refused; such nodes show Upgrade required in Clusters & nodes |
Enable access logs
- Open Sites, select the site, and open the Logs tab.
- Select 1%, 10%, or 100% in Access log sample rate. The choice is saved at once and publishes a new configuration revision; the console shows Saved.
- Verify: after some requests, click Search in the query form; records appear.
| Field | Values | Default | Effect |
|---|---|---|---|
| Access log sample rate | Off / 1% / 10% / 100% | Off | Share of requests recorded; the API takes an integer of 0–10000 in 1/10,000 units |
| Recorded fields | Not recorded |
|---|---|
| Time, client IP, method, Host, path, status, bytes sent, duration (ms), cache status, sample rate, node ID | Query strings, request headers, cookies, request and response bodies |
With Record JA4 in access logs on (Security → Challenges of the site), logs also record the JA4 TLS client fingerprint (empty over plain HTTP): the table gets a JA4 column and the CSV a ja4 column. Once it is off, the console stops keeping the field. JA4 format: JA4.
With OWASP CRS on, requests that matched rules also record the rule IDs (at most 16 per request, ascending) and whether CRS blocked them: the table gets a CRS column (rule IDs and a Blocked badge) and the CSV wafRuleIds (space-separated) and wafBlocked columns.
Rules of the configuration phase can set a sample rate for the requests they match with Log sample rate (%); those logs are kept even while the site's sample rate is Off, see Override settings.
Access logs need the node capability access-logs-v1, and JA4 also ja4-v1. A configuration rollback keeps the current sample rate and JA4 setting and never re-enables logging that was turned off.
Search and export logs
-
On the Logs tab, enter From and To, and optionally Status, Client IP, Path prefix, and Request ID.
-
Click Search.
-
For a file, click Export CSV.
-
To act on a request, click ⋯ at the end of its row; the action runs on the same page:
Action Behavior Ban IP Opens the ban dialog with the site and the client IP filled in; the Scope can change to Global, see Bans Purge URL After a confirmation, creates a URL purge of the request's host and path, see Purge and prefetch Exclude CRS rule N Only on rows that matched CRS rules, one item per matched rule; after a confirmation, the rule ID is added to the site's exclusions, see OWASP CRS. Initialization, blocking evaluation and correlation rules (901xxx, 949xxx, 959xxx, 980xxx) are not offered: excluding them turns blocking off The toast that follows links to the bans, the purge tasks or the CRS settings.
| Item | Behavior |
|---|---|
| Time range | Defaults to the last hour; the earliest is UTC midnight 6 days ago and the latest 5 minutes from now; the end must be after the start |
| Filters | Status: exact; client IP: exact; path: prefix; request ID: exact |
| Request ID | Each entry shows the request ID the node settled (the same as the X-Request-Id response header and the one on error pages), in the CSV as the requestId column; empty for logs of older nodes |
| Rows | The UI shows at most 100 rows ("Showing the first 100 rows. Narrow your search."); CSV holds at most 1,000 rows ("Exported the first 1,000 rows. Narrow the time range for other records.") |
| CSV | Every cell is quoted with quotes escaped; values starting with =, +, -, or @ get a leading ' so spreadsheets do not treat them as formulas |
| Callers | A console session or an access key, read-only keys included; service accounts cannot call it |
Log collection and storage
| Item | Behavior |
|---|---|
| Collection | Nodes report in batches of up to 1,000 over mTLS; a durable per-node cursor stops retries from writing twice |
| Node memory queue | About 2,000 entries (8 MiB); when full, new entries are dropped |
| Node disk queue | logs-spool.json (0600) in the state directory, up to 10,000 entries and 32 MiB; when full, the oldest unsent batches are dropped with a warning |
| Loss | A process or host crash between the end of a request and the batch reaching disk loses those entries |
lite storage | PostgreSQL partitions by UTC day, keeping today and the previous 6 days; a background job maintains partitions every minute |
clickhouse storage | ReplacingMergeTree with daily partitions and a 7-day TTL, deduplicated with FINAL at query time; when a ClickHouse write fails the batch is not acknowledged and the node retries, with no fallback to PostgreSQL |
Use ClickHouse storage
-
In the console's
.env, enable theanalyticsprofile and set the analytics mode and the ClickHouse password:.env COMPOSE_PROFILES=analytics EDGEWEIR_ANALYTICS=clickhouse EDGEWEIR_CLICKHOUSE_PASSWORD=<password> -
Start the Compose deployment; with
COMPOSE_PROFILES, every laterdocker composecommand includes ClickHouse:docker compose up -d -
Verify: on the System settings page, Analytics shows
clickhouse.
Defaults of EDGEWEIR_CLICKHOUSE_URL, EDGEWEIR_CLICKHOUSE_DATABASE, and EDGEWEIR_CLICKHOUSE_USER and settings for an external ClickHouse are in Environment variables.
| Item | Behavior |
|---|---|
| Switching | Affects new writes only; historical logs and analytics are not migrated |
| Minute analytics | Also written to the ClickHouse minute_stats table (ReplacingMergeTree versioned by the node batch sequence); use FINAL for direct analysis |
| Charts and alerts | Still use the exact PostgreSQL minute, hour, and day rollups, so both modes count the same way; sampled logs are never used as full traffic counts |
| Site deletion | The console stops authorizing queries for the site's logs at once; raw ClickHouse data expires by the 7-day TTL |
Create an access key
-
Open Personal settings from the user menu (bottom of the sidebar) and find the Access keys card.
-
Enter Name (1–64 characters;
defaultwhen empty) and select Scope. -
Click Create.
-
Copy the key shown. It is shown once (Shown once).
-
Verify:
curl -s -o /dev/null -w '%{http_code}\n' -H 'x-api-key: <key>' https://console.example.com/api/v1/sitesThe output is
200.
| Field | Values | Default | Effect |
|---|---|---|---|
| Name | 1–64 characters | default | Display name in the list |
| Scope | Read only / Read and write | Read and write | Read-only keys call only GET endpoints and rules.validate; other endpoints return 403 (ACCESS_KEY_READ_ONLY) |
Request format and endpoints are in API and endpoints.
Revoke an access key
- In the Access keys card of Personal settings, click Revoke key for the key and confirm.
- Verify: the key shows Revoked; requests with it return 401.
| Item | Behavior |
|---|---|
| List | The Access keys card lists every key, with scope and "Last used: …" |
| Identity | A key calls the API as the console account; the audit log shows the actor type AccessKey |
| Issuing | Keys can be created only in a signed-in console session (the Personal settings page, or accessKeys.create over /rpc); no access key, including read-write and legacy keys, can create new keys: POST /api/v1/access-keys returns 403 (ACCESS_KEY_SESSION_REQUIRED) |
| Legacy keys | Legacy keys without a scope keep read-write access; revoke and recreate them by purpose |
Limits
| Item | Description |
|---|---|
| Sample rate | The UI offers Off, 1%, 10%, and 100%; other rates through the API |
| Log retention | About 7 days in both storage modes; not configurable |
| Completeness | Access logs have bounded queues and can lose entries; they are not an audit ledger; analytics are not billing-grade counts |
| Top URLs / IPs | Estimates that can miss infrequent items |
| History migration | Switching the analytics mode migrates no history |
Troubleshooting
| Symptom | Cause | Action |
|---|---|---|
| "No matching logs" | Sample rate off or too low; time range before retention; the node has not applied the revision that enables logs or lacks access-logs-v1 | Check the sample rate, time range, and the node's Applied revision |
| "End time must be after start time" | Invalid time range | Adjust the times |
| "Showing the first 100 rows. Narrow your search." | More than 100 matches | Narrow the range, add filters, or export CSV |
| "Exported the first 1,000 rows. …" | More than 1,000 matches | Export in several ranges |
| Analytics charts are empty | The site has no traffic, or nodes are not reporting | Check node heartbeats and whether the site's domains are published |
| A node shows Upgrade required | The node does not support sequenced analytics reports or a capability the current configuration needs | Upgrade the node, see Node upgrades |
| "Sign in to create an access key" | The create endpoint was called with an access key | Create keys in a signed-in console session |
| "This access key is read only" | A read-only key called a write endpoint | Create a read-write key |