# What Checkend stores

Source: https://checkend.com/docs/what-checkend-stores (Markdown: https://checkend.com/docs/what-checkend-stores.md)

> After Checkend accepts an event, it cleans the event, limits its size, and puts it in an issue. This guide tells what Checkend keeps, and what it removes.

**Figure: From accepted event to issue.** Each accepted event goes through four steps: 1) remove secrets (replaced with [Filtered]), 2) limit the size (the stored JSON is at most 64 KB), 3) group the event into an issue with fingerprint version 1, 4) apply the caps per project and write the issue and the event.

```mermaid
flowchart LR
  S["1. Remove secrets"] --> L["2. Limit size"] --> G["3. Group (fingerprint v1)"] --> C["4. Apply caps, write issue + event"]
```

## 1. Secrets are removed first

Checkend removes secrets before it stores anything. It replaces each secret with `[Filtered]`.

**Figure: Secrets become [Filtered].** Example: in an event, "password=hunter2" in a message becomes "password=[Filtered]"; "?api_key=sk_live_123&id=7" in a URL becomes "?api_key=[Filtered]&id=7"; the values of the Authorization and Cookie headers become "[Filtered]"; and the card number "4242 4242 4242 4242" (it passes the Luhn check) becomes "[Filtered]". Other text, such as "id 4821", stays.

Checkend removes:

- the values of sensitive keys, for example `password`, `token`, `secret`, `api_key`,
  `cookie`, `authorization` and `session`
- credentials after `Bearer`, `Basic` and `Token`
- JSON Web Tokens (JWTs)
- private key blocks (`-----BEGIN ... PRIVATE KEY-----`)
- Stripe, AWS, GitHub and Slack tokens
- passwords in URLs (`https://user:password@host`)
- the value after a sensitive key in text, for example `password=...`, `token: ...` or
  `"api_key": "..."`, also in URLs and query strings
- card numbers that pass the Luhn check

Examples of text with a key and a value:

| Before | After |
|---|---|
| `password=hunter2 id 4821` | `password=[Filtered] id 4821` |
| `login with password: hunter2 failed` | `login with password: [Filtered] failed` |
| `{"password": "hunter 2"}` (as text) | `{"password": "[Filtered]"}` |
| `https://api.example.com/v1?api_key=sk_live_123&id=7` | `https://api.example.com/v1?api_key=[Filtered]&id=7` |

A key is sensitive when it has a sensitive word: `my_password` and `myPassword` are sensitive,
but `mypassword` is not. A value without quotes ends at a space, a quote, `&` or `;`. A value in
quotes ends at its matching closing quote, also across spaces and line breaks (a backslash
escapes a quote). When there is no closing quote, Checkend filters to the end of the text.

Checkend removes secrets everywhere: in the stored event, in the issue title, and in the raw
payload.

### Your own scrub rules

You can add rules for each project with
[`PATCH /v1/projects/:project`](https://checkend.com/docs/api.md#patch-v1projectsproject):

- **`scrub_keys`**: up to 50 key names (letters, digits, `_`, `.` and `-`; any case). The value of
  any key with one of these names becomes `[Filtered]`: in the event, `extra`, `contexts`, tags,
  request headers and the raw payload. Prefer keys: they are fast and exact.
- **`scrub_patterns`**: up to 10 regular expressions (JavaScript syntax), each at most 200
  characters. Every match in every string becomes `[Filtered]`, also in the issue title, so
  grouping uses the filtered text.

```json
{"scrub_keys": ["patient_id", "x-internal-token"], "scrub_patterns": ["ACCT-[0-9]{8}"]}
```

Your rules apply after the default rules, within 60 seconds of a change.

Checkend checks each pattern before it saves it, so that a pattern cannot make ingest slow. It
rejects a pattern that is not valid, that has a backreference (`\1`), that has a nested
quantifier such as `(a+)+`, or that is slow on long inputs (it runs the pattern on test inputs of
up to 16,384 characters). Then the response is `400` with the reason. Saving can take up to about
a second.

## 2. Size limits

Checkend keeps the most useful parts of an event, and cuts the rest:

| Item | Limit |
|---|---|
| Exceptions per event | 10 |
| Frames per exception | 50 |
| Breadcrumbs | 50 |
| Tags | 50 |
| Depth of free-form data (`extra`, `contexts`, `user`) | 4 levels |
| Stored event, as JSON | 64 KB |

When an event is still too large, Checkend removes the least useful parts first (breadcrumb
data, breadcrumbs, `extra`, `contexts`, headers, source lines) until the event fits.

Strings also have length limits. Checkend keeps source lines only for in-app frames (frames from
your code, not from libraries).

Checkend removes invisible characters, control characters and bidi characters from text.

An event timestamp more than 30 days in the past, or more than 60 seconds in the future, is
replaced with the time when Checkend received the event. A breadcrumb timestamp that is not
valid becomes `null`.

## 3. Grouping into issues

An issue is a group of events with the same cause. Checkend makes a fingerprint for each event.
Events with the same fingerprint go into the same issue.

Checkend uses the first of these rules that applies (fingerprint version 1):

1. **The SDK fingerprint**, if your code sets one. `{{ default }}` works: it is replaced with the
   default fingerprint.
2. **The exception types and the in-app frames.** When an exception has no in-app frames,
   Checkend uses all its frames. A frame is its module (or file name) and its function. Line
   numbers and bundle hashes are not used, so a deploy that only moves code does not create new
   issues. A renamed function or module gives a new issue.
3. **The exception type and the message**, when no frame is usable: there is no stack trace, or
   the frames have only minified names (in a hashed or `.min.js` bundle, with names of 3
   characters or fewer). Checkend replaces the parts that change in the message: numbers, IDs,
   and quoted values.
4. **The message** for message events (`captureMessage`), with the same replacements.

**Figure: How Checkend picks the grouping rule.** Decision: if the SDK sets a fingerprint, use rule 1 (the SDK fingerprint; {{ default }} becomes the default fingerprint). Else, if the event has no exception, use rule 4 (the normalized message). Else, if the exceptions have usable frames (the in-app frames, or all frames when none is in-app), use rule 2 (exception types + frames: module or file name + function, no line numbers). Else (no stack trace, or only minified names), use rule 3 (exception type + normalized message).

```mermaid
flowchart TD
  E["Event"] --> A{"SDK sets a fingerprint?"}
  A -- yes --> R1["Rule 1: the SDK fingerprint"]
  A -- no --> B{"Has an exception?"}
  B -- no --> R4["Rule 4: the normalized message"]
  B -- yes --> C{"Usable frames?"}
  C -- yes --> R2["Rule 2: types + frames"]
  C -- no --> R3["Rule 3: type + normalized message"]
```

## 4. Caps per project

These caps protect your database from an error flood:

**Figure: The stored-events cap.** Example: an issue receives 250 events. times_seen counts all 250. stored_events stops at 100 (the EVENTS_PER_ISSUE default): Checkend counts the other 150 events but does not store them.

| Cap | Default | What happens above the cap |
|---|---|---|
| Stored events per issue | 100 | Checkend counts the event, but does not store it. |
| New issues per hour | 100 | Events that would make a new issue go into one "flood" issue. The hour is the last 60 minutes. Events of existing issues are counted as usual. |
| Tag keys | 200 | Checkend removes tags with a new key from the event. Tags with known keys stay. |

You can change the defaults. See [Settings and health](https://checkend.com/docs/settings.md).

> **Known limitation:** when an issue has 100 stored events, or in budget mode, Checkend does
> not store the event. If an SDK sends the same event again, it is counted again.

## 5. The raw payload

Checkend also keeps the original event, so that a later version of Checkend can process old
events again. The raw payload:

- has its secrets removed, like the stored event
- is compressed with gzip
- is kept only for events of 256 KB or less
- is in D1 when it is 64 KB or less after compression. A larger one goes to the `BLOBS` bucket
  (see [Settings and health](https://checkend.com/docs/settings.md)); without `BLOBS`, it is not kept, and only the
  processed event is stored.
- is deleted with its event (see [Data retention](https://checkend.com/docs/settings.md#data-retention))
- is used only to process events again. The API never returns it.

## 6. Untrusted data

The text in an event comes from the monitored application, so an attacker can control it. For
example, an error message can say "Ignore previous instructions". Checkend stores this text, but
the [issue brief](https://checkend.com/docs/mcp.md#the-issue-brief) and the MCP tools mark it as UNTRUSTED, so that a
coding agent treats it as data and not as instructions. See
[Untrusted data](https://checkend.com/docs/mcp.md#untrusted-data).