---
title: "Storage · coding.sh docs"
description: "The SQLite index, the content-addressed blob store, sinks, and how to plug in a remote store."
canonical: https://coding.sh/docs/storage
human_url: https://coding.sh/docs/storage
markdown_url: https://coding.sh/docs/storage.md
agents_url: https://coding.sh/agents.md
llms_url: https://coding.sh/llms.txt
updated: 2026-11-11
---

# Storage

The SQLite index, the content-addressed blob store, sinks, and how to plug in a remote store.

coding.sh keeps two stores with different shapes, plus any number of write-only mirrors. All three are interfaces (`src/core/types.ts`), chosen by URL scheme in the config.

## Index (built)

Small, queryable rows: sessions, events, eval tasks, harness bundles, runs, labels, datasets and experiments. The default is one SQLite file through the built-in `node:sqlite`, in WAL mode, with an FTS5 table over prompt, reply and tool text for `coding.sh search`. Writes are idempotent on event id, so re-ingesting the same spool changes nothing.

[`coding.sh sql "select …"`](https://coding.sh/docs/cli/sql) runs one read-only query against it. [`coding.sh export sessions|events|labels`](https://coding.sh/docs/cli/export) streams rows as JSONL or CSV, on every store. The tables and columns are in [Query and export](https://github.com/jackulau/coding.sh/blob/main/docs/reference/query.md).

## Blobs (built)

Large, write-once content: full tool output, raw source payloads, diffs. Event payloads larger than `maxInlineBytes` (4 KB) move to a blob and the event keeps a reference. Each blob is keyed by the sha256 of its uncompressed bytes, so the same `AGENTS.md` read in a thousand sessions is stored once.

Blobs are compressed with zstd level 3. One that compresses to under 4 KiB is a row of `blobs/pack.db` (SQLite); a larger one is `blobs/ab/cd/<sha256>.zst`, written to a temp file then renamed, so readers never see half a blob.

## Config

config.toml

```
# ~/.coding.sh/config.toml  (coding.sh config get prints the effective values)
[store]
index = "sqlite://~/.coding.sh/index.db"
blobs = "file://~/.coding.sh/blobs"

[capture]
mode = "full"                 # or "metadata": no prompts, code or tool output
redact = ["secrets"]          # "secrets", "env", or "none"
redactPatterns = []           # extra regexes
exclude = []                  # repo globs never captured
maxInlineBytes = 4096         # larger payloads move to blobs

[retention]
keepSpoolDays = 7
toolOutputDays = 90
```

A repository can carry its own `.coding.sh/config.toml`, but because it arrives with `git clone` it is untrusted: it may only tighten capture (add redaction, exclusions or metadata mode) and adjust snapshot settings. It can never add a sink or move a store, which would let a repository send your sessions somewhere.

## Redaction and metadata mode

- Redaction runs before anything is written to the index, blobs or sinks. The built-in `secrets` rules key on vendor token prefixes and on secret-looking names next to values, so code, paths and hashes pass through.
- `capture.mode = "metadata"` keeps ids, timings, kinds, counts and usage, and drops prompts, code and tool output.
- A session in which a vendor token was redacted gets the `secret:exposed` tag: the agent and its model provider saw it, so rotate it. `coding.sh sessions --flag secret:exposed` lists them.
- [The model API proxy](https://coding.sh/docs/how-to/capture-with-proxy) redacts each request and response before storing it, and never stores `Authorization`, `x-api-key` or cookies.

A value that got past redaction can be removed afterwards. Put it in a file, one value per line, and count the copies first:

shell

```
coding.sh purge --literal leaked.txt --dry-run   # count the copies
coding.sh purge --literal leaked.txt             # replace them
coding.sh purge --rule 'sk-ant-api03-[A-Za-z0-9_-]{80,}'   # every match of a shape
```

Every copy is replaced, in the index, blobs, spool and eval runs: with `[redacted:purged]` in the index and blobs, with asterisks in spool and eval run files. SQLite store only; rotate the secret too. [purge](https://coding.sh/docs/cli/purge) has the details, and [Storage, sinks and privacy](https://github.com/jackulau/coding.sh/blob/main/docs/guides/privacy.md) covers every redaction rule.

## Sinks (built)

A sink mirrors events to somewhere else. Each sink keeps its own cursor and retries with backoff, so a remote that is down never blocks local capture. Remote sinks default to `metadata` mode: content leaves your machine only when you set `mode = "full"`.

config.toml

```
[[sinks]]
kind = "otlp"                           # OTLP/HTTP JSON traces to {endpoint}/v1/traces
name = "collector"
endpoint = "http://localhost:4318"
mode = "metadata"                       # the default for every sink

[[sinks]]
kind = "http"                           # POSTs {schema, v, mode, events, sessions}
endpoint = "https://relay.example.com/ingest"
mode = "full"
headers = { Authorization = "Bearer ${RELAY_TOKEN}" }   # read from the environment
```

- `otlp`: OTLP/HTTP JSON. A session becomes an `invoke_agent` span, tools `execute_tool` spans and replies `chat` spans, with `gen_ai.*` attributes pinned to semconv 1.37.0. Span ids derive from event ids, so a resent batch de-duplicates.
- `http`: posts coding.sh/trace JSON as is, for a collector or relay that speaks it.
- Sinks mirror every event local ingest stores, from hooks and from imported transcripts, and events from the [OTLP receiver](https://coding.sh/docs/otel).
- `coding.sh serve` mirrors every batch it commits to the sinks in its own config, so a team can set one there instead of on each laptop.

## Plugging in a remote store

Built-in schemes are `sqlite://` and `memory://` for the index, and `file://` and `memory://` for blobs. For any other scheme the CLI imports an npm package by convention:

- `coding.sh-store-<scheme>` exporting `indexStore(url, config)` and/or `blobStore(url, config)`;
- `coding.sh-sink-<kind>` exporting `sink(sinkConfig, config)`.

The package must be resolvable from where coding.sh is installed. A missing package gives an error that names it; a package that fails to load surfaces its own error.

javascript

```
// coding.sh-store-postgres/index.mjs (the real package, abridged)
export function indexStore(url, config) {
  return PgIndexStore.open(url);   // resolves to an IndexStore
}

// coding.sh-sink-kafka/index.mjs (hypothetical package)
export function sink(sinkConfig, config) {
  return { name: sinkConfig.name ?? "kafka", mode: sinkConfig.mode, async write(events, sessions, blobs) { /* … */ return { ok: true }; } };
}
```

The interfaces a plugin implements, abridged. A test suite in `src/store/conformance.ts` runs against the built-in stores and can be pointed at a new one.

src/core/types.ts (abridged)

```
interface IndexStore {
  readonly kind: string;
  migrate(): Promise<void>;
  close(): Promise<void>;
  putEvents(batch: AgentEvent[]): Promise<{ inserted: number }>;   // idempotent on event.id
  upsertSession(session: Session): Promise<void>;
  getSession(id: string): Promise<Session | undefined>;
  listSessions(filter?: SessionFilter): Promise<Session[]>;
  listEvents(sessionId: string, opts?: { kinds?: string[]; limit?: number; offset?: number }): Promise<AgentEvent[]>;
  search(text: string, opts?: { limit?: number; agent?: string }): Promise<SearchHit[]>;
  getCursor(source: string): Promise<number>;
  setCursor(source: string, offset: number): Promise<void>;
  // plus tasks, harness bundles, runs, labels, referencedBlobs(),
  // and optional datasets, experiments, judges, kv state and compact()
}

interface BlobStore {
  readonly kind: string;
  put(data: Uint8Array | string): Promise<string>;   // returns the sha256; dedupes
  get(ref: string): Promise<Uint8Array | undefined>;
  getText(ref: string): Promise<string | undefined>;
  has(refs: string[]): Promise<boolean[]>;
  list(): AsyncIterable<string>;
  delete(ref: string): Promise<void>;
}

interface Sink {
  readonly name: string;
  readonly mode: "full" | "metadata";
  write(batch: AgentEvent[], sessions: Session[], blobs: BlobStore): Promise<{ ok: boolean; error?: string; retryAfterMs?: number }>;
}
```

Postgres, ClickHouse and S3: `coding.sh-store-postgres`, `coding.sh-store-clickhouse` and `coding.sh-store-s3` (S3, R2, MinIO) are on npm and back [production mode](https://coding.sh/docs/production). Install one next to coding.sh. The npm package ships the TypeScript types for `coding.sh/store` and `coding.sh/store/conformance`, so a plugin built outside the repository imports them.

## Retention and gc

`coding.sh gc` deletes spool files older than `keepSpoolDays` once every sink has them, tool-output blobs older than `toolOutputDays` (diffs and eval tasks are kept), and unreferenced blobs, and trims the judge cache to its size cap. `--compact` also runs SQLite `VACUUM`; `--dry-run` shows what would go.

To set coding.sh up for a user, follow the agent workflow at [/agents.md](https://coding.sh/agents.md).