Nov 6, 2026 · 63 changes

coding.sh v0.1.0

The first public release, still pre-1.0. coding.sh records what your AI agents do, with coding agents first-class, on your own machine, shows it in a local viewer, and turns those sessions into evals and A/B experiments. It also runs as a server on SQLite or Postgres, with SDKs for apps that call model APIs.

Install 6

# curl -fsSL https://coding.sh/install | sh checks for Node 22.15+, installs with npm, runs init and opens the viewer.
# Or npm install -g coding.sh && coding.sh init. init shows one plan and asks one Y/n. Install guide.
# coding.sh with no command opens the viewer, and sets up capture on its first run.
# Commands print one dim line when a release is out, checked in the background at most once a day.
# coding.sh update installs the new release. The check is off in CI and can be turned off.
# coding.sh uninstall restores each agent config byte for byte if nothing else has edited it since.

Capture 13

# Hooks record prompts, tool calls, file changes, turns and errors to a local spool, then ingest them into sessions.
# Secrets are redacted before anything is stored. Privacy.
# run spools secret flag values (--openai-api-key X, --api-key=X) redacted, and ingest redacts them in any argv.
# Pi, Goose, Copilot CLI, Droid, Gemini CLI and Antigravity sessions seen by hooks and by ingest --backfill store each call, reply and turn once, and count tokens once.
# A session seen by hooks and by ingest --backfill keeps one session start and one end, for every agent.
# A Claude Code or Codex compaction seen by hooks and by ingest --backfill is stored once; Codex transcripts no longer record each compaction twice.
# Cursor prompts and replies seen by hooks and by ingest --backfill are stored once, though Cursor transcripts carry no timestamps.
# coding.sh run -- <cmd> wraps any agent and snapshots the tree before and after in shadow git, not your repo.
# Every model call becomes an llm.call event with model, tokens, duration and time to first token.
# Subagent starts and ends are captured and linked to their child sessions.
# No ingest step: reads, the viewer and MCP ingest new events first; a session end starts a locked background ingest.
# coding.sh collector takes OTLP/HTTP traces and logs, protobuf or JSON, on 127.0.0.1:4318. OTLP setup.
# The receiver reads OTel GenAI semconv, OpenInference, the Vercel AI SDK, and Claude Code, Codex and Gemini CLI logs.

Agents 5

# Verified on real sessions: Claude Code, Codex and Antigravity. Agent matrix.
# Fixture-tested so far: opencode, cursor, gemini-cli, copilot-cli, amp, pi, droid, goose, t3code and aider.
# Every adapter has fixture tests, and an ingest --backfill path where the agent keeps history.
# T3 Code threads import from its local store; Codex and Claude Code runs in T3 count once, agent in provider.
# coding.sh usage --by agent-model compares tokens per turn and cache-read share across harnesses.

Cost 3

# Calls with tokens and no cost get one from a dated price table (Claude, OpenAI, Gemini, open models), marked estimated; billed and reported costs win. Where cost comes from.
# [prices."<model>"] in config.toml sets your own rates; usage --reprice recomputes stored estimates.
# A model call nested in another, or a repeated response id, counts once in session totals.

Storage 2

# One SQLite index (node:sqlite, no native deps) and a content-addressed, zstd-compressed blob store. Storage.
# OTLP and HTTP sinks send metadata only by default.

Viewer 1

# coding.sh ui serves a local viewer: sessions, search, a timeline of turns, tools and subagents, and outcome labels.

MCP 2

# coding.sh mcp lets agents query sessions, failures, latency, usage, subagents and experiments over stdio. MCP.
# init registers it with Claude Code, Codex, Cursor, OpenCode and Gemini CLI. --no-mcp skips that.

CLI 1

# 27 commands, from init and sessions to eval, exp and serve, all with --json output. CLI reference.

Evals 7

# Repo-replay tasks start from the snapshot taken before the agent ran. Evals.
# eval validate checks verifiers: a do-nothing run fails, the reference fix passes k of k, near-misses fail, nothing leaks.
# Datasets with deterministic graders, command graders and LLM judges.
# Judges are calibrated on human labels with Cohen's kappa on a held-out split. Reports flag uncalibrated judges.
# Experiments run two arms on the same cases: a paired bootstrap CI of the difference and exact McNemar on flips.
# Harness compare adds per-arm Wilson intervals and the first step where the traces diverge.
# Tool evals for CLI and MCP surfaces: description size, schema checks, per-task tool-call metrics in a sandbox.

Production 13

# coding.sh serve runs ingest (OTLP/HTTP and POST /v1/ingest), the viewer and its API on one port. Production.
# Set the store with CODING_SH_STORE_URL or DATABASE_URL, and blobs with CODING_SH_BLOB_URL (file, postgres, s3).
# Writes are acked after commit, a full queue answers 429 with Retry-After, and retries never duplicate.
# Ingest keys pick the project and read keys open the viewer. serve keygen and serve healthcheck help.
# /healthz, /readyz, Prometheus /metrics, and one writer lock with --role viewer replicas.
# CODING_SH_RETENTION_DAYS sets retention and CODING_SH_CAPTURE_MODE=metadata drops message text and tool I/O.
# coding.sh-store-postgres keeps the index, and blobs by default, in Postgres. Passes the store conformance suite.
# Store URLs take sqlite://, postgres:// or postgresql://.
# coding.sh-store-s3 keeps blobs in S3, R2 or MinIO. SigV4, no AWS SDK, credentials only from AWS_* env vars.
# A Dockerfile with both store plugins, a Compose stack (serve, Postgres, MinIO), and examples in examples/production.
# Deploy recipes for Fly, Railway and Render. Deploy guides.
# --store <url> or CODING_SH_STORE_URL points sessions, show, search, mcp, dataset and exp at the serve database.
# sessions and search take --project.

SDKs 3

# @codingdotsh/sdk for TypeScript: observe(client) for OpenAI and Anthropic, observeFetch(), aiSdkTracer(), withSession.
# coding-sh for Python: observe(client) and OTel setup.
# Both batch, retry on 429 and never throw into your app. Not on npm or PyPI yet.

Integrations 5

# Sessions sent through the Vercel AI Gateway are tagged gateway:vercel. AI Gateway.
# coding.sh gateway status|enrich adds billed cost, serving provider, client session join and the vmc/ model per call.
# The receiver maps gateway trace drains and AI SDK gateway spans.
# compare --gateway and the gateway grader provider run arms across models with one key.
# eve: span enrichment, a read-only importer for its local traces, and an experiment task that drives a local agent.

Docs 2

# README with a quickstart, the agent matrix, OTLP setup, storage, privacy and uninstall.
# CONTRIBUTING, SECURITY and a release checklist in docs/RELEASE.md.

From CHANGELOG.md · RSS · All releases