GitHub
Changelog

Nov 8, 2026 · 177 changes · page 2 of 5

coding.sh v0.1.2

coding.sh runs on Windows, and CI tests it there.

Capture 12

  • # Added coding.sh init imports the sessions each agent already keeps on this machine, so the viewer the installer opens shows your own history. It shows progress, stops after 8 seconds and imports the rest in the background; the viewer says "Importing past sessions…" until it is done. --no-import skips it. An empty viewer shows the import command, coding.sh ingest --backfill, and what each imported session shows. What init changes.
  • # Added coding.sh proxy is a local pass-through for the Anthropic and OpenAI APIs. Each model call is logged with its redacted request (system prompt, tool definitions, messages) and response, usage with cache reads and writes, status, time to first byte and token, and rate limits, inside the Claude Code or Codex session its hooks capture. Auth headers are forwarded and never stored. It adds 0.34 ms per streamed chunk and 1.96 ms to first byte (CI fails over 2 ms and 5 ms). Capture with the proxy.
  • # Added Proxy calls record reasoning effort (including Anthropic's output_config.effort), service tier and speed from the request when the response has none, and compaction: true on calls that compacted the conversation, including Codex's /responses/compact, which is now logged. What is stored
  • # Added The proxy stores a call's messages in content-addressed blocks shared with the session's earlier calls, instead of the whole conversation again on every call, and reads the body back exactly. A synthetic 500-call session took 5× fewer bytes (828 KiB instead of 4,116 KiB); reading a call back takes about 10 ms. Works on SQLite, Postgres and ClickHouse stores; calls stored whole still read, and gc and purge follow the blocks. What is stored
  • # Added Proxy calls from a Claude Code subagent land in the subagent's own session instead of the parent's. Calls record the parent session, Claude Code's agent id and Codex's parent thread, fork and subagent kind when the client sends them, the account as a 16-character hash (the account id is not stored, in the event or the request body), and behind CLIProxyAPI the credential index it used. A session seen only through the proxy links to its parent. What is stored
  • # Added CI runs real OpenCode, Pi, Goose, Factory Droid, GitHub Copilot CLI and Aider sessions on Linux and Windows against a stub model, checks the recorded prompt, tool call, file change, model and reply, and checks that ingest --backfill adds no duplicates. Install guide.
  • # Added Sessions stored before an adapter's parsing changed are re-parsed from their transcripts and spool in the background, so this release's dedupe fixes reach data already captured. coding.sh ingest --rebuild does it now. Troubleshooting.
  • # Added Claude Code Bash commands that edit files (sed -i, a heredoc write, tee) record a file.change per file with its diff, taken from Claude Code's own edit diff, even when the command then failed. A nested instructions file or skills folder loaded mid-session, a Skill call's skill folder and the skills restored after /compact become harness.loaded events and show in show and the viewer timeline. Stored Claude Code sessions are re-parsed to pick them up. CI checks both on a real Claude Code 2.1.319 session. Adapters.
  • # Fixed Backfill follows symlinked and junctioned transcript directories and files, and skips dangling links. Before, a ~/.claude/projects made of links ingested nothing. Adapters.
  • # Fixed Claude Code subagents that fire no SubagentStart or SubagentStop (web-fetch) are linked to their parent from the parent's Agent tool result, with their prompt, result, model and end, instead of showing as a top-level session with no prompt. Adapters.
  • # Fixed The background re-parse after an upgrade no longer blocks reads: it holds the ingest lock at most a second at a time and ingests new hook lines between holds, so sessions and the viewer answer in about a second instead of waiting 8 to 24 seconds. It writes blobs in batches, resumes where a cut-short run stopped, and works out stats and flags again for every stored session when an upgrade changes them. Listing a large spool is faster too. Troubleshooting.
  • # Fixed A full ingest of a 25k-line spool takes 4 s instead of 294 s on macOS, and 39 s instead of 153 s on Linux: blobs are made durable once per checkpoint instead of with two fsyncs per line, and each checkpoint commits its events, session rows and spool cursors in one transaction, so a crash mid-drain neither loses nor duplicates events.

Examples 4

  • # Added examples/plugin-ab A/B tests a Claude Code plugin against plain Claude Code on ten small tasks in a container, with coding.sh compare: pass rate, tokens, cost split by token type, wall time, reply length and a blind clarity judge, paired by task with bootstrap intervals. It takes any plugin directory and ships a small demo plugin. The example.
  • # Added The skill eval tests an edit to the skill: run.sh <out-dir> --candidate <skill-dir> runs the pinned skill as arm A and the edited copy as arm B, graded the same way, with the --promote line in the report. TASKS="01 06" runs some of the tasks. Skill eval example.
  • # Fixed The skill eval's live site check counts a network command that uses $LIVE_SITE as touching the site; it missed those, and passed runs that had probed the site. Scenario 06 no longer tells the agent to record open questions instead of waiting, since its pass is to stop and ask. Skill eval example.
  • # Fixed The skill eval's live mode grades fairly: --help and help calls never count as writes, a refused env ls reads "refused secret read", live task copies name the team, project and site only through $VERCEL_SCOPE, $VERCEL_PROJECT and $LIVE_SITE, the site checks match the team's real domains (kept from the agent), and out/checks records each run's task and arm. run.sh <out-dir> --regrade grades stored runs again without calling the agent. Evaluate an Agent Skill.

Improve 3

  • # Added coding.sh improve lists what goes wrong most in this repo's sessions (failure flags, and any command failing the same way in two or more sessions), each with an example session, the AGENTS.md line that fixes it and the next commands. improve fix N adds the line, uncommitted. improve check compares the sessions before and after each committed AGENTS.md, rule or skill change, with how often the agent follows each new line, says Helped, Regressed, No clear change or Too early, and exits 1 on a regression; --watch re-checks as sessions arrive. No API key needed. Improve your agent every week.
  • # Added improve check lists, under each change, the sessions still flagged after it with their flags and start time (five, or all with --sessions; every flagged session before and after in --json), and the MCP harness_changes tool returns them as after.flaggedSessions, so your agent can answer "which sessions still failed after it?". Did it help?
  • # Fixed improve check --repo <DIR|all> counts the sessions that ran in other checkouts and worktrees of the repo. Before, it ignored --repo and counted only sessions whose repo root was the current checkout, so a repo whose agents work in worktrees showed every change as "No sessions yet". Improve.

MCP 4

  • # Added improve and harness_changes tools, and improve and did-it-help prompts, run the same loop from inside your agent: what to fix first, and whether an AGENTS.md change helped, without a shell. Improve your agent every week.
  • # Added An eval_init tool lets your agent build an eval suite for you, with the same sources as eval init; it writes files only with confirm: true. MCP server.
  • # Added exp_session_note adds a redacted note to the project's experiment session (the hypothesis before a run, the reading after it) and exp_session_summary returns its timeline, paged, with the same blocks as exp session summary --json, so an agent can pick an investigation back up after a context reset. Without a session both return a not_found error naming coding.sh exp session new --name <topic>. Explore.
  • # Fixed Tool calls no longer wait for a big ingest: they answer within 1.5 s from what is indexed, with pending: {files, indexing} in the result. CLI reference.

Harness 2

  • # Added coding.sh harness audit lists each memory and instruction file (Claude auto-memory, AGENTS.md and the other instructions files, rules, skills) with the sessions that loaded it, read it and wrote it, its size and its load cost in tokens, and flags files never read, written but never read back, and skills listed but never used. --agent, --since (default 30d) and --json. Memory and instruction files.
  • # Fixed harness audit skips a file or directory it can't stat (a symlink loop, or a Windows junction Node can't follow) instead of failing with "unknown error". Memory and instruction files.

Judges 11

  • # Added LLM graders take "provider": "claude-code", which grades through claude -p on a Claude subscription (CLAUDE_CODE_OAUTH_TOKEN from claude setup-token) instead of an API key. The judge runs with no tools, setting sources, plugins, MCP servers or saved session, so the hooks of the arm under test never reach it. Evaluate an Agent Skill.
  • # Added Judge prompts escape data tags in the judged text and say it is untrusted, so an output can't close </output> and order a pass. judge calibrate --probe-injection judge.json plants a "respond pass" order in human-fail cases and reports how many verdicts flip; any flip exits 1. Default-prompt judges get new version ids. Evals.
  • # Added Checklist items can grade rule compliance from the trace with no model call: ran-command, read-path, wrote-identifier, and follows-rules, which checks every imperative line that names a backticked command, flag, path or identifier (`Never use git push --force `) against what the agent did. Evals guide.
  • # Added judge calibrate --grader tests audits a test or command verifier: --sample-passes 40 lists unlabelled passing cases to label, and the audit reports precision (how many passes a person would accept) with a Wilson interval and the false accepts. exp show and exp compare then print the precision and a pass rate corrected for it. Evals guide.
  • # Added follows-rules reads rule-file frontmatter: a Claude Code rule with paths:, a Cursor rule with globs: (or neither glob nor alwaysApply), or a Copilot rule with applyTo: is graded only after the agent touches a matching file or opens the rule. Before that the agent never saw it, so it no longer counts as skipped. Evals guide
  • # Added judge calibrate prints the judge's discrimination gap and label efficiency η, and warns when auditing outputs directly would need fewer human labels: label efficiency η 0.21 · auditing outputs directly needs ~4.8x fewer human labels for the same precision. --judge-cost 0.001 adds the human label cost above which the judge pays off. The calibration state is unchanged; --json carries calibration.efficiency (Choi et al., arXiv 2610.35698). Evals.
  • # Added exp compare --pairwise and judge graders in exp compare print how often a resample of the cases picks the same winner, and n*, the smallest size where that holds 95% of the time: stable B better in 97% of resamples (n*=50) or unstable B better in 61% of resamples (n* censored). An uncalibrated judge that never settles warns "pick or calibrate the judge before collecting more verdicts" (arXiv 2611.04090). --json carries stability. Evals.
  • # Added judge sample <exp> --budget 60 --out plan.json says which cases to label next: a seeded random sample stratified by the judge's verdict and confidence, with labels in proportion to stratum size, so the labels don't favour the doubtful cases. judge calibrate --plan plan.json then prints the human-audited pass rate: human-audited pass rate 0.71 [0.63, 0.79] (42/50 of plan labelled; worst case ±0.09) (Choi et al., arXiv 2610.35698). Evals guide.
  • # Added New no-test-tamper checklist check, with no model call: it answers no when a test change hid a failure (a skip marker, a test deleted while red, an assertion edit that alone turned the tests green), cites the change and the test runs around it, and quotes what the agent said: weaken test/sum.test.js by Edit in e42 (turned failing tests green), disclosed: "…". Saying why never makes it pass. Evals guide.
  • # Added LLM graders take "provider": "codex", which grades through codex exec on a ChatGPT subscription. Evals.
  • # Fixed The Anthropic and OpenAI judge providers read ANTHROPIC_BASE_URL and OPENAI_BASE_URL, like the vendors' SDKs, so judges and drafts reach a proxy, gateway or local stub without --base-url. Judges.

Stores 1

  • # Added The Postgres and ClickHouse store plugins answer --where in the database, with every value bound: session and event keys, has:, is:, bare words, sort: and --facets, for sessions, usage, export, the viewer API and MCP. sessions --where and facets no longer stop at the newest 20,000 sessions there, and event keys and bare words no longer fail with unsupported_store. Query language.

From CHANGELOG.md · RSS · All releases