Unreleased · 146 changes · page 2 of 4

coding.sh v0.1.6

Refine 10

# Added coding.sh refine mine ranks failure patterns across sessions with no model call: session flags, normalized error lines (20 TS2307 errors in 5 sessions are one pattern), instructions you typed again and again, and failed eval runs, each with its count, sessions and example events, redacted. refine propose --mined FILE shows the top patterns to the proposer. Mining failure patterns.
# Added coding.sh refine diagnose asks a model what went wrong in recent sessions and groups the answers into Signals with stable ids; refine signals lists them. Every problem needs a citation that code checks against the redacted trace, severity is computed by code, replies are cached so a rerun makes no call, and with no API key it prints a $0 plan of the sessions and tokens it would send. Diagnosing sessions into Signals.
# Added refine test|run --suite --canary-model <model> re-tests a kept edit that deletes or rewords lines (--canary all: every kept edit) under a cheaper model and rejects it as canary-regression when the pass rate falls there; refine recheck <r_…> --model runs the same check later. A subtractive edit kept without a canary gets a note. Wrapper agents take a {model} slot in their flags. Canary re-test.
# Added coding.sh refine distill writes an experiment's runs (or recent sessions) as a redacted corpus, leaving out held-out tasks, and has Claude Code or Codex analyze it with its own code and write one skill in an isolated directory. The skill is size-capped, checked by the critics and ledgered as a proposed candidate with its provenance for refine test. Distilling a skill.
# Added The proposal prompt lists, per earlier edit that was not kept, the tasks it fixed, regressed and left failing, read from the ledger, and asks for lines aimed at the still-failing tasks. With no such rounds the prompt is unchanged. Refine loop.
# Fixed refine signals show <id> takes the Signal id or a prefix of it again; it rejected every id with Unexpected argument. An unknown id says No Signal <id> in <file> and exits 2. Diagnosing sessions into Signals
# Fixed refine log, the refine test verdict line and the refine apply refusal lead with the canary's reason for a round the canary rejected or left inconclusive, not the reason the candidate was first kept. Canary re-test
# Fixed refine distill --keep-corpus DIR no longer deletes a directory that holds a file named SCHEMA.md next to other work. Only a directory that holds nothing but an earlier corpus is replaced, entry by entry; any other non-empty directory is refused and left as it is. With --dry-run nothing is written to DIR. Distilling a skill
# Fixed refine diagnose --session takes an id prefix or latest, as show does. An id that matches no session is a not-found error (exit 1) instead of an empty plan. Diagnosing sessions into Signals
# Fixed refine mine reports failed eval runs with the default --repo: a run counts for the repo when its task starts from it or belongs to a suite under its evals/. The summary line and --json (evalRuns) say how many eval runs were read. Mining failure patterns

Export 2

# Added coding.sh export --evidence <dir> writes each matching session as a folder of files: index.md with stats and a timeline, session.json, events.jsonl, and one file per tool call with its input and full output. Everything is redacted first, files are size-capped, and the same store gives the same bytes. Query and export.
# Added eval export --format atif writes the task's source session as an ATIF-v1.8 trajectory (redacted, with subagents embedded). --format harbor now writes task.toml schema 1.4 with an explicit verifier mode, and the trajectory under solution/. Both pass Harbor 0.25's loaders. Session to eval.

Capture 14

# Added coding.sh doctor --conformance runs each agent's backfill over the transcripts on this machine and checks the events: duplicate ids, timestamps, tool and subagent pairing, parent links, double-counted usage, sessions without a prompt and failed commands marked ok. It only reads, never opens the store, and exits 1 on a violation. Troubleshooting.
# Added A tool call that loads a skill names it on its tool.start (payload.skill, and payload.skills when it loaded several): a Skill tool call, a read of a file under an agent's skills/<name>/ folder, a cat, head or sed of one by a literal path, or an MCP read of a skill:// resource. Every adapter goes through the same check. Adapters.
# Added Copilot CLI hook events keep the OpenTelemetry trace from the traceparent Copilot 1.0.107 passes its hooks, and ingest records which session the trace belongs to, so an OTLP span of that trace that names no conversation lands in the hook session. Adapters.
# Added coding.sh run -- aider records each reply's tokens and cost while Aider runs: the wrapper reads Aider's analytics log (a temp AIDER_ANALYTICS_LOG, or the log you named) and the cache split from its chat history, so live Aider sessions have usage and cost, not only eval runs. Adapters
# Changed Outside Windows the hook agents run is a POSIX sh script, ~/.coding.sh/bin/hook.sh, which writes the same spool line without starting node: under bash 5, about 3 ms more than /bin/true per hook call for payloads up to 256 KB, instead of about 30 ms (Hook). It hands --reply, eval runs, gateway base URLs and anything else it does not cover to hook.mjs. Run coding.sh init again to switch existing hooks. Adapters.
# Fixed A Claude Code response with parallel tool calls is one llm.call again, with its usage counted once. Claude Code writes a tool's result between the lines of the response that called it, and each part used to become its own call; the call is now timed from the request's api-request line. A subagent that another subagent spawned hangs off that subagent's session instead of the main one. Stored sessions are parsed again on the next ingest. Adapters.
# Fixed A Codex subagent thread has a prompt and a title: the task its parent sent it (an agent_message addressed to the thread) is its user.message, with from, taskName and encrypted. Codex encrypts the task text, so the prompt says which agent asked for which task. Adapters.
# Fixed OpenCode history imports when OpenCode has set its database aside as opencode.db.schema-mismatch-<ms> and not yet written a new one, and doctor says when a database has no session tables coding.sh knows or cannot be opened, where backfill used to import nothing without a word. Adapters.
# Fixed An Antigravity command that exited nonzero is a failed tool call with its exitCode, read from the result's The command exited with code N. line, and the result's Created At/Completed At time it. Every Antigravity tool call used to count as ok. Adapters.
# Fixed The sh hook no longer hangs when an agent starts it with stdin closed. Under macOS /bin/sh and bash 3.2 it never exited and left a child process behind. It now treats a closed stdin as an empty payload and exits 0 at once, as hook.mjs does. Adapters
# Fixed hook.sh no longer fails the agent's hook call when it hands an event to hook.mjs and node cannot run (not on the agent's PATH, a version manager's shim that fails, a missing hook.mjs): it exits 0 and still prints the --echo-continue reply. Before, an eval run, a gateway base URL or --reply made the hook exit 127 on such a machine. That event is not captured. Adapters
# Fixed Claude Code hooks registered in two settings files with different commands (user settings on hook.sh, a project's .claude/settings.local.json still on node …/hook.mjs) no longer store each prompt, turn end and session end twice. The spool lines of one hook firing (the same payload, at most 2 s apart) get the same event ids; the same prompt sent again later is still a second event. coding.sh init also looks for project hooks at the root of the repository it runs in, reports them and skips the user install unless --force. Stored Claude Code sessions are parsed again. Adapters
# Fixed A session whose hooks ran with a stale CLAUDE_CODE_SESSION_ID or CODEX_THREAD_ID is no longer hidden as a subagent of a session that is not in the store or had already ended. The marker links a child only when the named session is in the store and was open when the child started; stored sessions with such a parent become top-level after the refold. Concepts
# Fixed When the ingest the hook starts fails, coding.sh status, the status line and coding.sh doctor print ingest stopped 4h ago with the command that fixes it, and status --json and doctor --json carry ingestStopped. status prints the home, the spool and that line even when the store can't be opened. Troubleshooting

Explore 1

# Added coding.sh stats and the viewer's new Explore page (#/explore) show sessions, failure rate, cost, tokens, tool error rate and p95 model-call time per day or week, split by agent, model, repo or flag, from one engine that GET /api/metrics also serves. Explore sessions, API.

Viewer 3

# Added A model call in coding.sh ui opens in a playground: the call is rebuilt as system prompt, messages and tools, you edit it, run it against Anthropic, OpenAI or any OpenAI-compatible server, see the answer beside the recorded one, and save it as a dataset case. Explore sessions, API.
# Added The viewer page arrives with the first sessions page drawn and its first responses embedded, so the list shows before the script runs (on Slow 4G with a 4x slower CPU, the first row shows about 2.7 s sooner, 95% CI 2.71 to 2.75 s), and a session the pointer rests on, the keyboard moves to, or a press lands on loads in the background, so it opens without a request. Explore sessions.
# Fixed "Open in playground" works for model calls in subagent and workflow-agent sessions: the request is rebuilt from the subagent's own conversation, without the calls of a subagent it started. Every such call used to answer 404 no_request. API

API 1

# Added GET /api/sessions/{id}/rows pages through one turn of a long session, with runs of routine events folded and failures always shown; the outline adds turns, failures, compactions, subagents and a minimap; GET /api/events/{id} returns one event; and GET /api/blobs/{sha256}?range=a-b reads part of a blob. API.

Harness A/B 5

# Added compare grades each run in a fresh copy of the task's start tree plus the agent's source changes. A conftest.py, sitecustomize.py, .pth or .npmrc the agent wrote, an edited test config or Makefile, and a changed test script in package.json no longer reach the verifier, and each run records what was left out in isolation. --no-isolation grades the agent's own tree. Verifier isolation
# Added The compare report adds cost and tokens per solved task, pass@k, pass^k and the flaky tasks with --k 2 or more, and how many paired tasks would detect the observed effect at 80% power. Efficiency, reliability and tasks needed
# Added compare --otel gives each agent an OTLP endpoint, a TRACEPARENT and coding_sh.run_id in its resource attributes, so the spans it exports are stored with the run and join its session. OpenTelemetry per run
# Added compare --sequential runs each arm's tasks in order and carries the agent's home (or --carry <path>) from one task to the next, and the report adds the pass rate by position. Sequential runs
# Added compare --pick N and --budget <USD> run the tasks that best tell harnesses apart in stored runs first and tasks every run passed last. --budget fits the tasks to an estimated cost, caps the spend and shows the estimate in the consent prompt; --dry-run lists the picks and runs no agent. Picking tasks and a budget

Attribution 4

# Added coding.sh attribution matches the lines each commit added to the captured session that wrote them, from the session's snapshots or its edit tool calls; a rebased commit still matches and lines a person wrote match no session. coding.sh blame <path>[:line] shows it per line, and coding.sh attribution export writes Agent Trace (agent-trace.dev, RFC 0.1.0) records. git-ai notes are read as a second source. Read-only on git. Attribute commits.
# Fixed Without snapshots (a store filled by ingest --backfill, or a session whose end snapshot is gone), a session is credited only for the lines its own Write, Edit and patch calls added, less the ones it later removed. Lines a person added to the file between two of the agent's edits used to be credited to the session once the agent read the file again. Sources cached by an older build are built again. Attribute commits
# Fixed A person's copy of a long line in another file of the same commit no longer takes the session's credit. Every file of a commit gets its same-file matches before a long line may match across files, so blame credits the agent's own line and shows - for the copy. Attribute commits
# Fixed A commit made on a branch that was no longer checked out when the session's end snapshot was taken got 0 lines, because an empty or unrelated snapshot diff hid the session's own edit tool calls. coding.sh attribution and coding.sh blame now take every file the session's tools named that the snapshot diff does not hold from those calls, at confidence 0.8. Attribute commits

Judges 7

# Added judge grade --trace reads codex exec --json, Cursor stream-json and OpenAI chat or Responses request bodies, and exits 2 with trace_incomplete when tool calls in the file did not reach the judge; --allow-partial-trace grades anyway with a warning, and --json carries traceCoverage. Checklists and trace checks
# Added "abstain": true on an llm-checklist lets the model answer cannot_decide and name what is missing; the item leaves the score, a critical abstention fails the case as undecided, and judge calibrate counts undecided cases apart. Without it, prompts and version ids are unchanged. Checklists and trace checks
# Added An llm-panel grader: one judge per provider scores 1-10 and the median decides, the pass target never reaches the prompt, anchor cases are re-scored fresh to report drift per member, and advisory members are shown but not counted. Judge panels
# Added pairedDiscrimination from coding.sh/judge measures whether a judge reads the trace: each passing case gets children with the same output and a failure in the trace (skipped verification, failing final test, ignored tool error, destructive command, skipped test), and the report gives the flag-rate difference per kind with an interval. Does the judge read the trace?
# Added createWasteJudge from coding.sh/judge asks a model which spans of a session were restated, padding, duplicate fetches or detours, drops fabricated quotes and misread streams, and prices the waste in output tokens. Waste judge
# Fixed A ran-command check with "ok": true no longer passes a failed command on OpenAI chat and Responses traces. judge grade --trace reads the exit code a tool result states on a line of its own (Process exited with code 1, exit code 0), and a result with no status or exit code makes the item N/A, not a pass. Checklists and trace checks
# Fixed judge grade --trace exits 2 with trace_incomplete for a file cut mid-line, a record with no type, role or kind, and a file with no JSON records; it used to grade them as whole traces. --json counts them in traceCoverage.unparsedLines and unreadRecords. An agent's plain-text stdout as the default trace is still no trace. Checklists and trace checks

Compare 2

# Added coding.sh compare --parity <A> <B> reads two pages (URLs, or commands that print one) and lists the fields that differ: status, redirects, title, description, canonical, robots, og: and twitter: tags, the hreflang count and JSON-LD types. It exits 0 on parity, 1 otherwise. Page parity.
# Fixed compare --parity reads a tag whose attribute value holds a raw > in full, so two pages that differ after it no longer read as the same. Page parity

From CHANGELOG.md · RSS · All releases