---
title: "coding.sh v0.1.6 (page 3) · Changelog"
description: "6, page 3 of 4: every change to Judges, Compare, Cost, OpenTelemetry, Collector, Docs, Benchmarks, Sessions and Workflows, one line each."
canonical: https://coding.sh/changelog/v0-1-6/3
human_url: https://coding.sh/changelog/v0-1-6/3
markdown_url: https://coding.sh/changelog/v0-1-6/3.md
agents_url: https://coding.sh/agents.md
llms_url: https://coding.sh/llms.txt
updated: 2026-11-11
---

[Changelog](https://coding.sh/changelog)

Unreleased · 149 changes · page 3 of 4

# coding.sh v0.1.6

## Judges

- Added `judge grade --trace` reads `codex exec --json`, Cursor stream-json and OpenAI chat or Responses request bodies, and exits 2 with `trace_incomplete` when tool calls in the file did not reach the judge; `--allow-partial-trace` grades anyway with a warning, and `--json` carries `traceCoverage`. [Checklists and trace checks](https://github.com/jackulau/coding.sh/blob/main/docs/guides/evals.md#checklists-and-trace-checks)
- Added `"abstain": true` on an `llm-checklist` lets the model answer `cannot_decide` and name what is missing; the item leaves the score, a critical abstention fails the case as undecided, and `judge calibrate` counts undecided cases apart. Without it, prompts and version ids are unchanged. [Checklists and trace checks](https://github.com/jackulau/coding.sh/blob/main/docs/guides/evals.md#checklists-and-trace-checks)
- Added An `llm-panel` grader: one judge per provider scores 1-10 and the median decides, the pass target never reaches the prompt, anchor cases are re-scored fresh to report drift per member, and advisory members are shown but not counted. [Judge panels](https://github.com/jackulau/coding.sh/blob/main/docs/guides/evals.md#judge-panels)
- Added `pairedDiscrimination` from `coding.sh/judge` measures whether a judge reads the trace: each passing case gets children with the same output and a failure in the trace (skipped verification, failing final test, ignored tool error, destructive command, skipped test), and the report gives the flag-rate difference per kind with an interval. [Does the judge read the trace?](https://github.com/jackulau/coding.sh/blob/main/docs/guides/evals.md#does-the-judge-read-the-trace)
- Added `createWasteJudge` from `coding.sh/judge` asks a model which spans of a session were restated, padding, duplicate fetches or detours, drops fabricated quotes and misread streams, and prices the waste in output tokens. [Waste judge](https://github.com/jackulau/coding.sh/blob/main/docs/guides/evals.md#waste-judge)
- Fixed A `ran-command` check with `"ok": true` no longer passes a failed command on OpenAI chat and Responses traces. `judge grade --trace` reads the exit code a tool result states on a line of its own (`Process exited with code 1`, `exit code 0`), and a result with no status or exit code makes the item N/A, not a pass. [Checklists and trace checks](https://github.com/jackulau/coding.sh/blob/main/docs/guides/evals.md#checklists-and-trace-checks)
- Fixed `judge grade --trace` exits 2 with `trace_incomplete` for a file cut mid-line, a record with no `type`, `role` or `kind`, and a file with no JSON records; it used to grade them as whole traces. `--json` counts them in `traceCoverage.unparsedLines` and `unreadRecords`. An agent's plain-text stdout as the default trace is still no trace. [Checklists and trace checks](https://github.com/jackulau/coding.sh/blob/main/docs/guides/evals.md#checklists-and-trace-checks)

## Compare

- Added `coding.sh compare --parity <A> <B>` reads two pages (URLs, or commands that print one) and lists the fields that differ: status, redirects, title, description, canonical, robots, `og:` and `twitter:` tags, the `hreflang` count and JSON-LD types. It exits 0 on parity, 1 otherwise. [Page parity](https://github.com/jackulau/coding.sh/blob/main/docs/guides/evals.md#page-parity).
- Fixed `compare --parity` reads a tag whose attribute value holds a raw `>` in full, so two pages that differ after it no longer read as the same. [Page parity](https://github.com/jackulau/coding.sh/blob/main/docs/guides/evals.md#page-parity)

## Cost

- Added `coding.sh cache doctor` grades the prompt cache (hit rate, A to F) and ranks what broke the cached prefix by extra cost: model switches, tool set changes, a volatile line such as a timestamp in the system prompt, other system prompt changes, rewritten history, idle gaps. Each cause has its break count, sessions, tokens re-billed, the fix and a `diff-calls` example. [Prompt cache health](https://github.com/jackulau/coding.sh/blob/main/docs/guides/explore.md#prompt-cache-health).
- Added `coding.sh usage --baseline` and the MCP `usage` tool's `baseline` compare each agent's cache hit rate with Requesty's published rate for that agent (April 2026), as a signed point delta, `-` for an agent it does not list. Without the flag the output is unchanged. [Prompt cache health](https://github.com/jackulau/coding.sh/blob/main/docs/guides/explore.md#prompt-cache-health).
- Added CLIProxyAPI is a gateway: sessions whose base URL is on port 8317 (or `CLIPROXYAPI_URL`) are tagged `gateway:cliproxyapi`, and `gateway enrich --gateway cliproxyapi` reads its usage queue with the management key and joins each record to its llm call by session id, or by time and model. The call gets reasoning effort, service tier, speed, time to first token, compaction, tokens, upstream cost and a hashed account id. The read empties the queue for other readers, so it runs only when named, prints a warning, and keeps every record until it joins. [LLM gateways](https://github.com/jackulau/coding.sh/blob/main/docs/integrations/ai-gateway.md#cliproxyapi).
- Fixed A Claude Code subagent whose transcript sits under more than one session is stored once, under the session that wrote it. A workflow run resumed in another session (`resumeFromRunId`), a symlinked run dir or a fork used to store each copy as its own subagent session, so its calls and cost counted again. [Adapters](https://github.com/jackulau/coding.sh/blob/main/docs/ADAPTERS.md#subagents).
- Fixed Copies already in the index are removed on the next re-parse. A session an older coding.sh stored from a copy of another session's transcript (a Claude Code subagent copied into a resumed workflow run or a fork) is deleted with its events once the session that wrote it is stored, so its calls and cost stop counting twice. Re-parse used to keep it as a session whose transcript is gone; a session whose transcript is really gone is still kept. `ingest --rebuild` prints how many it removed. [Troubleshooting](https://github.com/jackulau/coding.sh/blob/main/docs/troubleshooting.md#old-sessions-show-doubled-counts).
- Fixed Claude Fable 5.2 and Mythos 5.2 are priced at their own rates ($8 input, $40 output, $0.20 cache hit, fast mode $16/$80) instead of Fable 5.1's $10/$50, which put standard calls 25% high and fast calls 37.5% low. Stored sessions are repriced on the next ingest. [Prices](https://github.com/jackulau/coding.sh/blob/main/docs/reference/config.md#prices).
- Fixed `usage` and `cache doctor` total every session in the window; they stopped at the newest 2,000 without saying so. Past 100,000 sessions (`CODING_SH_SCAN_CAP`) they print `Stopped after N sessions: narrow --since or --where.` and set `truncated` in `--json`. [Configuration](https://github.com/jackulau/coding.sh/blob/main/docs/reference/config.md#paths-and-stores)
- Fixed `coding.sh cache doctor` counts every session in the window in its breaks, re-billed tokens, extra USD and grade, so they match `usage` whatever `--limit` is. `--limit` (default 200) now only bounds how many sessions are read to classify causes: a session past it keeps its stored cause or counts as `unclassified`, and `--json` adds `breaksNotRead`. Before, the totals and the grade covered only the sessions that were read. [Explore](https://github.com/jackulau/coding.sh/blob/main/docs/guides/explore.md#prompt-cache-health)
- Fixed A `gateway enrich` record with no upstream cost no longer removes `(estimated)` from a session's cost. Only a record with a billed cost marks the call as billed. [LLM gateways](https://github.com/jackulau/coding.sh/blob/main/docs/integrations/ai-gateway.md)

## OpenTelemetry

- Added The receiver reads reasoning tokens (`gen_ai.usage.reasoning.output_tokens`, the AI SDK's `reasoningTokens`, Codex's `reasoning_token_count`, Gemini CLI's `thoughts_token_count`) into `payload.reasoningTokens` on the `llm.call`; skills (`gen_ai.skill.name`, Claude Code's `skill_activated`, Codex's `codex.skill_invocation`) as `payload.skill` and a `skill:<name>` tag; `gen_ai.conversation.compacted` and Claude Code's `compaction` event as `context.compact`; and `gen_ai.main_agent.name`, Claude Code's `agent_id`/`parent_agent_id` and Codex's `codex.agent_response` for the agent and its subagents. Spans without them map as before. [OpenTelemetry](https://coding.sh/docs/otel).
- Added An eve or durable-harness agent can send its Agent Runs spans to `coding.sh collector` as well as to Vercel, with one extra span processor; each run becomes a session with its turn, model calls, tool calls and usage, under the agent `durable-harness` for durable-harness. [Tee Agent Runs](https://github.com/jackulau/coding.sh/blob/main/docs/how-to/tee-agent-runs.md).
- Fixed A span that reports a cache read or cache write of 0 tokens no longer adds `cacheReadTokens: 0` or `cacheWriteTokens: 0` to the call's usage; non-zero counts and cost are unchanged. [OpenTelemetry](https://coding.sh/docs/otel).
- Fixed eve's `agent.step` and approval spans become `notification` events (`eve.step`, …), as the eve mapping documents, instead of generic `span` events. [eve](https://github.com/jackulau/coding.sh/blob/main/docs/integrations/eve.md).

## Collector

- Added `coding.sh collector import --clickhouse <url>` reads agent traces from an OpenTelemetry Collector's ClickHouse `otel_traces` table (Map or JSON attributes) into the local store, with the same mapping, redaction and pricing as spans posted to the collector. It keeps its place per table and row filter, so a second run imports nothing new; `--since`, `--from-start`, `--where`, `--table`, `--all`, `--dry-run` and `--json` shape it. [Pull from ClickHouse](https://github.com/jackulau/coding.sh/blob/main/docs/how-to/self-host-team.md#pull-from-clickhouse).
- Changed `coding.sh collector` stores concurrent requests together, in one transaction with one blob flush, and no longer keeps every source span as a blob (`--keep-raw` turns that back on; `serve` still keeps them). On the desk, 8 clients posting 61-span requests went from 1,437-3,075 to 5,585-6,865 spans/s, with request p95 from 283-613 ms to 125-155 ms. A long session per client is not faster. [OTLP collector](https://github.com/jackulau/coding.sh/blob/main/docs/perf.md#otlp-collector).
- Fixed `coding.sh collector import` keeps its place per table and row filter (`--all`, each `--where`), so a run without the filter reads the rows a filtered run skipped. A `--since` given on the command line reads from that time again, and the new `--from-start` does the same with the default 7 days, for a root span that reached the table more than an hour late. Nothing is stored twice. [Pull from ClickHouse](https://github.com/jackulau/coding.sh/blob/main/docs/how-to/self-host-team.md#pull-from-clickhouse)
- Fixed `coding.sh collector import` stores each page with its sessions before it saves its place, so a run that loses ClickHouse part-way leaves no events without a session. A dropped connection or a 502, 503 or 504 is tried three more times; a run that still fails says how many spans it read and prints the command that reads the rest. Before, the next run exited 0 with fewer sessions than traces. [Pull from ClickHouse](https://github.com/jackulau/coding.sh/blob/main/docs/how-to/self-host-team.md#pull-from-clickhouse)

## Docs

- Added A guide to whether a terse mode saves money on your tasks: run them with and without a be-concise rule, skill or plugin, read pass rate and cost with intervals, and split output tokens from the input the instruction adds. It cites our A/B of a terse-output plugin: output −8%, cost +11%, same pass rate and clarity. Its commands are checked against the CLI in tests. [Terse mode guide](https://github.com/jackulau/coding.sh/blob/main/docs/guides/terse-mode.md).
- Changed An adapter that nobody has run on a live install is now labelled "Not run live" in the README, the agents guide and the site. Only Cursor CLI and Amp are in that state; both need an account login. [Agents](https://coding.sh/docs/agents)
- Changed The instructions file is called AGENTS.md wherever any agent is meant: the home page, docs titles and prose, command examples, `--help` and the MCP tool descriptions. Lists of the files coding.sh reads start with AGENTS.md, and the site, the docs, `--help` and the hints `coding.sh improve` prints name no other instructions file: where a sentence is about one agent's own file, it says "the agent's instructions file". The five-minute guide moved to `/docs/how-to/test-agents-md-change`; the old URL redirects to it. No command changed which file it reads or writes. [Test an AGENTS.md change in five minutes](https://coding.sh/docs/how-to/test-agents-md-change).
- Changed The docs pages are shorter and easier to skim: each starts with what it is for and one real example, flags are in tables, and depth sits in "How it is computed" blocks or behind a link. A site test keeps bullets, paragraphs and guide pages within word budgets. [Experiments](https://coding.sh/docs/experiments).
- Fixed Status claims match the code and the registries again. Tool evals and workflows are tagged built, and `coding.sh`, `@codingdotsh/sdk` and the Postgres, ClickHouse and S3 store plugins are no longer called unpublished; the Python SDK is still not on PyPI. The dogfood plan gives each step's state with code pointers. A site test fails when a page calls something planned whose module exists, or when the agent list disagrees with the adapters in the code. [What is built](https://coding.sh/docs#status).
- Fixed The troubleshooting page lists each error reason under the exit code the CLI gives it, and the session rows example in the API reference is the real answer for the page's example session. [Errors by reason](https://github.com/jackulau/coding.sh/blob/main/docs/troubleshooting.md#errors-by-reason), [API](https://github.com/jackulau/coding.sh/blob/main/docs/reference/api.md)

## Benchmarks

- Added `bench show` and the report add a routing headroom check: the any-contender pass rate against the best single contender, compared with a Rasch model where no contender specializes, so per-case routing is only suggested when the gap beats run noise. [Routing headroom](https://github.com/jackulau/coding.sh/blob/main/docs/guides/benchmarks.md#routing-headroom)
- Added Contenders that cross models with harnesses (agents, or `axes = { model, harness }` on any contender) get a grid: interaction residuals with intervals, rank reversals, Cross-Model Fit per harness and leave-one-model-out regret. [Model × harness grid](https://github.com/jackulau/coding.sh/blob/main/docs/guides/benchmarks.md#model--harness-grid)
- Fixed Routing headroom now prints the best single contender's own pass rate, the one on the leaderboard, and measures the gap and its expected range from it. With several trials it used to average the best contender of each trial, so the rate beside the name was too high and the gap too small. Runs stored before the fix keep their old numbers. [Routing headroom](https://github.com/jackulau/coding.sh/blob/main/docs/guides/benchmarks.md#routing-headroom)
- Fixed The model × harness grid no longer states noise as fact. A rank reversal is listed only when the 95% interval of both pass-rate gaps excludes 0, each leave-one-model-out regret and the mean carry a 95% interval (`regretCi` and `meanRegretCi` in `--json`), no cell is starred under 20 paired cases, a bound that rounds to 0 no longer stars some cells of a 2 × 2 and not others, and the legend says the stars are per cell and not adjusted for the number of cells. [Model × harness grid](https://github.com/jackulau/coding.sh/blob/main/docs/guides/benchmarks.md#model--harness-grid)

## Sessions

- Added An agent started by another agent's shell tool (`codex exec` or `gemini -p` from Claude Code's Bash tool) is linked to that session as a child of type `harness`, and `coding.sh show` lists it under the parent. The link comes from the session id Claude Code and Codex export to the commands they run, else from the parent's shell call that ran the child's CLI while the child started in the same directory. [Concepts](https://github.com/jackulau/coding.sh/blob/main/docs/concepts.md).
- Changed `stats.toolErrors`, `repeated-tool-failure` and the `failure-streak` derail count only real failures. A search that found nothing (`grep`, `rg` exiting 1 with no output), a probe that answered no (`test -f`, `command -v`) and a `diff` that found differences are counted in the new `stats.benignToolErrors`, and a stopped call in `stats.interruptedToolCalls`. Stored sessions are counted again on the next ingest. [API](https://github.com/jackulau/coding.sh/blob/main/docs/reference/api.md#get-apisessionsid).
- Changed A shell command run from a REPL or code tool's cell (`bash("npm test")`, `subprocess.run(["pytest"])`, `os.system`, an IPython `!` line, a `%%bash` cell) counts as verification for `no-verification`, as the same command in a shell tool does. [Concepts](https://github.com/jackulau/coding.sh/blob/main/docs/concepts.md).
- Fixed `show`, `tree`, `usage`, `improve`, `harness audit` and the other reads answer within the read budget while a re-parse runs, from what is indexed. `show`, `tree`, `usage`, `replay` and `harness audit` used to ingest every pending capture first, with no budget. They leave the spool to the re-parse, say how many files wait (on stderr, or `pending` with `reparsing: true` in JSON), and a session not found says it may be among them. [Configuration](https://github.com/jackulau/coding.sh/blob/main/docs/reference/config.md).

## Workflows

- Added `coding.sh ui` shows workflow runs at `#/workflows`: one bar per agent grouped by phase, the critical path outlined, barrier idle shaded, a concurrency strip against the cap, links to each agent's session, and per-phase and metrics tables. On a phone it opens as a table. The page reads `/api/workflows`, the same documents `workflows show --json` prints. [Workflows guide](https://github.com/jackulau/coding.sh/blob/main/docs/guides/workflows.md#capture-and-measure)
- Added Verifier cases take named `[[truth.tests]]` and score the share that pass, reporting tests marked `before = "pass"` that fail after the run as regressions. Judge cases (`kind = "judge"`, a `rubric`) are graded by an LLM judge and flagged uncalibrated until `coding.sh judge calibrate` has calibrated that judge version. `workflows run` now scores every case kind as `workflows score` does. [Workflows guide](https://github.com/jackulau/coding.sh/blob/main/docs/guides/workflows.md#cases)
- Added `workflows run` checks each seeded bug on its case's `[start]` snapshot before any run: the bug's test fails seeded and passes with the patch reverted (`--no-validate` skips it). A case whose truth file sits inside its `[start]` tree no longer loads. [Workflows guide](https://github.com/jackulau/coding.sh/blob/main/docs/guides/workflows.md#cases)
- Added A command-driver variant with `fromCheckpoint` or `replayFrom` passes `--from-checkpoint <id>` or `--replay-from <node>` to the program, for LangGraph checkpoint forks and node replays. [Workflows guide](https://github.com/jackulau/coding.sh/blob/main/docs/guides/workflows.md#replays)

[Previous page v0.1.6, page 2](https://coding.sh/changelog/v0-1-6/2) [Next page v0.1.6, page 4](https://coding.sh/changelog/v0-1-6/4)

From [CHANGELOG.md](https://github.com/jackulau/coding.sh/blob/main/CHANGELOG.md) · [RSS](https://coding.sh/changelog.xml) · [All releases](https://coding.sh/changelog)

To set coding.sh up for a user, follow the agent workflow at [/agents.md](https://coding.sh/agents.md).