--- title: "coding.sh refine · coding.sh docs" description: "Improve AGENTS.md and skills in tested rounds, and protect their safety rules. Every option of coding.sh refine and its subcommands, from its own --help." canonical: https://coding.sh/docs/cli/refine human_url: https://coding.sh/docs/cli/refine markdown_url: https://coding.sh/docs/cli/refine.md agents_url: https://coding.sh/agents.md llms_url: https://coding.sh/llms.txt updated: 2026-11-11 --- # coding.sh refine Improve AGENTS.md and skills in tested rounds, and protect their safety rules. Every option of coding.sh refine and its subcommands, from its own --help. coding.sh refine --help ``` coding.sh refine Improve AGENTS.md and skills in tested rounds, and protect their safety rules Commands propose Ask a model for lines that address your sessions' most severe flags; stores them as a candidate test Test candidate edits against your current files; keep, reject or inconclusive per candidate, exit 0 when one is kept apply Write a kept candidate to your files, backing up what it replaces (never commits) rollback Put back what an apply replaced, from its backup log Every round in this repo: verdict, effect CI, guard result, cost and reason, and whether the loop plateaued; --suite climbs show pass rate, cost, time and size per round and the Pareto front run Loop: propose (or take --candidate), test, apply when kept; stop at --rounds, --max-cost or a plateau protected List the harness lines an edit may not delete, reword or relax check Check a harness edit for deleted, reworded or relaxed protected lines; exit 1 when it has any mine Rank failure patterns across sessions: repeated error lines, repeated instructions you typed, flags and failed eval runs, with counts and example sessions ($0) recheck Re-run a suite round's base and candidate under another model (a canary) and record the verdict; suggests a rollback when the edit regresses there distill Hand a coding agent the raw runs of an experiment (or this repo's recent sessions) to analyze with its own code and write one skill; the skill is checked by the critics and ledgered as a candidate for refine test diagnose Diagnose recent sessions with a model into Signals: cited problems grouped into issues, severity scored by code. Without an API key it prints a $0 plan signals List the Signals refine diagnose wrote, or show one with its cited evidence and fix score Score diagnoses against human labels: occurrence, Signal and session precision and recall, coverage, repeat agreement Notes A protected line is inside a … block, matches a [refine] protect regex, or pairs a gate word (never, don't, must not, only, before, ask, confirm, verify, check, require) with a safety topic (credentials, secrets, access tokens, private keys, .env, prod, deploy, push, --force, force-push, destructive, --no-verify, rm -rf, delete, drop table, reset --hard, migrations, permissions, sudo, chmod, payments, PII, another user, owner, dry-run, backups). Set [refine] autoProtect = false in your user config to keep only markers and patterns. check rejects an edit that deletes or rewords a protected line (protected-line), or adds a line that waives one (skip, no need, only … when, unless, without asking, already verified …) and shares a backticked token, a word or a safety topic with it (relaxes-protected). exp ab runs the same check on its arms and warns; with --gate a protected-line problem fails the run. test decides each candidate in stages: the protected-line check, the harmless and duplicate critics on the added lines, an A/B against your current files on the decision points exp ab mines (the acceptance gate: ship goes on, hold is inconclusive, don't ship rejects), then the guard set: the decision points earlier kept edits won must not go from solved to unsolved. A run stopped at --max-cost is inconclusive. The ledger and every tested candidate's files live under the data dir (refine/-/). Rejected candidates and their reasons go into the next proposal prompt; protected lines go in as READ-ONLY. With --suite, each round runs the suite's repo tasks --trials times (default: the suite's trials, else 3) with the current files and with the candidate, interleaved, as one durable job; --model is the agent's model, --concurrency the parallel sessions, --max-cost caps agent and proposer spend. The candidate is kept when the pass rate is significantly better (one-sided paired permutation test over tasks, or the acceptance gate says ship), or when it holds within --margin (one-sided bootstrap lower bound) and a listed objective is significantly better by --min-gain (per-task log ratios, one-sided permutation test); a hard regression or a don't-ship gate rejects it, and so does a listed objective significantly worse, unless the pass rate is significantly better and the rise is within --max-cost-rise (default 50%; the verdict prints the trade). Below 5 tasks no test can reach significance. With --suite and a primary objective other than pass, the proposer cuts instead of adding: it sees the target file, its largest sections and what made the last runs slow or expensive (tokens, turns, repeated tool calls, long outputs), and replies with edits: find and replace an exact string, or replace a whole section by its heading. The protected-line check and the critics still run on the result. With --suite and --canary-model, a kept candidate that deletes or rewords existing lines (every kept candidate with --canary all) runs again with both arms under the canary model. It is rejected (canary-regression) when a task the base passed in every attempt passes in at most a third of the candidate's, when more of its passing runs tamper with tests, or when the paired bootstrap lower bound of the pass-rate change is at or below -10pp (--canary-delta). With fewer than 3 tasks differing on each side the decision stands, marked canary-uninformative. A subtractive candidate kept without a canary model gets a note that a success-only gate cannot see a lost safeguard the loop's model did not need. recheck runs the same check later for any suite round. distill writes the corpus as episodes.jsonl (one row per run or session: pass, verifier exit, tamper, turns, tool calls and errors, tokens, cost, flags), tool_calls.jsonl (one row per call with 500-character previews and an input hash), a redacted transcript per episode and a copy of the current harness files under prompt/. Cancelled, superseded and infrastructure-error runs, and runs of tasks tagged split:dev, split:test, split:heldout or refine:heldout, are left out. The distiller works in a temp directory and must write out/SKILL.md with name and description frontmatter, at most 160 lines and 10 KB; the harmless and duplicate critics then check it. Each skill is ledgered (op propose) as proposed or rejected; none is applied. diagnose sends each session (cut to the first prompt, the last 40 events and the events around failures when longer than 120) inside an untrusted block with the session's flags as rule candidates. An occurrence survives only when a quote it cites is in the cited event; ins.* types, tool.invalid-args, out.false-success and out.check-subverted also need the broken requirement cited. A second call groups new occurrences into Signals (existing occurrences never move) and a third writes one fix per changed Signal. Severity is computed from the cited factors and how many analyzed sessions show the issue. Replies are cached by model, prompt and catalog, so a rerun on unchanged sessions makes no call and writes the same signals.json. A Signal with no new occurrence in the next 20 analyzed sessions of the same agent and repo is quiet, which is not evidence a fix worked. score matches by code, never a model: a prediction matches a label in the same session with the same component (type, and name compared after Unicode, case, camelCase and separator normalization) whose top-3 type ids share one id, under a maximum one-to-one matching. Unmatched items count against precision or recall; a rate with nothing to divide by prints —. Without --pred, each diagnosed session flag is a prediction with no component, so it matches on session and type ids alone, and per-flag precision is reported. score writes nothing. ``` ## `coding.sh refine propose` Ask a model for lines that address your sessions' most severe flags; stores them as a candidate. coding.sh refine propose --help ``` coding.sh refine propose [options] Ask a model for lines that address your sessions' most severe flags; stores them as a candidate Options --since Sessions started since (default: 14d) --agent Only sessions of this agent (repeatable, or a comma list) --repo Only sessions under this directory (default: this git repo) --provider Who proposes, answers the next turn and judges it: an API, or claude-code / codex to run `claude -p` / `codex exec` on your subscription (default: anthropic; with no API key set, a logged-in Claude Code, else Codex) --base-url API base URL (default: ANTHROPIC_BASE_URL / OPENAI_BASE_URL, else the provider's) --proposer-model Model that writes the edit (default: --judge-model, else --model, else claude-sonnet-5-5 or gpt-6-sol) --proposer-command Shell command that writes the edit instead of the proposer model: the prompt on stdin, the JSON reply on stdout (e.g. 'claude -p') --target Harness file the proposal edits, or a skill directory for its SKILL.md (default: AGENTS.md, else the repo's Claude Code or Gemini instructions file) --mined Also show the proposer the patterns in this file (the output of refine mine --json) --dry-run Print the findings and the prompt; make no model call Examples coding.sh refine propose coding.sh refine propose --target AGENTS.md --dry-run ``` ## `coding.sh refine test` Test candidate edits against your current files; keep, reject or inconclusive per candidate, exit 0 when one is kept. coding.sh refine test --help ``` coding.sh refine test ... [options] Test candidate edits against your current files; keep, reject or inconclusive per candidate, exit 0 when one is kept Arguments candidate Edited file, directory, git:REV, harness bundle or stored r_… id; a file replaces the harness file at its repo path, else the one with its name (./drafts/AGENTS.md replaces AGENTS.md) Options --behavior Decision points to test: unverified-claim, done-while-failing, unchanged-retry (default: all) --anchor Module exporting your own anchor(s) (see the evals guide) --since Sessions started since (default: 14d) --agent Only sessions of this agent (repeatable, or a comma list) --repo Only sessions under this directory (default: this git repo) --provider Who proposes, answers the next turn and judges it: an API, or claude-code / codex to run `claude -p` / `codex exec` on your subscription (default: anthropic; with no API key set, a logged-in Claude Code, else Codex) --base-url API base URL (default: ANTHROPIC_BASE_URL / OPENAI_BASE_URL, else the provider's) --judge-model Model for the judge (default: --model) -n, --limit Maximum decision points (default: 40) --model Model for the next turn (default: the one the decision points were recorded with, else claude-sonnet-5-5 or gpt-6-sol) --trials Next turns per case and arm (default: 1) -c, --concurrency Parallel model calls (default: 4) --max-cost Refuse to start when the estimate is over this, and stop once the spend passes it --max-guard Most tasks in the guard set (the tasks earlier kept edits won), newest kept edits first (default: 24) --cache Reuse stored model responses (--no-cache to ask the model again) (default: true) --lambda Acceptance gate: what breaking one solved decision point costs, in fixed ones (default: 2) --gate-alpha Acceptance gate: one-sided significance level (default: 0.05) --tau Acceptance gate: largest share of solved decision points B may break (default: 0.10) --allow-protected-edit Allow the edit to delete, reword or relax this protected line (repeatable; never implied by --force) --suite Judge candidates on this eval suite of repo tasks (evals/) instead of mined decision points; a skill --target uses evals/ when it exists --objective With --suite: what to improve, primary first: pass, cost, tokens, wall, llm, turns, size (default: pass). The pass rate must hold within --margin whatever the list --margin With --suite: how far the pass rate may drop for a cheaper or faster candidate to be kept, e.g. 5pp (default: 2pp) --min-gain With --suite: smallest relative gain in cost, tokens, time, turns or size that counts, e.g. 20% (default: 10%) --max-cost-rise With --suite: how far a listed cost, tokens, time, turns or size objective may rise for a significant pass-rate gain, e.g. 100%, or none (default: refine.maxCostRise, else 50%) (default: 50%) --harness With --suite: stored harness bundle the suite runs, the repo's harness files laid over it (default: the suite's, else a snapshot of the repo) --reuse With --suite: copy earlier rounds' finished runs of the unchanged base arm (baseline) or both arms (both) instead of running them again; the pair then did not run interleaved, so the round warns about drift (default: none) (default: none) --reuse-max-age With --reuse: only copy runs that finished this recently, e.g. 3d or 12h (default: 7d) --canary-model With --suite: re-run a kept candidate's two arms under this cheaper model and reject it on a regression there (default: [refine] canary_model) --canary With --suite: which kept candidates the canary re-tests: the ones that delete or reword existing lines (subtractive), every one (all), or none (off) (default: subtractive) --canary-delta With --canary-model: how far the canary pass rate's lower bound may fall before the candidate is rejected, e.g. 10pp (default: 10pp) --resume [] Continue an interrupted round: finished trials are kept, only the rest run (no ID: the newest unfinished one with the same config) --retries Retries per trial after a rate limit, 5xx or network error, with backoff; a failed setup step is retried once and a refused login never (failures are never retried) (default: 3) --stop-after-infra Stop once N trials in a row fail on infrastructure after their retries (a refused login, a failed setup, rate limits, 5xx, network): no new trial starts and the run is no quality verdict; 0 never stops (default: 5) --trial-timeout Time limit per trial, retries included; a trial over it fails -y, --yes Answer yes to confirmation prompts. An agent passes it only once the user approves --dry-run Show the checks, decision points, guard set size and cost estimate; make no model calls Examples coding.sh refine test ./drafts/AGENTS.md coding.sh refine test r_3f9a1c02e1 --behavior unverified-claim --max-cost 2 coding.sh refine test ./a/AGENTS.md ./b/AGENTS.md --dry-run --json ``` ## `coding.sh refine apply` Write a kept candidate to your files, backing up what it replaces (never commits). coding.sh refine apply --help ``` coding.sh refine apply [candidate] [options] Write a kept candidate to your files, backing up what it replaces (never commits) Arguments candidate Candidate id (default: the latest kept candidate not yet applied) Options --force Apply a candidate that was not kept, or whose files changed since its test (recorded in the ledger) Examples coding.sh refine apply coding.sh refine apply r_3f9a1c02e1 ``` ## `coding.sh refine rollback` Put back what an apply replaced, from its backup. coding.sh refine rollback --help ``` coding.sh refine rollback [candidate] [options] Put back what an apply replaced, from its backup Arguments candidate Candidate or apply id (default: the latest apply) Options --force Roll back though the file changed since the apply (those changes are lost) Examples coding.sh refine rollback coding.sh refine rollback r_3f9a1c02e1 ``` ## `coding.sh refine log` Every round in this repo: verdict, effect CI, guard result, cost and reason, and whether the loop plateaued; --suite climbs show pass rate, cost, time and size per round and the Pareto front. coding.sh refine log --help ``` coding.sh refine log Every round in this repo: verdict, effect CI, guard result, cost and reason, and whether the loop plateaued; --suite climbs show pass rate, cost, time and size per round and the Pareto front Examples coding.sh refine log coding.sh refine log --json ``` ## `coding.sh refine run` Loop: propose (or take --candidate), test, apply when kept; stop at --rounds, --max-cost or a plateau. coding.sh refine run --help ``` coding.sh refine run [options] Loop: propose (or take --candidate), test, apply when kept; stop at --rounds, --max-cost or a plateau Options --rounds Most rounds (default: 3, none with --until-plateau) --candidate Test these in order instead of asking the proposer (repeatable) --until-plateau With --suite: keep climbing until the e-detector says the climb stopped gaining; --resume continues an interrupted round --proposer-model Model that writes the edit (default: --judge-model, else --model, else claude-sonnet-5-5 or gpt-6-sol) --mined Also show the proposer the patterns in this file (the output of refine mine --json) --proposer-command Shell command that writes the edit instead of the proposer model: the prompt on stdin, the JSON reply on stdout (e.g. 'claude -p') --target Harness file the proposal edits, or a skill directory for its SKILL.md (default: AGENTS.md, else the repo's Claude Code or Gemini instructions file) --behavior Decision points to test: unverified-claim, done-while-failing, unchanged-retry (default: all) --anchor Module exporting your own anchor(s) (see the evals guide) --since Sessions started since (default: 14d) --agent Only sessions of this agent (repeatable, or a comma list) --repo Only sessions under this directory (default: this git repo) --provider Who proposes, answers the next turn and judges it: an API, or claude-code / codex to run `claude -p` / `codex exec` on your subscription (default: anthropic; with no API key set, a logged-in Claude Code, else Codex) --base-url API base URL (default: ANTHROPIC_BASE_URL / OPENAI_BASE_URL, else the provider's) --judge-model Model for the judge (default: --model) -n, --limit Maximum decision points (default: 40) --model Model for the next turn (default: the one the decision points were recorded with, else claude-sonnet-5-5 or gpt-6-sol) --trials Next turns per case and arm (default: 1) -c, --concurrency Parallel model calls (default: 4) --max-cost Refuse to start when the estimate is over this, and stop once the spend passes it --max-guard Most tasks in the guard set (the tasks earlier kept edits won), newest kept edits first (default: 24) --cache Reuse stored model responses (--no-cache to ask the model again) (default: true) --lambda Acceptance gate: what breaking one solved decision point costs, in fixed ones (default: 2) --gate-alpha Acceptance gate: one-sided significance level (default: 0.05) --tau Acceptance gate: largest share of solved decision points B may break (default: 0.10) --allow-protected-edit Allow the edit to delete, reword or relax this protected line (repeatable; never implied by --force) --suite Judge candidates on this eval suite of repo tasks (evals/) instead of mined decision points; a skill --target uses evals/ when it exists --objective With --suite: what to improve, primary first: pass, cost, tokens, wall, llm, turns, size (default: pass). The pass rate must hold within --margin whatever the list --margin With --suite: how far the pass rate may drop for a cheaper or faster candidate to be kept, e.g. 5pp (default: 2pp) --min-gain With --suite: smallest relative gain in cost, tokens, time, turns or size that counts, e.g. 20% (default: 10%) --max-cost-rise With --suite: how far a listed cost, tokens, time, turns or size objective may rise for a significant pass-rate gain, e.g. 100%, or none (default: refine.maxCostRise, else 50%) (default: 50%) --harness With --suite: stored harness bundle the suite runs, the repo's harness files laid over it (default: the suite's, else a snapshot of the repo) --reuse With --suite: copy earlier rounds' finished runs of the unchanged base arm (baseline) or both arms (both) instead of running them again; the pair then did not run interleaved, so the round warns about drift (default: none) (default: none) --reuse-max-age With --reuse: only copy runs that finished this recently, e.g. 3d or 12h (default: 7d) --canary-model With --suite: re-run a kept candidate's two arms under this cheaper model and reject it on a regression there (default: [refine] canary_model) --canary With --suite: which kept candidates the canary re-tests: the ones that delete or reword existing lines (subtractive), every one (all), or none (off) (default: subtractive) --canary-delta With --canary-model: how far the canary pass rate's lower bound may fall before the candidate is rejected, e.g. 10pp (default: 10pp) --resume [] Continue an interrupted round: finished trials are kept, only the rest run (no ID: the newest unfinished one with the same config) --retries Retries per trial after a rate limit, 5xx or network error, with backoff; a failed setup step is retried once and a refused login never (failures are never retried) (default: 3) --stop-after-infra Stop once N trials in a row fail on infrastructure after their retries (a refused login, a failed setup, rate limits, 5xx, network): no new trial starts and the run is no quality verdict; 0 never stops (default: 5) --trial-timeout Time limit per trial, retries included; a trial over it fails -y, --yes Answer yes to confirmation prompts. An agent passes it only once the user approves --dry-run Show the findings, decision points, guard set size and per-round estimate; make no model calls Examples coding.sh refine run --rounds 5 --max-cost 10 coding.sh refine run --candidate ./v1/AGENTS.md --candidate ./v2/AGENTS.md coding.sh refine run --dry-run ``` ## `coding.sh refine protected` List the harness lines an edit may not delete, reword or relax. coding.sh refine protected --help ``` coding.sh refine protected [options] List the harness lines an edit may not delete, reword or relax Options --a Harness to read (default: the repo's current AGENTS.md and other instructions files, rules and skills) Examples coding.sh refine protected coding.sh refine protected --a git:main --json ``` ## `coding.sh refine check` Check a harness edit for deleted, reworded or relaxed protected lines; exit 1 when it has any. coding.sh refine check --help ``` coding.sh refine check [options] Check a harness edit for deleted, reworded or relaxed protected lines; exit 1 when it has any Options --a Before (default: git HEAD when B is omitted, else the repo's current files) --b After; a file replaces the file of the same path in A (default: your uncommitted edits) --allow-protected-edit Allow the edit to delete, reword or relax this protected line (repeatable; never implied by --force) Examples coding.sh refine check coding.sh refine check --b ./drafts/AGENTS.md coding.sh refine check --a git:main --b . --json ``` ## `coding.sh refine mine` Rank failure patterns across sessions: repeated error lines, repeated instructions you typed, flags and failed eval runs, with counts and example sessions ($0). coding.sh refine mine --help ``` coding.sh refine mine [options] Rank failure patterns across sessions: repeated error lines, repeated instructions you typed, flags and failed eval runs, with counts and example sessions ($0) Options --since Sessions started since (default: 30d) --agent Only sessions of this agent (repeatable, or a comma list) --repo Only sessions under this directory, and eval runs of tasks that start from it or of the suites under its evals/ (default: this git repo) --min-support Keep a pattern seen in at least this many sessions over this many days (default: 3,2) -n, --limit Most patterns shown (default: 20) Examples coding.sh refine mine coding.sh refine mine --since 90d --repo all --json coding.sh refine mine --min-support 2,1 --agent codex ``` ## `coding.sh refine recheck` Re-run a suite round's base and candidate under another model (a canary) and record the verdict; suggests a rollback when the edit regresses there. coding.sh refine recheck --help ``` coding.sh refine recheck [options] Re-run a suite round's base and candidate under another model (a canary) and record the verdict; suggests a rollback when the edit regresses there Arguments candidate The round's r_… id Options --model The model to re-run under (default: [refine] canary_model) --trials Runs per task and arm (default: the suite's trials, else 3) --canary-delta How far the pass rate's lower bound may fall before it counts as a regression (default: 10pp) --max-cost Stop starting runs once agent spend reaches this; the recheck is then inconclusive --concurrency Agent sessions at once (default: 4) --dry-run Print what would run; start no agent -y, --yes Answer yes to confirmation prompts. An agent passes it only once the user approves Examples coding.sh refine recheck r_3f9a1c02e1 --model claude-haiku-4-5 coding.sh refine recheck r_3f9a1c02e1 --model gpt-5-mini --max-cost 3 --yes --json ``` ## `coding.sh refine distill` Hand a coding agent the raw runs of an experiment (or this repo's recent sessions) to analyze with its own code and write one skill; the skill is checked by the critics and ledgered as a candidate for refine test. coding.sh refine distill --help ``` coding.sh refine distill [options] Hand a coding agent the raw runs of an experiment (or this repo's recent sessions) to analyze with its own code and write one skill; the skill is checked by the critics and ledgered as a candidate for refine test Options --experiment Distill from this experiment's runs (default: captured sessions) --since Without --experiment: sessions started since (default: 30d) --agent Without --experiment: only sessions of this agent (repeatable, or a comma list) --repo Without --experiment: only sessions under this directory (default: this git repo) --distiller The coding agent that analyzes the corpus (default: claude-code) --model The distiller's model (default: the agent's own) --n Independent skills to draw, each in its own directory (default: 1) --max-cost Start no new draw once distiller spend reaches this (default: 5) --min-episodes Refuse a corpus with fewer episodes (default: 20) --min-failures Refuse a corpus with fewer failing or flagged episodes (default: 5) --keep-corpus Write the corpus here and keep it: an empty or new directory, or one that holds only an earlier corpus, which is replaced. Nothing is written there with --dry-run --dry-run Write the corpus to a temp dir, print its counts and the plan; start no agent -y, --yes Answer yes to confirmation prompts. An agent passes it only once the user approves Examples coding.sh refine distill --experiment exp_01J9Z --dry-run coding.sh refine distill --experiment exp_01J9Z --n 3 --yes coding.sh refine distill --since 60d --agent codex --distiller codex --keep-corpus ./corpus --yes --json ``` ## `coding.sh refine diagnose` Diagnose recent sessions with a model into Signals: cited problems grouped into issues, severity scored by code. Without an API key it prints a $0 plan. coding.sh refine diagnose --help ``` coding.sh refine diagnose [options] Diagnose recent sessions with a model into Signals: cited problems grouped into issues, severity scored by code. Without an API key it prints a $0 plan Options --since Sessions started since (default: 7d) --agent Only sessions of this agent (repeatable, or a comma list) --repo Only sessions under this directory (default: this git repo) --session Exactly these sessions: an id, a prefix or latest (repeatable; ignores --since, --repo and --agent) --max-sessions Most sessions diagnosed, newest first (default: 50) --budget-tokens Stop starting uncached sessions past this many estimated input tokens; the next run picks up the rest --provider anthropic, openai, claude-code or codex (default: the API whose key is set; with none, a $0 plan) --model The diagnosing model (default: the provider's current one) --concurrency Sessions diagnosed at once (default: 4) --no-split One Signal per component and type, with no grouping call --include-runs Also diagnose eval-run sessions --dry-run Print the plan and make no call Examples coding.sh refine diagnose coding.sh refine diagnose --since 30d --budget-tokens 400000 coding.sh refine diagnose --session 7f3a9c --provider claude-code --json ``` ## `coding.sh refine signals` List the Signals refine diagnose wrote, or show one with its cited evidence and fix. coding.sh refine signals --help ``` coding.sh refine signals [show] [id] List the Signals refine diagnose wrote, or show one with its cited evidence and fix Arguments show show : one Signal's occurrences, citations and fix id With show: the Signal id, or a prefix of it Examples coding.sh refine signals coding.sh refine signals show 3fa1c2 coding.sh refine signals --json ``` ## `coding.sh refine score` Score diagnoses against human labels: occurrence, Signal and session precision and recall, coverage, repeat agreement. coding.sh refine score --help ``` coding.sh refine score [options] Score diagnoses against human labels: occurrence, Signal and session precision and recall, coverage, repeat agreement Arguments labels Labels JSONL: one occurrence per line, {sessionId, signalId, component: {type, name}, typeIds: [1-3 ids], nature?}; a line with only sessionId marks a session with nothing found Options --pred Predictions in the labels format (repeatable; 2 or more adds repeat agreement). Default: the labelled sessions' flags from the store Examples coding.sh refine score labels.jsonl --pred diagnoses.jsonl coding.sh refine score labels.jsonl --pred run1.jsonl --pred run2.jsonl --pred run3.jsonl --json coding.sh refine score labels.jsonl ``` [All commands](https://coding.sh/docs/cli) · [Global options](https://coding.sh/docs/cli#global) To set coding.sh up for a user, follow the agent workflow at [/agents.md](https://coding.sh/agents.md).