--- title: "coding.sh FAQ: privacy, performance and alternatives" description: "Privacy, performance, status, and how coding.sh relates to hosted tracing tools." canonical: https://coding.sh/docs/faq human_url: https://coding.sh/docs/faq markdown_url: https://coding.sh/docs/faq.md agents_url: https://coding.sh/agents.md llms_url: https://coding.sh/llms.txt updated: 2026-11-11 --- # FAQ Privacy, performance, status, and how coding.sh relates to hosted tracing tools. ## How do you say coding.sh, and is it the same as “coding shell”? It is one tool with one name. The `.sh` is a domain, and because `.sh` files are shell scripts it reads as “coding shell”, so people also write it coding shell, coding sh or codingsh. They all mean this: open-source, local-first observability and evals for AI agents, run from your shell. ## Can I use it today? Yes, if you accept early software (v0.1.4): capture, the viewer, datasets, graders, judges and comparisons work, and the package's test suite passes. Nothing has a stable interface yet, and drafting repo-replay tasks from real sessions is still blocked (see [Evals](https://coding.sh/docs/harness-evals#repo-replay)). ## Does anything leave my machine? Not unless you configure a [sink](https://coding.sh/docs/storage#sinks), and sinks default to metadata mode, which sends no prompts, code or tool output. The receiver binds to 127.0.0.1. LLM judges call the provider you configure, with your key, only when you run an experiment that uses one. ## What about secrets in my sessions? Redaction runs before anything is written: API keys with known vendor prefixes, bearer tokens, and values next to secret-looking names (OPENAI_API_KEY=…, password: …) become [redacted:kind]. You can add your own patterns, exclude repositories entirely, or switch to metadata mode. ## Will the hooks slow my agent down? The hook reads stdin, appends one line to a file and exits. It makes no network calls, runs no git commands, spawns nothing and parses nothing beyond JSON.parse. Normalization happens later, at ingest. If capture fails, the hook still exits 0 and logs the failure. ## Will it mess with my agent configs? init shows every edit with --dry-run, backs up each file before changing it, marks its own entries, and uninstall removes only those. It never answers a permission prompt for you. Eval runs use isolated config directories, so your global setup is not touched. For Codex, coding.sh never writes hook trust; you approve the hooks yourself. ## How is this different from LangSmith, Braintrust or Judgment Labs? They are hosted platforms that capture and grade traces. coding.sh does the same on your machine, with no account, and it is MIT licensed. It also treats coding agents as first-class: adapters for a dozen agents, subagents as child sessions, and tasks that replay from the repository state before the agent ran, scored by a verifier that has itself been checked (drafting those tasks from captured sessions doesn't work yet). ## My agent isn't a coding agent. Does it work? Yes. Anything that exports OpenTelemetry traces works through `coding.sh collector`: the Vercel AI SDK, the Claude Agent SDK, the OpenAI Agents SDK, LangGraph, or your own code. See [OpenTelemetry](https://coding.sh/docs/otel). For evals, any program that reads JSON on stdin or answers HTTP can be the task. ## My coding agent isn't supported. Wrap it: `coding.sh run -- ` records repository snapshots before and after and every changed file. If it has hooks, an adapter is the most useful contribution; a recorded session from it helps too. Open an [issue](https://github.com/jackulau/coding.sh/issues). ## Why does my LLM judge say uncalibrated? A judge's scores count only after it agrees with human labels: Cohen's κ of at least 0.6 on 20 or more held-out cases it wasn't tuned on. Label sessions in `coding.sh ui` and run `coding.sh judge calibrate`. See [calibration](https://coding.sh/docs/evals#calibration). ## Where is my data, and how do I delete it? In `~/.coding.sh` (or `CODING_SH_HOME`). `coding.sh gc` applies retention; deleting the directory removes everything. `coding.sh uninstall` removes the hooks and keeps the data. ## What is the license? MIT. The source is on [GitHub](https://github.com/jackulau/coding.sh). To set coding.sh up for a user, follow the agent workflow at [/agents.md](https://coding.sh/agents.md).