GETTING STARTED

Connect your traces

You already have the traces — your trace store has been writing your agent's judgments down the whole time. Coldworks reads them where they live: on your machine. A batch trace export is a file transfer, not an integration.

The three steps

Export your traces from your store (about 30 seconds — every store already has this).
Run uvx coldworks-audit run traces.jsonl locally.
Read the report it writes next to the file.

ILLUSTRATIVE — NO REAL STORE BEHIND THIS PAGE

$ uvx coldworks-audit run traces.jsonl

read 41,218 spans · 9,406 tool calls · your tool names, kept
2,871 exact repeats across 214 groups

  → 1,704 tier-0 cache candidates   results agree on every repeat
  → 1,167 repeats, results disagree  nondeterminism findings

not counted as findings, not hidden:
  322 judgments stated only in prose · 118 unanchorable

report: ./coldworks-audit.html

Candidates, never savings — a repeated judgment is a candidate for compilation, not a booked dollar. The report prints what it could not read in the same breath as what it could.

From Langfuse

Langfuse observations carry name, input, output, and token usage — everything the audit needs. Three rungs, all running with your credentials on your compute:

Rung	How it works	Status
File	Traces → Export in the Langfuse UI, then `coldworks-audit run` on the downloaded file.	DESIGNED
Standing file	Langfuse's scheduled export writes fresh traces to your S3 / GCS bucket on its own timer; your cron line re-audits whatever lands. Two schedulers, both yours — there is no Coldworks daemon.	DESIGNED
Direct pull	`coldworks-audit run --from langfuse` reads `LANGFUSE_PUBLIC_KEY` / `LANGFUSE_SECRET_KEY` from your environment and pages the observations API itself — incremental via `--since`.	PLANNED · AFTER FILE PATH

From LangSmith

Open the project, export runs, audit the file. Where the export carries token usage the report weights the headline by tokens rather than span count.

From OpenTelemetry

Point your collector's file exporter at a directory and audit what lands there. For OTel-shaped exports without usage fields, the headline stays in span counts and says so.

What leaves your machine: nothing

Thing	Where it goes
Your traces	Read from your disk, never transmitted.	NEVER LEAVES
Your documents (`--docs`)	Hashed and matched locally, never transmitted.	NEVER LEAVES
The audit report	An HTML file on your disk. Forward it yourself, or don't.	STAYS LOCAL
Derived figures	A future opt-in `--push` could publish headline figures to a hosted registry page. It does not exist, and ships only under an explicitly signed exception.	DOES NOT EXIST

This is doctrine, not a feature flag: Coldworks compiles judgments into code you own and you run. Auditing starts on the same side of the boundary the guards live on — yours.

Day 2: make it standing

The report footer prints one line for your scheduler — cron, GitHub Actions, or GitLab CI. A shell command in your infrastructure; never an app you install or a marketplace listing.

REPORT FOOTER · ILLUSTRATIVE

# re-audit whatever lands in ./exports, every night at 02:10
10 2 * * *  uvx coldworks-audit run ./exports --diff >> audit.log

Each run appends to a local append-only ledger, and --diff reports what changed: new repeated judgments since the last run. That number rising is your agent re-buying the same decision with fresh tokens.

Honest limit: the cron line monitors only if fresh exports keep landing in that directory. Langfuse's scheduled bucket export closes the loop today; the direct --from langfuse pull is the planned successor. There is no Coldworks daemon, resident process, or phone-home in any rung.

PREVIOUSQuickstart NEXTThe audit CLI