Artifact

Gate Before You Publish: Deterministic Claim-Level Checks for LLM-Drafted Career Content and a Provenance Audit of a 12.8k-Record Generated Corpus
# Artifact: Gate Before You Publish

Technical report by Yousof Selim, v1.0, 2 November 2026. Everything here was run on 1–2 November 2026
(Asia/Kuala_Lumpur) on macOS (Darwin 27.0.0), Node v26.0.0, pnpm 11.9.0, Python 3 (standard library only).

**No paid model was called.** All CareerOS results come from its deterministic gate code and its
deterministic (template) provider. They are not measurements of a language model.

## Layout

```
careeros/
  cases.ts                  the 47 evaluation cases (synthetic fixtures; CareerOS 63bcd5c, scripts/eval/cases.ts)
  eval-output-63bcd5c.txt   stdout of `pnpm eval` on a clean worktree of 63bcd5c
  gate-study.ts             harness for E1–E5 (ablation, ungated baseline, pre-M4 gates, probes)
  probes.ts                 the 24 exploratory paraphrase probes (labels fixed before the first run)
  gate-study-results.json   raw output of gate-study.ts
adelante/
  content_quality_audit.py  Adelante's audit script, changed only to take input/output paths (see .diff)
  content_quality_audit.diff
  run-corpus-audit.ts       runs Adelante's functions/src/corpusAudit.ts on a library file
  draw_sample.py            seeded uniform random sample, joined with the rule-based findings
  annotations-200.json      codebook and single-annotator labels for the 200-record sample
  summarise_sample.py       rates and Wilson 95% intervals (sample) and census counts (whole asset)
  results/                  outputs (long per-record ID lists trimmed to counts), input SHA-256s, the sample
figures/make_figures.py     draws the four SVG figures from the files above
```

## What is not here, and why

CareerOS is a private repository. Its gate code (`packages/core/src/gates.ts`), the template
provider and the pipeline are not reproduced here. The report describes the gate rules in enough
detail to re-implement them; the harness, cases, probes and raw outputs are complete. To re-run the
CareerOS experiments you need read access to the repository at commit
`63bcd5c1db2f8d3d5e1ff54221518591eca2784d`. Adelante's model and curation modules
(`functions/src/model.ts`, `functions/src/curation.ts`, `functions/src/corpusAudit.ts`) and its
12.8k-record library are likewise not copied in full; the 200 sampled records are included in
`adelante/results/sample-200.json` as a representative public subset.

## Commands

### CareerOS (E1–E5)

```sh
# a clean worktree; the original checkout is never touched
git -C <careeros> worktree add --detach <tmp>/careeros-63bcd5c 63bcd5c
cd <tmp>/careeros-63bcd5c
pnpm install --frozen-lockfile --ignore-scripts
env -u ANTHROPIC_API_KEY pnpm eval                       # E1 headline (never pass --live)
mkdir -p scripts/research/pre-m4
cp <artifact>/careeros/{gate-study.ts,probes.ts} scripts/research/
git show 70b84e6^:packages/core/src/gates.ts > scripts/research/pre-m4/gates.ts
git show 70b84e6^:packages/core/src/enums.ts > scripts/research/pre-m4/enums.ts
env -u ANTHROPIC_API_KEY pnpm exec tsx scripts/research/gate-study.ts > gate-study-results.json
```

`gate-study.ts` disables a gate by dropping that gate's findings from the full report and
recomputing the verdict. In `validateVariant` each gate only appends its own findings and no gate
reads another's result, so this is the same as switching the gate off.

### Adelante (provenance audit)

Inputs are read from git objects, so the Adelante working tree is not touched:

```sh
git -C <adelante> show 3e0d06d:assets/content/library.json > data/library-3e0d06d.json   # before the 21 Sep audit
git -C <adelante> show af1aea2:assets/content/library.json > data/library-af1aea2.json   # current (asset last changed in f7d479e)
for f in model curation corpusAudit; do git -C <adelante> show af1aea2:functions/src/$f.ts > src/$f.ts; done
python3 -I content_quality_audit.py data/library-3e0d06d.json results/content-metrics-3e0d06d.json
python3 -I content_quality_audit.py data/library-af1aea2.json results/content-metrics-af1aea2.json
npx tsx run-corpus-audit.ts data/library-3e0d06d.json results/corpus-audit-3e0d06d.json
npx tsx run-corpus-audit.ts data/library-af1aea2.json results/corpus-audit-af1aea2.json
python3 -I draw_sample.py data/library-af1aea2.json results/corpus-audit-af1aea2.json 200 20261102 results/sample-200.json
python3 -I summarise_sample.py results/sample-200.json annotations-200.json results/corpus-audit-af1aea2.json data/library-af1aea2.json
```

Expected input hashes are in `adelante/results/input-asset-sha256.txt`
(`728a70ab…` for 3e0d06d, matching the "initial asset" hash in Adelante's audit; `a1fcdb34…` for af1aea2).

### Figures

```sh
python3 -I figures/make_figures.py <output-dir>
```

## Annotation caveat

The issue classes M1–M5 in `annotations-200.json` were labelled by one annotator (the AI assistant
drafting the report, under Yousof Selim's direction), from a written codebook, without a second
annotator, adjudication or source searches. Treat the rates as a first estimate.