{ "id": "definition-of-done-proof", "type": "loadout", "title": "Definition of done for agents, with proof", "summary": "Make \"done\" mean the feature works, a test shows it, and no test was weakened, with a git-diff check for changed assertions, skips and deleted tests.", "description": "A definition of done written for people says \"tests pass\", and everyone knows that also means someone clicked through the feature. Agents read it literally. When a test fails, changing the expected value, deleting the assertion or adding a skip marker is the cheapest way to green, and the agent reports done. Developers keep finding this out after merging.\n\nThis loadout rewrites done for agents: the behaviour was run and seen, new behaviour has a new test (a bug fix has one that failed first), the full suite passes, nothing was checked by \"should work\", and no existing test was loosened. A standard-library Python script reads the git diff of test files and reports changed expectations, removed assertions, added skip/xfail/only/ignore markers, deleted tests and changed snapshots. It works across pytest, Jest/Vitest, Go, Rust, JUnit and Swift Testing, and counts new tests without blocking them. Human-approved test changes go in an allow list that only the human edits. An optional Claude Code Stop hook turns the check into a wall, and a report template puts evidence next to every claim.\n", "highlights": [ "Done = seen working + a test that shows it + no weakened tests + what wasn't checked", "Diff check for changed expectations, removed asserts, skips, deleted tests, snapshots", "Works on pytest, Jest/Vitest, Go, Rust, JUnit and Swift Testing files", "Human-only allow list for test changes that were genuinely wrong", "Optional Claude Code Stop hook (exit 2, loop-safe) and a done-report template" ], "version": "1.0.0", "license": "MIT", "author": { "name": "SetupBay", "url": "https://setupbay.com" }, "credits": [ { "title": "Claude Code docs: hooks reference (Stop hook, exit code 2, stop_hook_active)", "url": "https://code.claude.com/docs/en/hooks" }, { "title": "@AliceInfoAi: agents read 'tests pass' literally", "url": "https://x.com/AliceInfoAi/status/2110219293904257246" }, { "title": "@Dev_Great_dev: editing the assertion is cheaper than fixing the bug", "url": "https://x.com/Dev_Great_dev/status/2110903441031643577" }, { "title": "@FastfixAI: tests green, feature still broken, a definition-of-done problem", "url": "https://x.com/FastfixAI/status/2110511793475027233" } ], "categories": [ "workflow", "ai-agents" ], "stacks": [ "git", "python", "bash", "pytest" ], "files": [ { "path": "AGENTS.md", "role": "instructions", "description": "The five-part definition of done and what to do when a test seems wrong.", "skill": null, "agent": null, "install_to": { "claude-code": "AGENTS.md", "codex": "AGENTS.md", "cursor": "AGENTS.md", "gemini-cli": "AGENTS.md", "antigravity-cli": "AGENTS.md", "copilot": "AGENTS.md" }, "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/AGENTS.md", "raw_latest_url": "https://setupbay.com/raw/definition-of-done-proof/latest/AGENTS.md", "bytes": 1288, "lang": "markdown", "content": "# AGENTS.md: definition of done for coding agents\n\n\"Done\" is a claim the human will rely on. Make it only with evidence.\n\n## Done means all of these\n1. **It works.** You ran the behaviour that was asked for (command, request, page) and saw the right result.\n2. **A test shows it.** New behaviour has a new test. A bug fix has a test that failed before the fix and passes now.\n3. **No test was weakened.** You did not change an existing assertion's expected value, delete an assertion or test,\n add skip/xfail/only/ignore markers, or accept new snapshots unread. The `done-proof` skill's tamper check\n prints `0 finding(s) to explain`.\n4. **The full suite passes**, with the summary line in your report (passed / failed / skipped).\n5. **You said what you could not check.**\n\n## When a test seems wrong\nStop. Name the test, what it expects, and why you think that is wrong. Change it only after the human agrees.\nApproved test files are listed in `.done-proof-allow` by the human, never by you.\n\n## Reporting\n- End with the done report from the `done-proof` skill: what was asked, how you saw it work, tests (command and\n summary, new tests, changed tests), what was not checked, what needs the human.\n- Never write \"should work\", \"probably\" or \"done\" without the evidence next to it.\n" }, { "path": "CLAUDE.md", "role": "adapter", "description": "Imports AGENTS.md for Claude Code.", "skill": null, "agent": "claude-code", "install_to": { "claude-code": "CLAUDE.md", "codex": null, "cursor": null, "gemini-cli": null, "antigravity-cli": null, "copilot": null }, "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/CLAUDE.md", "raw_latest_url": "https://setupbay.com/raw/definition-of-done-proof/latest/CLAUDE.md", "bytes": 382, "lang": "markdown", "content": "# CLAUDE.md\n\n@AGENTS.md\n\n## Claude Code notes\n\n- Skills from this loadout live in `.claude/skills/`: `done-proof`. Use them when their description matches the task.\n- Follow AGENTS.md above; where this file and AGENTS.md disagree, AGENTS.md wins.\n\n\n" }, { "path": "GEMINI.md", "role": "adapter", "description": "Imports AGENTS.md for Gemini CLI.", "skill": null, "agent": "gemini-cli", "install_to": { "claude-code": null, "codex": null, "cursor": null, "gemini-cli": "GEMINI.md", "antigravity-cli": null, "copilot": null }, "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/GEMINI.md", "raw_latest_url": "https://setupbay.com/raw/definition-of-done-proof/latest/GEMINI.md", "bytes": 383, "lang": "markdown", "content": "# GEMINI.md\n\n@./AGENTS.md\n\n## Gemini CLI notes\n\n- Skills from this loadout live in `.agents/skills/`: `done-proof`. Use them when their description matches the task.\n- Follow AGENTS.md above; where this file and AGENTS.md disagree, AGENTS.md wins.\n\n\n" }, { "path": "skills/done-proof/SKILL.md", "role": "skill", "description": "Prove done with a failing-then-passing test, a run of the feature and the tamper check.", "skill": "done-proof", "agent": null, "install_to": { "claude-code": ".claude/skills/done-proof/SKILL.md", "codex": ".agents/skills/done-proof/SKILL.md", "cursor": ".agents/skills/done-proof/SKILL.md", "gemini-cli": ".agents/skills/done-proof/SKILL.md", "antigravity-cli": ".agents/skills/done-proof/SKILL.md", "copilot": ".agents/skills/done-proof/SKILL.md" }, "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/SKILL.md", "raw_latest_url": "https://setupbay.com/raw/definition-of-done-proof/latest/skills/done-proof/SKILL.md", "bytes": 2937, "lang": "markdown", "content": "---\nname: done-proof\ndescription: Prove a coding task is done before saying so, with evidence the feature works and proof the tests were not weakened. Use before reporting any task as done, fixed or complete, when tests fail and it is tempting to change them, or when the user asks for proof.\n---\n\n# Done means proven\n\n\"Tests pass\" was written for people, who also clicked through the feature. Agents read it literally, and changing an\nassertion is cheaper than fixing a bug. This skill makes \"done\" mean three things: the behaviour works, a test shows\nit, and no existing test was loosened to get there.\n\n## 1. Before you change code\n- Write down what done looks like for this task in one or two lines: the user-visible result, and the check that\n shows it (a command, a page, an output).\n- For a bug: write or find a test that fails because of the bug. Run it and keep the failure output. That failure is\n half of your proof.\n\n## 2. While you work\n- Never edit an existing assertion's expected value, delete an assertion or a test, or add `skip`, `xfail`, `.only`,\n `@Disabled`, `t.Skip` or `#[ignore]` to get to green. Fix the code.\n- If you believe a test is wrong (the spec changed, the test was buggy), stop and say so: which test, what it expects,\n why that is wrong. Change it only after the human agrees, and they add the file to `.done-proof-allow`.\n- Never accept updated snapshots without reading the diff and saying why the new output is right.\n\n## 3. Check the tests were not weakened\n```bash\npython3 .agents/skills/done-proof/scripts/test_tamper.py # uncommitted changes\npython3 .agents/skills/done-proof/scripts/test_tamper.py --base main # the whole branch\n```\n(Claude Code installs the skill under `.claude/skills/`; use that path there.)\nIt must print `0 finding(s) to explain`. It reports changed expectations, removed assertions, added skip markers,\ndeleted tests and changed snapshots in test files. New tests and new assertions are counted, never blocked.\n\n## 4. Show it works\n- Run the full test command and keep the summary line (passed, failed, skipped).\n- Run the feature itself: the CLI with real input, the endpoint with a real request, the page in a browser (screenshot or\n the visible text). Unit tests alone do not show a feature works end to end.\n- Re-run the failing test from step 1: it passes now.\n\n## 5. Report with evidence\nUse `references/done-report.md`. Every claim has the command or observation behind it. Put what you could not check\nunder \"Not checked\". Never write \"should work\".\n\n## Optional: make it a wall (Claude Code)\nA rule asks; a hook enforces. `scripts/done-gate.sh` is a Stop hook that runs the tamper check and refuses to let\nthe session end while it has findings (exit 2, with `stop_hook_active` respected so it cannot loop). To install it,\nmerge `references/stop-hook.example.json` into `.claude/settings.json` (ask the human first; it changes their settings).\n" }, { "path": "skills/done-proof/scripts/test_tamper.py", "role": "reference", "description": "Git-diff check for weakened, skipped or deleted tests.", "skill": "done-proof", "agent": null, "install_to": { "claude-code": ".claude/skills/done-proof/scripts/test_tamper.py", "codex": ".agents/skills/done-proof/scripts/test_tamper.py", "cursor": ".agents/skills/done-proof/scripts/test_tamper.py", "gemini-cli": ".agents/skills/done-proof/scripts/test_tamper.py", "antigravity-cli": ".agents/skills/done-proof/scripts/test_tamper.py", "copilot": ".agents/skills/done-proof/scripts/test_tamper.py" }, "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/scripts/test_tamper.py", "raw_latest_url": "https://setupbay.com/raw/definition-of-done-proof/latest/skills/done-proof/scripts/test_tamper.py", "bytes": 6954, "lang": "markdown", "content": "#!/usr/bin/env python3\n\"\"\"Find changes that make tests pass without fixing anything: weakened, skipped or deleted tests.\n\nUsage: python3 test_tamper.py [--base REF] [--allow FILE]\n --base compare the working tree (committed + uncommitted) with REF (default: HEAD; use main or origin/main for a branch)\n --allow file listing test paths a human approved changing, one per line (default: .done-proof-allow if present)\n\nLooks only at test files (tests/, test/, __tests__/, spec/, *_test.*, *.test.*, *.spec.*, test_*.py, *Tests.swift, *Test.java).\nReports, per file:\n removed-assert a line with an assertion was deleted or changed\n changed-expect an existing assertion now expects a different value\n added-skip a skip / xfail / only / ignore marker was added\n deleted-test a test function or case was removed\n snapshot a snapshot file was changed\nNew tests and new assertions are fine and are only counted.\nExit 1 if anything is found outside the allow list, else 0. Read-only; needs git.\n\"\"\"\nimport os\nimport re\nimport subprocess\nimport sys\n\nTEST_PATH = re.compile(r\"(^|/)(tests?|__tests__|spec|specs)/|(_test|\\.test|\\.spec|_spec)\\.[a-z]+$|(^|/)test_[^/]*\\.py$|Tests?\\.(swift|java|kt|cs)$|_test\\.go$\")\nSNAPSHOT = re.compile(r\"(\\.snap$|__snapshots__/|\\.ambr$|snapshots?/)\")\nASSERT = re.compile(r\"\\b(assert\\w*|expect\\s*\\(|should\\b|\\.to(Be|Equal|Have|Match|Throw|Contain)\\w*|XCTAssert\\w*|#expect|#require|require\\.\\w+|t\\.(Error|Fatal|Fail)\\w*|assert_eq!|assert_ne!|assert!|refute\\w*)\", re.I)\nSKIP = re.compile(r\"(@pytest\\.mark\\.(skip|xfail)|pytest\\.skip\\(|@unittest\\.skip|\\b(it|test|describe|context)\\.(skip|only|todo)\\b|\\bx(it|describe|test)\\s*\\(|\\bf(it|describe)\\s*\\(|@Disabled\\b|@Ignore\\b|\\bt\\.Skip(Now|f)?\\(|#\\[ignore\\]|\\.disabled\\(|skip\\s*:\\s*true|\\bpending\\s*\\()\")\nTEST_DEF = re.compile(r\"^\\s*(def test_\\w+|async def test_\\w+|func Test\\w+|fn test_\\w+|#\\[test\\]|@Test\\b|(it|test)\\s*\\(\\s*['\\\"`])\")\n\n\ndef git(*args):\n r = subprocess.run([\"git\", *args], capture_output=True, text=True)\n if r.returncode != 0:\n sys.exit(f\"git {' '.join(args)} failed: {r.stderr.strip()}\")\n return r.stdout\n\n\ndef expected_value(line):\n \"\"\"The value side of an assertion, roughly: what comes after ==, toBe(, toEqual(, assertEqual(x, ...\"\"\"\n m = re.search(r\"(==|!=|\\bis\\b|\\.to\\w*\\(|assert\\w*\\([^,]+,)\\s*(.+)$\", line)\n return m.group(2).strip().rstrip(\")\").strip() if m else None\n\n\ndef subject(line):\n m = re.search(r\"(assert\\w*\\s*\\(?|expect\\s*\\()\\s*([^=,)]+)\", line)\n return re.sub(r\"\\s+\", \"\", m.group(2)) if m else None\n\n\ndef main(argv):\n base = \"HEAD\"\n allow_file = \".done-proof-allow\"\n if \"--base\" in argv:\n base = argv[argv.index(\"--base\") + 1]\n if \"--allow\" in argv:\n allow_file = argv[argv.index(\"--allow\") + 1]\n allowed = set()\n if os.path.exists(allow_file):\n allowed = {l.strip() for l in open(allow_file) if l.strip() and not l.startswith(\"#\")}\n git(\"rev-parse\", \"--verify\", base)\n # Untracked test files count as new tests: add them to the diff with intent-to-add would change the index, so list them.\n untracked = [p for p in git(\"ls-files\", \"--others\", \"--exclude-standard\").splitlines() if TEST_PATH.search(p)]\n diff = git(\"diff\", \"--no-color\", \"--unified=0\", \"--find-renames\", base, \"--\")\n findings, new_tests, new_asserts = [], 0, 0\n path, removed, added = None, [], []\n\n def flush():\n nonlocal new_tests, new_asserts\n if not path or not TEST_PATH.search(path) and not SNAPSHOT.search(path):\n return\n if SNAPSHOT.search(path):\n if removed or added:\n findings.append((path, \"snapshot\", \"snapshot changed; confirm the new output is right, not just accepted\"))\n return\n rem_asserts = [l for l in removed if ASSERT.search(l)]\n add_asserts = [l for l in added if ASSERT.search(l)]\n add_by_subject = {}\n for l in add_asserts:\n add_by_subject.setdefault(subject(l), []).append(l)\n matched = set()\n for l in rem_asserts:\n s = subject(l)\n twins = add_by_subject.get(s) or []\n if twins:\n t = twins.pop(0)\n matched.add(id(t))\n if expected_value(l) != expected_value(t):\n findings.append((path, \"changed-expect\", f\"{l.strip()[:90]} -> {t.strip()[:90]}\"))\n else:\n findings.append((path, \"removed-assert\", l.strip()[:120]))\n for l in added:\n if SKIP.search(l):\n findings.append((path, \"added-skip\", l.strip()[:120]))\n rem_defs = [l for l in removed if TEST_DEF.search(l)]\n norm = lambda l: re.sub(r\"\\.(skip|only|todo)\\b|^x(?=(it|test|describe)\\b)\", \"\", l.strip())\n add_defs = {l.strip() for l in added if TEST_DEF.search(l) or SKIP.search(l)}\n add_norm = {norm(l) for l in add_defs}\n for l in rem_defs:\n if norm(l) not in add_norm: # a test that was only marked skip is reported once, as added-skip\n findings.append((path, \"deleted-test\", l.strip()[:120]))\n new_tests += max(0, len([l for l in add_defs if TEST_DEF.search(l)]) - len(rem_defs))\n new_asserts += len([l for l in add_asserts if id(l) not in matched])\n\n for line in diff.splitlines():\n if line.startswith(\"diff --git\"):\n flush()\n path, removed, added = None, [], []\n elif line.startswith(\"+++ \"):\n p = line[4:]\n path = p[2:] if p.startswith(\"b/\") else (None if p == \"/dev/null\" else p)\n elif line.startswith(\"--- \") and path is None:\n p = line[4:]\n if p != \"/dev/null\":\n path = p[2:] if p.startswith(\"a/\") else p\n elif line.startswith(\"-\") and not line.startswith(\"---\"):\n removed.append(line[1:])\n elif line.startswith(\"+\") and not line.startswith(\"+++\"):\n added.append(line[1:])\n flush()\n for p in untracked:\n text = open(p, encoding=\"utf-8\", errors=\"replace\").read().splitlines()\n new_tests += sum(1 for l in text if TEST_DEF.search(l))\n new_asserts += sum(1 for l in text if ASSERT.search(l))\n for l in text:\n if SKIP.search(l):\n findings.append((p, \"added-skip\", l.strip()[:120]))\n\n blocking = [f for f in findings if f[0] not in allowed]\n for p, kind, msg in findings:\n note = \" (approved in allow list)\" if p in allowed else \"\"\n print(f\"{p}: {kind}: {msg}{note}\")\n print(f\"test-tamper: base {base}, {len(blocking)} finding(s) to explain, {len(findings) - len(blocking)} approved, {new_tests} new test(s), {new_asserts} new assertion(s)\")\n if blocking:\n print(\"Not done: fix the code, not the test. If a test was wrong, say why and ask the human to add the file to .done-proof-allow.\", file=sys.stderr)\n return 1\n return 0\n\n\nif __name__ == \"__main__\":\n sys.exit(main(sys.argv[1:]))\n" }, { "path": "skills/done-proof/scripts/done-gate.sh", "role": "reference", "description": "Optional Claude Code Stop hook that runs the tamper check.", "skill": "done-proof", "agent": null, "install_to": { "claude-code": ".claude/skills/done-proof/scripts/done-gate.sh", "codex": ".agents/skills/done-proof/scripts/done-gate.sh", "cursor": ".agents/skills/done-proof/scripts/done-gate.sh", "gemini-cli": ".agents/skills/done-proof/scripts/done-gate.sh", "antigravity-cli": ".agents/skills/done-proof/scripts/done-gate.sh", "copilot": ".agents/skills/done-proof/scripts/done-gate.sh" }, "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/scripts/done-gate.sh", "raw_latest_url": "https://setupbay.com/raw/definition-of-done-proof/latest/skills/done-proof/scripts/done-gate.sh", "bytes": 927, "lang": "markdown", "content": "#!/usr/bin/env bash\n# Claude Code Stop hook: refuse \"done\" while test files were weakened, skipped or deleted.\n# Exit 2 keeps Claude working and shows it the findings. Respects stop_hook_active so it never loops.\n# Base ref: DONE_PROOF_BASE (default HEAD, i.e. this session's uncommitted changes).\nset -u\ninput=$(cat)\nprintf '%s' \"$input\" | python3 -c 'import json,sys; sys.exit(0 if json.load(sys.stdin).get(\"stop_hook_active\") else 1)' 2>/dev/null && exit 0\ncd \"${CLAUDE_PROJECT_DIR:-.}\" || exit 0\ngit rev-parse --is-inside-work-tree >/dev/null 2>&1 || exit 0\nscript=\"${CLAUDE_PROJECT_DIR:-.}/.claude/skills/done-proof/scripts/test_tamper.py\"\n[ -f \"$script\" ] || script=\".agents/skills/done-proof/scripts/test_tamper.py\"\n[ -f \"$script\" ] || exit 0\nif ! out=$(python3 \"$script\" --base \"${DONE_PROOF_BASE:-HEAD}\" 2>&1); then\n printf 'You changed tests to make them pass. You are not done:\\n%s\\n' \"$out\" >&2\n exit 2\nfi\nexit 0\n" }, { "path": "skills/done-proof/references/stop-hook.example.json", "role": "reference", "description": "Settings snippet for the Stop hook.", "skill": "done-proof", "agent": null, "install_to": { "claude-code": ".claude/skills/done-proof/references/stop-hook.example.json", "codex": ".agents/skills/done-proof/references/stop-hook.example.json", "cursor": ".agents/skills/done-proof/references/stop-hook.example.json", "gemini-cli": ".agents/skills/done-proof/references/stop-hook.example.json", "antigravity-cli": ".agents/skills/done-proof/references/stop-hook.example.json", "copilot": ".agents/skills/done-proof/references/stop-hook.example.json" }, "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/references/stop-hook.example.json", "raw_latest_url": "https://setupbay.com/raw/definition-of-done-proof/latest/skills/done-proof/references/stop-hook.example.json", "bytes": 357, "lang": "json", "content": "{\n \"hooks\": {\n \"Stop\": [\n {\n \"hooks\": [\n {\n \"type\": \"command\",\n \"command\": \"bash\",\n \"args\": [\"${CLAUDE_PROJECT_DIR}/.claude/skills/done-proof/scripts/done-gate.sh\"],\n \"timeout\": 60,\n \"statusMessage\": \"Checking tests were not weakened\"\n }\n ]\n }\n ]\n }\n}\n" }, { "path": "skills/done-proof/references/done-report.md", "role": "reference", "description": "The done report template.", "skill": "done-proof", "agent": null, "install_to": { "claude-code": ".claude/skills/done-proof/references/done-report.md", "codex": ".agents/skills/done-proof/references/done-report.md", "cursor": ".agents/skills/done-proof/references/done-report.md", "gemini-cli": ".agents/skills/done-proof/references/done-report.md", "antigravity-cli": ".agents/skills/done-proof/references/done-report.md", "copilot": ".agents/skills/done-proof/references/done-report.md" }, "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/references/done-report.md", "raw_latest_url": "https://setupbay.com/raw/definition-of-done-proof/latest/skills/done-proof/references/done-report.md", "bytes": 631, "lang": "markdown", "content": "# Done report (copy into your final message)\n\n**Asked for:** \n\n**Works:** \n\n**Tests:**\n- Command: `` → \n- New tests: , each failed before the fix (`` showed ) and passes now\n- Existing tests changed: none / \n- test-tamper: `0 finding(s) to explain`\n\n**Not checked:** \n\n**Needs you:** \n" } ], "requires": { "tools": [ "git", "python3" ], "env": [], "mcp": [] }, "tested": { "status": "untested", "last_run": null, "fixture": null, "per_agent": { "claude-code": "untested", "codex": "untested", "cursor": "untested", "gemini-cli": "untested", "antigravity-cli": "untested", "copilot": "untested" } }, "security": { "scanned": true, "scanner": "intake-precheck/2", "verdict": "clean", "network": false, "scripts": [ "test_tamper.py", "done-gate.sh" ], "notes": "The check only runs read-only git commands (rev-parse, diff, ls-files) and reads test files. The optional Stop hook runs that check and nothing else, and is installed only if you merge the settings snippet." }, "featured": false, "deprecated": false, "last_checked": "2026-10-24", "published": "2026-10-24", "updated": "2026-10-24", "compatibility": { "claude-code": "native", "codex": "native", "cursor": "native", "gemini-cli": "native", "antigravity-cli": "native", "copilot": "native" }, "counts": { "rules": 1, "skills": 1, "mcp": 0 }, "url": "https://setupbay.com/loadouts/definition-of-done-proof/", "api_url": "https://setupbay.com/api/loadouts/definition-of-done-proof.json", "verification": { "level": "verified-install", "status": "verified", "label": "Verified install", "verified": true, "checked_at": "2026-10-24T10:30:13Z", "commit": "42646ff9a52315018c9b0947c291c7ad980e1dc7", "content_current": true, "agent_tests": "not-run", "counts": { "pass": 9, "fail": 0, "skip": 0, "not_applicable": 1 }, "meaning": "Level 1: automated checks on the files (parse, install paths, MCP config, safety scan, references, size, licence, starter build). No AI agent was run.", "checks": [ { "id": "listing", "name": "Listing", "status": "pass", "summary": "Schema valid; 8 files declared, none missing or extra", "details": [ "tested.status: untested (agent tests are separate from Level 1)" ] }, { "id": "parse", "name": "Files parse", "status": "pass", "summary": "All 8 files parse; front matter valid", "details": [] }, { "id": "adapters", "name": "Adapters", "status": "pass", "summary": "CLAUDE.md and GEMINI.md import AGENTS.md and are up to date", "details": [ "CLAUDE.md: matches the generator", "GEMINI.md: matches the generator" ] }, { "id": "install-paths", "name": "Install paths", "status": "pass", "summary": "Correct paths for all 6 agents; layouts re-discovered", "details": [ "Claude Code: 7 files, 1 skill in .claude/skills/", "Codex CLI: 6 files, 1 skill in .agents/skills/", "Cursor: 6 files, 1 skill in .agents/skills/", "Gemini CLI: 7 files, 1 skill in .agents/skills/", "Antigravity CLI: 6 files, 1 skill in .agents/skills/", "GitHub Copilot: 6 files, 1 skill in .agents/skills/" ] }, { "id": "mcp", "name": "MCP config", "status": "n/a", "summary": "No MCP servers in this loadout", "details": [] }, { "id": "safety", "name": "Safety scan", "status": "pass", "summary": "Clean: no credentials, risky shell, hidden text or covert instructions", "details": [ "Scanner intake-precheck/2: verdict clean; network no (listing says no)" ] }, { "id": "references", "name": "References", "status": "pass", "summary": "4 references between files resolve for every agent", "details": [] }, { "id": "size", "name": "Size limits", "status": "pass", "summary": "8 files, 13.5 KB total; largest skills/done-proof/scripts/test_tamper.py 6.8 KB", "details": [ "Limits: 30 files, 32 KB per file, 128 KB total, 500 lines per skill or rule" ] }, { "id": "licence", "name": "Licence", "status": "pass", "summary": "MIT, by SetupBay; text in the repository LICENSE", "details": [] }, { "id": "starter-build", "name": "Starter build", "status": "pass", "summary": "In a real git repo, the done-proof tamper check passes a clean fix with a new test, then catches a changed expected value, a removed assertion, an added pytest skip, a deleted test, a Vitest it.skip and a changed snapshot; an allow-list entry clears an approved file; the Stop hook exits 2 on tamperi", "details": [ "Applied 13 loadout files at every agent's paths", "✓ set up a repo: bash -c cp -r project repo && cd repo && git init -q && git config user.email fixture@example.com && git config user.name fixture && git add -A && git commit -qm base && echo ready", "✓ a real fix with a new test passes: bash -c cd repo && printf '\\n\\ndef test_two_items_half_off():\\n assert total([10, 10], 0.5) == 10.0\\n' >> tests/test_price.py && python3 ../.agents/skills/done-proof/scripts/test_tamper.py; echo \"exit=$?\"; git checkout -q -- .", "✓ weakened pytest file is caught: bash -c cd repo && printf 'import pytest\\nfrom src.price import total\\n\\n\\ndef test_total():\\n assert total([1, 2]) == 4\\n\\n\\n@pytest.mark.skip(reason=\"flaky\")\\ndef test_discount():\\n assert total([10], 0.1) == 9.0\\n' > tests/test_price.py && python3 ../.agents/skills/done-proof/scripts/test_tamper.py; echo \"exit=$?\"", "✓ Vitest it.skip and a snapshot change are caught: bash -c cd repo && sed -i 's/ it(\"counts lines\"/ it.skip(\"counts lines\"/' web/cart.test.js && mkdir -p web/__snapshots__ && echo 'exports[`cart 1`] = `2`;' > web/__snapshots__/cart.test.js.snap && git add -A web/__snapshots__ && git commit -qm snap -- web/__snapshots__ && echo 'exports[`cart 1`] = `3`;' > web/__snapshots__/cart.test.js.snap && python3 ../.agents/skills/done-proof/scripts/test_tamper.py --base HEAD; echo \"exit=$?\"", "✓ allow list clears only the approved file: bash -c cd repo && printf 'tests/test_price.py\\n' > .done-proof-allow && python3 ../.agents/skills/done-proof/scripts/test_tamper.py; echo \"exit=$?\"", "✓ Stop hook blocks with exit 2, and steps aside on stop_hook_active: bash -c cd repo && rm .done-proof-allow && export CLAUDE_PROJECT_DIR=$PWD && mkdir -p .agents && cp -r ../.agents/skills .agents/ && echo '{\"stop_hook_active\": false}' | bash ../.agents/skills/done-proof/scripts/done-gate.sh; echo \"exit=$?\"; echo '{\"stop_hook_active\": true}' | bash ../.agents/skills/done-proof/scripts/done-gate.sh; echo \"active-exit=$?\"", "✓ the settings snippet is valid JSON with a Stop hook: python3 -c import json; d=json.load(open('.agents/skills/done-proof/references/stop-hook.example.json')); h=d['hooks']['Stop'][0]['hooks'][0]; print('stop hook:', h['args'][0].rsplit('/',1)[-1])", "Covers: Each tamper pattern on real git diffs: changed expected value, removed assertion, pytest skip, deleted test, Vitest it.skip, changed snapshot", "Covers: New tests and assertions are counted and never blocked", "Covers: The human-only allow list clears approved files and nothing else", "Covers: The Stop hook's exit 2 on findings and its stop_hook_active loop guard", "Covers: Not covered: whether an agent follows the rules (agent tests are separate)" ] } ] }, "install": { "claude-code": { "files": [ { "path": "AGENTS.md", "target": "AGENTS.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/AGENTS.md", "role": "instructions" }, { "path": "CLAUDE.md", "target": "CLAUDE.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/CLAUDE.md", "role": "adapter" }, { "path": "skills/done-proof/SKILL.md", "target": ".claude/skills/done-proof/SKILL.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/SKILL.md", "role": "skill" }, { "path": "skills/done-proof/scripts/test_tamper.py", "target": ".claude/skills/done-proof/scripts/test_tamper.py", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/scripts/test_tamper.py", "role": "reference" }, { "path": "skills/done-proof/scripts/done-gate.sh", "target": ".claude/skills/done-proof/scripts/done-gate.sh", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/scripts/done-gate.sh", "role": "reference" }, { "path": "skills/done-proof/references/stop-hook.example.json", "target": ".claude/skills/done-proof/references/stop-hook.example.json", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/references/stop-hook.example.json", "role": "reference" }, { "path": "skills/done-proof/references/done-report.md", "target": ".claude/skills/done-proof/references/done-report.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/references/done-report.md", "role": "reference" } ], "manual_steps": [], "zip_url": "https://setupbay.com/download/definition-of-done-proof/claude-code.zip" }, "codex": { "files": [ { "path": "AGENTS.md", "target": "AGENTS.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/AGENTS.md", "role": "instructions" }, { "path": "skills/done-proof/SKILL.md", "target": ".agents/skills/done-proof/SKILL.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/SKILL.md", "role": "skill" }, { "path": "skills/done-proof/scripts/test_tamper.py", "target": ".agents/skills/done-proof/scripts/test_tamper.py", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/scripts/test_tamper.py", "role": "reference" }, { "path": "skills/done-proof/scripts/done-gate.sh", "target": ".agents/skills/done-proof/scripts/done-gate.sh", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/scripts/done-gate.sh", "role": "reference" }, { "path": "skills/done-proof/references/stop-hook.example.json", "target": ".agents/skills/done-proof/references/stop-hook.example.json", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/references/stop-hook.example.json", "role": "reference" }, { "path": "skills/done-proof/references/done-report.md", "target": ".agents/skills/done-proof/references/done-report.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/references/done-report.md", "role": "reference" } ], "manual_steps": [], "zip_url": "https://setupbay.com/download/definition-of-done-proof/codex.zip" }, "cursor": { "files": [ { "path": "AGENTS.md", "target": "AGENTS.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/AGENTS.md", "role": "instructions" }, { "path": "skills/done-proof/SKILL.md", "target": ".agents/skills/done-proof/SKILL.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/SKILL.md", "role": "skill" }, { "path": "skills/done-proof/scripts/test_tamper.py", "target": ".agents/skills/done-proof/scripts/test_tamper.py", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/scripts/test_tamper.py", "role": "reference" }, { "path": "skills/done-proof/scripts/done-gate.sh", "target": ".agents/skills/done-proof/scripts/done-gate.sh", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/scripts/done-gate.sh", "role": "reference" }, { "path": "skills/done-proof/references/stop-hook.example.json", "target": ".agents/skills/done-proof/references/stop-hook.example.json", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/references/stop-hook.example.json", "role": "reference" }, { "path": "skills/done-proof/references/done-report.md", "target": ".agents/skills/done-proof/references/done-report.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/references/done-report.md", "role": "reference" } ], "manual_steps": [], "zip_url": "https://setupbay.com/download/definition-of-done-proof/cursor.zip" }, "gemini-cli": { "files": [ { "path": "AGENTS.md", "target": "AGENTS.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/AGENTS.md", "role": "instructions" }, { "path": "GEMINI.md", "target": "GEMINI.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/GEMINI.md", "role": "adapter" }, { "path": "skills/done-proof/SKILL.md", "target": ".agents/skills/done-proof/SKILL.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/SKILL.md", "role": "skill" }, { "path": "skills/done-proof/scripts/test_tamper.py", "target": ".agents/skills/done-proof/scripts/test_tamper.py", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/scripts/test_tamper.py", "role": "reference" }, { "path": "skills/done-proof/scripts/done-gate.sh", "target": ".agents/skills/done-proof/scripts/done-gate.sh", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/scripts/done-gate.sh", "role": "reference" }, { "path": "skills/done-proof/references/stop-hook.example.json", "target": ".agents/skills/done-proof/references/stop-hook.example.json", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/references/stop-hook.example.json", "role": "reference" }, { "path": "skills/done-proof/references/done-report.md", "target": ".agents/skills/done-proof/references/done-report.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/references/done-report.md", "role": "reference" } ], "manual_steps": [], "zip_url": "https://setupbay.com/download/definition-of-done-proof/gemini-cli.zip" }, "antigravity-cli": { "files": [ { "path": "AGENTS.md", "target": "AGENTS.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/AGENTS.md", "role": "instructions" }, { "path": "skills/done-proof/SKILL.md", "target": ".agents/skills/done-proof/SKILL.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/SKILL.md", "role": "skill" }, { "path": "skills/done-proof/scripts/test_tamper.py", "target": ".agents/skills/done-proof/scripts/test_tamper.py", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/scripts/test_tamper.py", "role": "reference" }, { "path": "skills/done-proof/scripts/done-gate.sh", "target": ".agents/skills/done-proof/scripts/done-gate.sh", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/scripts/done-gate.sh", "role": "reference" }, { "path": "skills/done-proof/references/stop-hook.example.json", "target": ".agents/skills/done-proof/references/stop-hook.example.json", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/references/stop-hook.example.json", "role": "reference" }, { "path": "skills/done-proof/references/done-report.md", "target": ".agents/skills/done-proof/references/done-report.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/references/done-report.md", "role": "reference" } ], "manual_steps": [], "zip_url": "https://setupbay.com/download/definition-of-done-proof/antigravity-cli.zip" }, "copilot": { "files": [ { "path": "AGENTS.md", "target": "AGENTS.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/AGENTS.md", "role": "instructions" }, { "path": "skills/done-proof/SKILL.md", "target": ".agents/skills/done-proof/SKILL.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/SKILL.md", "role": "skill" }, { "path": "skills/done-proof/scripts/test_tamper.py", "target": ".agents/skills/done-proof/scripts/test_tamper.py", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/scripts/test_tamper.py", "role": "reference" }, { "path": "skills/done-proof/scripts/done-gate.sh", "target": ".agents/skills/done-proof/scripts/done-gate.sh", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/scripts/done-gate.sh", "role": "reference" }, { "path": "skills/done-proof/references/stop-hook.example.json", "target": ".agents/skills/done-proof/references/stop-hook.example.json", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/references/stop-hook.example.json", "role": "reference" }, { "path": "skills/done-proof/references/done-report.md", "target": ".agents/skills/done-proof/references/done-report.md", "raw_url": "https://setupbay.com/raw/definition-of-done-proof/1.0.0/skills/done-proof/references/done-report.md", "role": "reference" } ], "manual_steps": [], "zip_url": "https://setupbay.com/download/definition-of-done-proof/copilot.zip" } }, "source_zip_url": "https://setupbay.com/download/definition-of-done-proof/source.zip" }