Definition of done for agents, with proof

# AGENTS.md: definition of done for coding agents

"Done" is a claim the human will rely on. Make it only with evidence.

## Done means all of these
1. **It works.** You ran the behaviour that was asked for (command, request, page) and saw the right result.
2. **A test shows it.** New behaviour has a new test. A bug fix has a test that failed before the fix and passes now.
3. **No test was weakened.** You did not change an existing assertion's expected value, delete an assertion or test,
   add skip/xfail/only/ignore markers, or accept new snapshots unread. The `done-proof` skill's tamper check
   prints `0 finding(s) to explain`.
4. **The full suite passes**, with the summary line in your report (passed / failed / skipped).
5. **You said what you could not check.**

## When a test seems wrong
Stop. Name the test, what it expects, and why you think that is wrong. Change it only after the human agrees.
Approved test files are listed in `.done-proof-allow` by the human, never by you.

## Reporting
- End with the done report from the `done-proof` skill: what was asked, how you saw it work, tests (command and
  summary, new tests, changed tests), what was not checked, what needs the human.
- Never write "should work", "probably" or "done" without the evidence next to it.

# CLAUDE.md

@AGENTS.md

## Claude Code notes

- Skills from this loadout live in `.claude/skills/`: `done-proof`. Use them when their description matches the task.
- Follow AGENTS.md above; where this file and AGENTS.md disagree, AGENTS.md wins.

<!-- From SetupBay "Definition of done for agents, with proof" v1.0.0 — https://setupbay.com/loadouts/definition-of-done-proof/ -->

# GEMINI.md

@./AGENTS.md

## Gemini CLI notes

- Skills from this loadout live in `.agents/skills/`: `done-proof`. Use them when their description matches the task.
- Follow AGENTS.md above; where this file and AGENTS.md disagree, AGENTS.md wins.

<!-- From SetupBay "Definition of done for agents, with proof" v1.0.0 — https://setupbay.com/loadouts/definition-of-done-proof/ -->

---
name: done-proof
description: Prove a coding task is done before saying so, with evidence the feature works and proof the tests were not weakened. Use before reporting any task as done, fixed or complete, when tests fail and it is tempting to change them, or when the user asks for proof.
---

# Done means proven

"Tests pass" was written for people, who also clicked through the feature. Agents read it literally, and changing an
assertion is cheaper than fixing a bug. This skill makes "done" mean three things: the behaviour works, a test shows
it, and no existing test was loosened to get there.

## 1. Before you change code
- Write down what done looks like for this task in one or two lines: the user-visible result, and the check that
  shows it (a command, a page, an output).
- For a bug: write or find a test that fails because of the bug. Run it and keep the failure output. That failure is
  half of your proof.

## 2. While you work
- Never edit an existing assertion's expected value, delete an assertion or a test, or add `skip`, `xfail`, `.only`,
  `@Disabled`, `t.Skip` or `#[ignore]` to get to green. Fix the code.
- If you believe a test is wrong (the spec changed, the test was buggy), stop and say so: which test, what it expects,
  why that is wrong. Change it only after the human agrees, and they add the file to `.done-proof-allow`.
- Never accept updated snapshots without reading the diff and saying why the new output is right.

## 3. Check the tests were not weakened
```bash
python3 .agents/skills/done-proof/scripts/test_tamper.py            # uncommitted changes
python3 .agents/skills/done-proof/scripts/test_tamper.py --base main  # the whole branch
```
(Claude Code installs the skill under `.claude/skills/`; use that path there.)
It must print `0 finding(s) to explain`. It reports changed expectations, removed assertions, added skip markers,
deleted tests and changed snapshots in test files. New tests and new assertions are counted, never blocked.

## 4. Show it works
- Run the full test command and keep the summary line (passed, failed, skipped).
- Run the feature itself: the CLI with real input, the endpoint with a real request, the page in a browser (screenshot or
  the visible text). Unit tests alone do not show a feature works end to end.
- Re-run the failing test from step 1: it passes now.

## 5. Report with evidence
Use `references/done-report.md`. Every claim has the command or observation behind it. Put what you could not check
under "Not checked". Never write "should work".

## Optional: make it a wall (Claude Code)
A rule asks; a hook enforces. `scripts/done-gate.sh` is a Stop hook that runs the tamper check and refuses to let
the session end while it has findings (exit 2, with `stop_hook_active` respected so it cannot loop). To install it,
merge `references/stop-hook.example.json` into `.claude/settings.json` (ask the human first; it changes their settings).

#!/usr/bin/env python3
"""Find changes that make tests pass without fixing anything: weakened, skipped or deleted tests.

Usage: python3 test_tamper.py [--base REF] [--allow FILE]
  --base   compare the working tree (committed + uncommitted) with REF (default: HEAD; use main or origin/main for a branch)
  --allow  file listing test paths a human approved changing, one per line (default: .done-proof-allow if present)

Looks only at test files (tests/, test/, __tests__/, spec/, *_test.*, *.test.*, *.spec.*, test_*.py, *Tests.swift, *Test.java).
Reports, per file:
  removed-assert   a line with an assertion was deleted or changed
  changed-expect   an existing assertion now expects a different value
  added-skip       a skip / xfail / only / ignore marker was added
  deleted-test     a test function or case was removed
  snapshot         a snapshot file was changed
New tests and new assertions are fine and are only counted.
Exit 1 if anything is found outside the allow list, else 0. Read-only; needs git.
"""
import os
import re
import subprocess
import sys

TEST_PATH = re.compile(r"(^|/)(tests?|__tests__|spec|specs)/|(_test|\.test|\.spec|_spec)\.[a-z]+$|(^|/)test_[^/]*\.py$|Tests?\.(swift|java|kt|cs)$|_test\.go$")
SNAPSHOT = re.compile(r"(\.snap$|__snapshots__/|\.ambr$|snapshots?/)")
ASSERT = re.compile(r"\b(assert\w*|expect\s*\(|should\b|\.to(Be|Equal|Have|Match|Throw|Contain)\w*|XCTAssert\w*|#expect|#require|require\.\w+|t\.(Error|Fatal|Fail)\w*|assert_eq!|assert_ne!|assert!|refute\w*)", re.I)
SKIP = re.compile(r"(@pytest\.mark\.(skip|xfail)|pytest\.skip\(|@unittest\.skip|\b(it|test|describe|context)\.(skip|only|todo)\b|\bx(it|describe|test)\s*\(|\bf(it|describe)\s*\(|@Disabled\b|@Ignore\b|\bt\.Skip(Now|f)?\(|#\[ignore\]|\.disabled\(|skip\s*:\s*true|\bpending\s*\()")
TEST_DEF = re.compile(r"^\s*(def test_\w+|async def test_\w+|func Test\w+|fn test_\w+|#\[test\]|@Test\b|(it|test)\s*\(\s*['\"`])")


def git(*args):
    r = subprocess.run(["git", *args], capture_output=True, text=True)
    if r.returncode != 0:
        sys.exit(f"git {' '.join(args)} failed: {r.stderr.strip()}")
    return r.stdout


def expected_value(line):
    """The value side of an assertion, roughly: what comes after ==, toBe(, toEqual(, assertEqual(x, ..."""
    m = re.search(r"(==|!=|\bis\b|\.to\w*\(|assert\w*\([^,]+,)\s*(.+)$", line)
    return m.group(2).strip().rstrip(")").strip() if m else None


def subject(line):
    m = re.search(r"(assert\w*\s*\(?|expect\s*\()\s*([^=,)]+)", line)
    return re.sub(r"\s+", "", m.group(2)) if m else None


def main(argv):
    base = "HEAD"
    allow_file = ".done-proof-allow"
    if "--base" in argv:
        base = argv[argv.index("--base") + 1]
    if "--allow" in argv:
        allow_file = argv[argv.index("--allow") + 1]
    allowed = set()
    if os.path.exists(allow_file):
        allowed = {l.strip() for l in open(allow_file) if l.strip() and not l.startswith("#")}
    git("rev-parse", "--verify", base)
    # Untracked test files count as new tests: add them to the diff with intent-to-add would change the index, so list them.
    untracked = [p for p in git("ls-files", "--others", "--exclude-standard").splitlines() if TEST_PATH.search(p)]
    diff = git("diff", "--no-color", "--unified=0", "--find-renames", base, "--")
    findings, new_tests, new_asserts = [], 0, 0
    path, removed, added = None, [], []

    def flush():
        nonlocal new_tests, new_asserts
        if not path or not TEST_PATH.search(path) and not SNAPSHOT.search(path):
            return
        if SNAPSHOT.search(path):
            if removed or added:
                findings.append((path, "snapshot", "snapshot changed; confirm the new output is right, not just accepted"))
            return
        rem_asserts = [l for l in removed if ASSERT.search(l)]
        add_asserts = [l for l in added if ASSERT.search(l)]
        add_by_subject = {}
        for l in add_asserts:
            add_by_subject.setdefault(subject(l), []).append(l)
        matched = set()
        for l in rem_asserts:
            s = subject(l)
            twins = add_by_subject.get(s) or []
            if twins:
                t = twins.pop(0)
                matched.add(id(t))
                if expected_value(l) != expected_value(t):
                    findings.append((path, "changed-expect", f"{l.strip()[:90]}  ->  {t.strip()[:90]}"))
            else:
                findings.append((path, "removed-assert", l.strip()[:120]))
        for l in added:
            if SKIP.search(l):
                findings.append((path, "added-skip", l.strip()[:120]))
        rem_defs = [l for l in removed if TEST_DEF.search(l)]
        norm = lambda l: re.sub(r"\.(skip|only|todo)\b|^x(?=(it|test|describe)\b)", "", l.strip())
        add_defs = {l.strip() for l in added if TEST_DEF.search(l) or SKIP.search(l)}
        add_norm = {norm(l) for l in add_defs}
        for l in rem_defs:
            if norm(l) not in add_norm:  # a test that was only marked skip is reported once, as added-skip
                findings.append((path, "deleted-test", l.strip()[:120]))
        new_tests += max(0, len([l for l in add_defs if TEST_DEF.search(l)]) - len(rem_defs))
        new_asserts += len([l for l in add_asserts if id(l) not in matched])

    for line in diff.splitlines():
        if line.startswith("diff --git"):
            flush()
            path, removed, added = None, [], []
        elif line.startswith("+++ "):
            p = line[4:]
            path = p[2:] if p.startswith("b/") else (None if p == "/dev/null" else p)
        elif line.startswith("--- ") and path is None:
            p = line[4:]
            if p != "/dev/null":
                path = p[2:] if p.startswith("a/") else p
        elif line.startswith("-") and not line.startswith("---"):
            removed.append(line[1:])
        elif line.startswith("+") and not line.startswith("+++"):
            added.append(line[1:])
    flush()
    for p in untracked:
        text = open(p, encoding="utf-8", errors="replace").read().splitlines()
        new_tests += sum(1 for l in text if TEST_DEF.search(l))
        new_asserts += sum(1 for l in text if ASSERT.search(l))
        for l in text:
            if SKIP.search(l):
                findings.append((p, "added-skip", l.strip()[:120]))

    blocking = [f for f in findings if f[0] not in allowed]
    for p, kind, msg in findings:
        note = " (approved in allow list)" if p in allowed else ""
        print(f"{p}: {kind}: {msg}{note}")
    print(f"test-tamper: base {base}, {len(blocking)} finding(s) to explain, {len(findings) - len(blocking)} approved, {new_tests} new test(s), {new_asserts} new assertion(s)")
    if blocking:
        print("Not done: fix the code, not the test. If a test was wrong, say why and ask the human to add the file to .done-proof-allow.", file=sys.stderr)
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

#!/usr/bin/env bash
# Claude Code Stop hook: refuse "done" while test files were weakened, skipped or deleted.
# Exit 2 keeps Claude working and shows it the findings. Respects stop_hook_active so it never loops.
# Base ref: DONE_PROOF_BASE (default HEAD, i.e. this session's uncommitted changes).
set -u
input=$(cat)
printf '%s' "$input" | python3 -c 'import json,sys; sys.exit(0 if json.load(sys.stdin).get("stop_hook_active") else 1)' 2>/dev/null && exit 0
cd "${CLAUDE_PROJECT_DIR:-.}" || exit 0
git rev-parse --is-inside-work-tree >/dev/null 2>&1 || exit 0
script="${CLAUDE_PROJECT_DIR:-.}/.claude/skills/done-proof/scripts/test_tamper.py"
[ -f "$script" ] || script=".agents/skills/done-proof/scripts/test_tamper.py"
[ -f "$script" ] || exit 0
if ! out=$(python3 "$script" --base "${DONE_PROOF_BASE:-HEAD}" 2>&1); then
  printf 'You changed tests to make them pass. You are not done:\n%s\n' "$out" >&2
  exit 2
fi
exit 0

{
  "hooks": {
    "Stop": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "bash",
            "args": ["${CLAUDE_PROJECT_DIR}/.claude/skills/done-proof/scripts/done-gate.sh"],
            "timeout": 60,
            "statusMessage": "Checking tests were not weakened"
          }
        ]
      }
    ]
  }
}

# Done report (copy into your final message)

**Asked for:** <the user's request in one line>

**Works:** <how you saw it work: the command and its output, the URL and what the page showed, or the screenshot path>

**Tests:**
- Command: `<exact test command>` → <N passed, 0 failed, 0 skipped>
- New tests: <names>, each failed before the fix (`<command>` showed <failure>) and passes now
- Existing tests changed: none / <file: what and why, approved by the human>
- test-tamper: `0 finding(s) to explain`

**Not checked:** <anything you could not run or see, and why>

**Needs you:** <decisions or manual checks, or "nothing">