Unity tests from the command line, read properly

# AGENTS.md: Unity project with command-line tests

Unity 6 project. C# changes are checked by the Unity Test Framework, run headless. The `unity-tests` skill runs
the tests and reads the results; use it after every C# change.

## Code layout for testable C#
- Game logic in plain C# classes (no `MonoBehaviour`) in the runtime assembly, so Edit Mode tests can run it in
  milliseconds. MonoBehaviours read input, call the logic and update the scene.
- Tests in `Assets/Tests/EditMode/` and `Assets/Tests/PlayMode/`, each with its own `.asmdef`. One test class per
  class under test, named `<Class>Tests`.
- Never edit `Library/`, `Temp/`, `Logs/`, `obj/` or `UserSettings/`. Never delete or regenerate `.meta` files.

## Running tests
- The Editor binary is in `UNITY_PATH`, matching `ProjectSettings/ProjectVersion.txt`. Never run a different version.
- The project must be closed in the Editor for a command-line run. If it is open, ask the human.
- Unity exits 0 when no tests ran. A run is only green when at least one test ran and none failed.

## Done means
Edit Mode tests ran (and Play Mode tests, if you changed behaviour that needs frames), the summary shows 0 failed
and more than 0 total, no test was ignored or weakened, and you reported the command and the summary line.

# CLAUDE.md

@AGENTS.md

## Claude Code notes

- Skills from this loadout live in `.claude/skills/`: `unity-tests`. Use them when their description matches the task.
- Follow AGENTS.md above; where this file and AGENTS.md disagree, AGENTS.md wins.

<!-- From SetupBay "Unity tests from the command line, read properly" v1.0.0 — https://setupbay.com/loadouts/unity-test-runner/ -->

# GEMINI.md

@./AGENTS.md

## Gemini CLI notes

- Skills from this loadout live in `.agents/skills/`: `unity-tests`. Use them when their description matches the task.
- Follow AGENTS.md above; where this file and AGENTS.md disagree, AGENTS.md wins.

<!-- From SetupBay "Unity tests from the command line, read properly" v1.0.0 — https://setupbay.com/loadouts/unity-test-runner/ -->

---
name: unity-tests
description: Run Unity Test Framework Edit Mode and Play Mode tests from the command line and read the NUnit results, including the zero-tests case Unity reports as success. Use after changing C# in a Unity project, when adding tests, or when the user asks whether the Unity tests pass.
---

# Unity tests from the command line

The Unity agent plugin can edit scripts and scenes; this skill is how you prove the C# still works. It runs the
Unity Test Framework headless and reads the result file, because the exit code alone is misleading:

| Exit | Meaning |
|---|---|
| 0 | every test passed **or no tests ran at all** (a filter that matches nothing, a missing test assembly) |
| 2 | one or more tests failed |
| 3 | run error: usually a compile error, so no tests ran |
| 4 | the test platform isn't installed |

## 1. Preconditions
- `ProjectSettings/ProjectVersion.txt` gives the Editor version. `UNITY_PATH` must point at that Editor's binary.
  If it isn't set, ask the human for the path; do not guess another version (it would re-import the project).
- **The project must not be open in the Editor.** Batch mode refuses a project that is already open. If the human
  has it open, ask them to close it, or to run the tests in Window → General → Test Runner and paste the result.
- `git status` clean, or the human knows you are about to run a re-import.

## 2. Edit Mode first
Edit Mode tests run plain C# without entering Play mode, in seconds. Put logic in plain classes so it can be tested
here, and keep MonoBehaviours thin.
- Test assembly: `Assets/Tests/EditMode/` with an `.asmdef` like `references/EditModeTests.asmdef.json`
  (Editor-only, references `UnityEngine.TestRunner` and `UnityEditor.TestRunner`, `nunit.framework.dll`, and the
  runtime assembly under test, constrained to `UNITY_INCLUDE_TESTS`). Save it with the `.asmdef` extension.
- Example test: `references/HealthTests.cs`. NUnit `[Test]` and `[TestCase]`; `Assert.That(x, Is.EqualTo(y))`.
- Play Mode tests (`[UnityTest]` coroutines, scenes, physics) go in `Assets/Tests/PlayMode/` with a separate
  `.asmdef` that is not Editor-only. Use them for behaviour that needs frames.

## 3. Run
```bash
python3 .agents/skills/unity-tests/scripts/unity_tests.py run --platform EditMode
python3 .agents/skills/unity-tests/scripts/unity_tests.py run --platform EditMode --filter "Game.Tests.HealthTests"
python3 .agents/skills/unity-tests/scripts/unity_tests.py run --platform PlayMode
```
(Claude Code installs the skill under `.claude/skills/`.) It writes `TestResults/<platform>.xml` and `.log`
(add `TestResults/` to `.gitignore`), deletes old ones first so a stale file is never read, prints the Unity
command, each failing test with its message and first stack line, compile errors from the log, and skipped tests
with their reasons. Already have a results file? `unity_tests.py read TestResults/editmode.xml`.

## 4. Read it like a reviewer
- `NO TESTS RAN` is a failure, whatever Unity's exit code says. Fix the filter, the `.asmdef` or the platform.
- `COMPILE` lines: fix those first; nothing else ran.
- Never mark a test `[Ignore]`, delete it, or loosen an assertion to get green. If a test is wrong, say why and ask.
- Report the summary line (total, passed, failed, skipped) and the command you ran.

#!/usr/bin/env python3
"""Run Unity Test Framework tests from the command line and read the NUnit results properly.

Usage:
  python3 unity_tests.py run [--platform EditMode|PlayMode] [--filter REGEX] [--category NAME] [--project .]
  python3 unity_tests.py read RESULTS.xml

`run` needs the Unity Editor binary in UNITY_PATH (for example
/Applications/Unity/Hub/Editor/6000.2.6f1/Unity.app/Contents/MacOS/Unity). The Editor must not have the project open.
It runs: Unity -batchmode -projectPath P -runTests -testPlatform X -testResults F -logFile L [-testFilter ...]
and leaves out -quit (the test run exits the Editor when it ends).

Why not trust the exit code alone: Unity exits 0 when every test passed and ALSO when no tests ran at all
(for example a -testFilter that matches nothing). 2 means tests failed, 3 a run error (often a compile error),
4 a platform that is not installed. This script fails on zero tests, lists each failure with its message,
and pulls compile errors out of the log.
Exit 0 only when at least one test ran and none failed. Standard library only.
"""
import os
import re
import subprocess
import sys
import xml.etree.ElementTree as ET

CODES = {0: "run completed", 2: "one or more tests failed", 3: "run error (compile error, crash or no test callbacks)", 4: "test platform not found"}


def read(path):
    if not os.path.exists(path):
        print(f"unity-tests: no results file at {path}; the run did not get as far as tests")
        return 1
    root = ET.parse(path).getroot()
    run = root if root.tag == "test-run" else root.find(".//test-run")
    if run is None:
        print(f"unity-tests: {path} is not an NUnit 3 test-run file")
        return 1
    a = {k: run.get(k, "0") for k in ("total", "passed", "failed", "skipped", "inconclusive", "result", "duration")}
    total, failed = int(a["total"]), int(a["failed"])
    print(f"unity-tests: {a['total']} total, {a['passed']} passed, {a['failed']} failed, {a['skipped']} skipped, {a['inconclusive']} inconclusive ({a['result']})")
    for case in run.iter("test-case"):
        if case.get("result") == "Failed":
            msg = (case.findtext("failure/message") or "").strip().splitlines()
            where = (case.findtext("failure/stack-trace") or "").strip().splitlines()
            print(f"FAILED {case.get('fullname')}: {msg[0] if msg else '(no message)'}")
            if where:
                print(f"   at {where[0].strip()}")
    for case in run.iter("test-case"):
        if case.get("result") == "Skipped":
            reason = (case.findtext("reason/message") or "").strip()
            print(f"skipped {case.get('fullname')}{': ' + reason if reason else ''}")
    if total == 0:
        print("unity-tests: NO TESTS RAN. Unity still exits 0 here. Check -testFilter/-testCategory, the test assembly's .asmdef, and -testPlatform.")
        return 1
    return 1 if failed else 0


def compile_errors(log):
    if not os.path.exists(log):
        return []
    text = open(log, encoding="utf-8", errors="replace").read()
    errs = re.findall(r"^(Assets/[^\n(]+\(\d+,\d+\): error CS\d+: [^\n]+)", text, re.M)
    if "another Unity instance is running" in text or "It looks like another Unity instance" in text:
        errs.insert(0, "The project is open in another Unity Editor. Ask the human to close it, or run the tests in the Test Runner window.")
    return list(dict.fromkeys(errs))


def run(argv):
    unity = os.environ.get("UNITY_PATH")
    if not unity:
        print("unity-tests: set UNITY_PATH to the Unity Editor binary for this project's version (ProjectSettings/ProjectVersion.txt)")
        return 2
    opt = lambda n, d=None: argv[argv.index(n) + 1] if n in argv else d
    project = os.path.abspath(opt("--project", "."))
    platform = opt("--platform", "EditMode")
    out = os.path.join(project, "TestResults")
    os.makedirs(out, exist_ok=True)
    results, log = os.path.join(out, f"{platform.lower()}.xml"), os.path.join(out, f"{platform.lower()}.log")
    for p in (results, log):
        if os.path.exists(p):
            os.remove(p)  # never read a stale file from an earlier run
    cmd = [unity, "-batchmode", "-projectPath", project, "-runTests", "-testPlatform", platform, "-testResults", results, "-logFile", log]
    if platform == "EditMode":
        cmd.insert(2, "-nographics")
    if opt("--filter"):
        cmd += ["-testFilter", opt("--filter")]
    if opt("--category"):
        cmd += ["-testCategory", opt("--category")]
    print("$ " + " ".join(cmd))
    code = subprocess.run(cmd).returncode
    print(f"unity exit {code}: {CODES.get(code, 'unknown')}")
    for e in compile_errors(log):
        print(f"COMPILE {e}")
    status = read(results)
    return status if code in (0, 2) else (code or 1)


def main(argv):
    if argv[:1] == ["read"] and len(argv) == 2:
        return read(argv[1])
    if argv[:1] == ["run"]:
        return run(argv[1:])
    print(__doc__.strip(), file=sys.stderr)
    return 2


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

{
  "name": "Game.Tests.EditMode",
  "rootNamespace": "Game.Tests",
  "references": [
    "Game.Runtime",
    "UnityEngine.TestRunner",
    "UnityEditor.TestRunner"
  ],
  "includePlatforms": ["Editor"],
  "excludePlatforms": [],
  "allowUnsafeCode": false,
  "overrideReferences": true,
  "precompiledReferences": ["nunit.framework.dll"],
  "autoReferenced": false,
  "defineConstraints": ["UNITY_INCLUDE_TESTS"],
  "versionDefines": [],
  "noEngineReferences": false
}

// Example Edit Mode test. Copy to Assets/Tests/EditMode/HealthTests.cs next to the test .asmdef.
// Tests plain C# logic (no scene, no frames), so it runs in milliseconds from the command line.
using NUnit.Framework;

namespace Game.Tests
{
    public class HealthTests
    {
        [Test]
        public void Damage_ReducesHealth_AndNeverGoesBelowZero()
        {
            var health = new Health(max: 10);
            health.TakeDamage(4);
            Assert.That(health.Current, Is.EqualTo(6));
            health.TakeDamage(50);
            Assert.That(health.Current, Is.EqualTo(0));
            Assert.That(health.IsDead, Is.True);
        }

        [TestCase(0)]
        [TestCase(-3)]
        public void Damage_IgnoresZeroOrNegative(int amount)
        {
            var health = new Health(max: 10);
            health.TakeDamage(amount);
            Assert.That(health.Current, Is.EqualTo(10));
        }
    }
}