Polars 2 guard against Polars 1 code

# AGENTS.md: Polars 2 project (Polars 1 guard)

This project uses **Polars 2** (`polars>=2.1,<3`). Most Polars code you have seen is Polars 0.19–1.x. Polars 2
removed many of those names, and changed some results without raising, so treat remembered Polars as suspect.

## Version facts
- Check the pinned version first: `uv pip show polars` or the `polars` line in `pyproject.toml`/`uv.lock`.
- Lazy queries default to the **streaming engine** in 2.0. It does not keep row order for joins, `group_by` or
  `unpivot` unless you ask (`maintain_order=...`) or sort.
- Removed APIs raise `AttributeRemovedError` or `ArgumentRemovedError` with a hint. Read the hint; don't guess.
- If you are unsure an API exists in Polars 2, look it up in the API reference (https://docs.pola.rs/api/python/stable/)
  or the upgrade guide (https://docs.pola.rs/releases/upgrade/2/). Never copy Polars 1 answers or pandas idioms.

## Rules
- Write Polars 2 only. The `polars-2-guard` skill has the full table; the common traps are:
  `unpivot` (not `melt`), `with_row_index` (not `with_row_count`), `group_by(...).len()` (not `.count()`),
  `how="full"` (not `"outer"`), `nulls_equal=` (not `join_nulls=`), `str.join` (not `str.concat`),
  `engine="streaming"` (not `streaming=True`), `min_samples=` (not `min_periods=`), `str.to_date()` (not `.cast(pl.Date)` on strings).
- Prefer expressions over `map_elements`. Never use pandas methods on a Polars frame.
- If row order matters, say so in code: `.sort(...)` or `maintain_order="left"` on joins.
- Don't upgrade or downgrade Polars, and don't set `POLARS_ENGINE_AFFINITY`, without asking the human.

## Before you say "done"
1. Run the `polars-2-guard` scanner on the files you touched; it must report `0 error(s)`. Read every `warn`.
2. Run the code or its tests against the pinned Polars (`uv run pytest`, `uv run python <script>.py`).
3. If you changed a query whose row order or row count could differ in Polars 2, say which and how you checked it.

# CLAUDE.md

@AGENTS.md

## Claude Code notes

- Skills from this loadout live in `.claude/skills/`: `polars-2-guard`. Use them when their description matches the task.
- Follow AGENTS.md above; where this file and AGENTS.md disagree, AGENTS.md wins.

<!-- From SetupBay "Polars 2 guard against Polars 1 code" v1.0.0 — https://setupbay.com/loadouts/polars-2-upgrade-guard/ -->

# GEMINI.md

@./AGENTS.md

## Gemini CLI notes

- Skills from this loadout live in `.agents/skills/`: `polars-2-guard`. Use them when their description matches the task.
- Follow AGENTS.md above; where this file and AGENTS.md disagree, AGENTS.md wins.

<!-- From SetupBay "Polars 2 guard against Polars 1 code" v1.0.0 — https://setupbay.com/loadouts/polars-2-upgrade-guard/ -->

---
description: Polars 2 only; never Polars 1 names removed in 2.0
globs: "**/*.py,**/*.ipynb"
alwaysApply: false
---

This project pins Polars 2. Lazy queries run on the streaming engine by default.

| Never (Polars 1 and older) | Always (Polars 2) |
|---|---|
| `df.melt(id_vars=, value_vars=)` | `df.unpivot(index=, on=)` |
| `with_row_count()` | `with_row_index()` (column `index`) |
| `group_by(k).count()`, `groupby(k)` | `group_by(k).len()` |
| `join(how="outer")`, `join_nulls=True` | `join(how="full")`, `nulls_equal=True` |
| `str.concat(",")` | `str.join(",")` |
| `collect(streaming=True)` | `collect(engine="streaming")` |
| `rolling_mean(w, min_periods=1)` | `rolling_mean(w, min_samples=1)` |
| `read_csv(dtypes=...)` | `read_csv(schema_overrides=...)` |
| `pl.col("d").cast(pl.Date)` on strings | `pl.col("d").str.to_date()` |
| `.apply(f)`, `.map_dict(d)` | native expressions, `.replace_strict(d)` |
| `concat(how="horizontal")` to pad | `concat(how="horizontal_extend")` |

- Row order: add `.sort(...)` or `maintain_order="left"` when the result order matters.
- `explode()` drops empty lists now; pass `empty_as_null=True` if you need a null row.
- After editing, run the `polars-2-guard` skill's scanner and the tests.

---
name: polars-2-guard
description: Check Python for Polars 1.x APIs that Polars 2 removed, and for calls whose results changed silently in 2.0, before saying a Polars change is done. Use after writing or editing Polars code, when Polars raises AttributeRemovedError or ArgumentRemovedError, or when porting older Polars or pandas snippets.
---

# Polars 2 guard

Polars 2.0 (6 October 2026) removed most names deprecated during 0.20 and 1.x, and changed defaults: lazy
queries now run on the streaming engine, which doesn't keep row order, `explode()` drops empty lists, and
horizontal `concat` refuses frames of different heights. Removed names fail loudly. The changed defaults don't.

## 1. Confirm the version
- `uv run python -c "import polars; print(polars.__version__)"` must print `2.x`. If it prints `1.x`, stop and ask
  whether the project is meant to move to Polars 2 before changing anything.

## 2. Scan
```bash
python3 .agents/skills/polars-2-guard/scripts/polars2_guard.py src tests
```
(Use the path the skill was installed to: `.claude/skills/...` for Claude Code, `.agents/skills/...` for the others.
With no arguments it scans the current folder, skipping virtualenvs and agent folders.)

- Every `error` line is an API Polars 2 removed. Fix it with the replacement the line gives;
  `references/rename-table.md` has the table with examples.
- `warn` lines are calls that still run but may now give a different answer. Decide for each one:
  - join or unpivot where order matters → add `maintain_order="left"` or `.sort(...)`;
  - `explode` where the row count matters → `empty_as_null=True`, or accept the new count;
  - a `BytesIO` round trip → `buf.seek(0)` before reading.
  If the current behaviour is what you want, add a `# guard: ok` comment to that line. That silences warnings,
  never errors.
- Run it again until it prints `0 error(s)`.

## 3. Let Polars check it
- Run the tests or the script with the pinned version: `uv run pytest -q` or `uv run python <script>.py`.
- An `AttributeRemovedError` or `ArgumentRemovedError` names the replacement. Use it.
- For pipelines whose output is compared elsewhere, compare row counts and a sorted sample before and after.

## 4. Report
Say how many Polars 1 calls you replaced, which silent changes you checked (order, counts, names) and how, and that
the scanner and tests were clean.

#!/usr/bin/env python3
"""Find Polars 1.x code that Polars 2 removed, and calls whose results changed silently in 2.0.

Usage: python3 polars2_guard.py [PATH ...]        (default: the current folder)
       python3 polars2_guard.py --list            (print every rule)

A "guard: ok" comment on a line silences its warnings (never its errors).
Prints one line per finding:  path:line: level [rule] what is wrong -> what to use
Exit 0 when nothing at "error" level is found, 1 when something is, 2 on bad usage.
Standard library only. Read-only: it never edits a file and makes no network calls.
"""
import fnmatch
import os
import re
import sys

# (rule id, level, regex, message, file patterns). Patterns run against each line with comments removed.
RULES = [
    ('melt', 'error', '\\.melt\\(',
     'melt() was removed -> unpivot(index=..., on=...) (id_vars -> index, value_vars -> on)', []),
    ('with-row-count', 'error', '\\.with_row_count\\(',
     "with_row_count() was removed -> with_row_index() (column is now 'index', not 'row_nr')", []),
    ('groupby-count', 'error', 'group_by\\([^)]*\\)\\s*\\.count\\(\\s*\\)',
     'group_by(...).count() was removed -> group_by(...).len()', []),
    ('groupby-old-name', 'error', '\\.groupby\\(',
     'groupby() -> group_by() (renamed in 0.19; also groupby_dynamic -> group_by_dynamic)', []),
    ('join-outer', 'error', 'how\\s*=\\s*[\\"\']outer(_coalesce)?[\\"\']',
     "join(how='outer') -> how='full' (outer_coalesce -> how='full', coalesce=True)", []),
    ('join-nulls', 'error', '\\bjoin_nulls\\s*=',
     'join_nulls= -> nulls_equal=', []),
    ('str-concat', 'error', '\\.str\\.concat\\(',
     'str.concat() was removed -> str.join() (default delimiter is now "", not "-")', []),
    ('lazy-fetch', 'error', '\\.fetch\\(\\s*\\d*\\s*\\)',
     'LazyFrame.fetch() was removed -> .head(n).collect()', []),
    ('lazy-profile', 'error', '\\.profile\\(\\s*\\)',
     'LazyFrame.profile() was removed (misleading on the streaming engine); time the collect() instead', []),
    ('streaming-flag', 'error', '\\bstreaming\\s*=\\s*(True|False)',
     "collect(streaming=...) was removed -> collect(engine='streaming' | 'in-memory')", []),
    ('csv-dtypes', 'error', '(read|scan)_csv\\([^)]*\\bdtypes\\s*=',
     'read_csv(dtypes=...) -> schema_overrides=... (a list must now cover every column)', []),
    ('min-periods', 'error', '\\bmin_periods\\s*=',
     'rolling_*(min_periods=...) -> min_samples=...', []),
    ('with-context', 'error', '\\.with_context\\(',
     "LazyFrame.with_context() was removed -> pl.concat([...], how='horizontal')", []),
    ('threadpool-size', 'error', 'pl\\.threadpool_size\\(',
     'pl.threadpool_size() -> pl.thread_pool_size()', []),
    ('expr-where', 'error', 'pl\\.col\\([^)]*\\)\\.where\\(',
     'Expr.where() was removed -> .filter()', []),
    ('apply', 'error', '\\.apply\\(',
     'apply() was removed in Polars 1.0 -> map_elements() (slow, last resort) or a native expression', []),
    ('map-dict', 'error', '\\.map_dict\\(',
     'map_dict() was removed -> replace() or replace_strict()', []),
    ('replace-default', 'error', '\\.replace\\([^)]*\\b(default|return_dtype)\\s*=',
     'replace(default=..., return_dtype=...) -> replace_strict(...)', []),
    ('missing-columns', 'error', '\\ballow_missing_columns\\s*=',
     "allow_missing_columns= -> missing_columns='insert' or 'raise'", []),
    ('read-rechunk', 'error', '(read|scan)_\\w+\\([^)]*\\brechunk\\s*=',
     'read_*/scan_*(rechunk=...) was removed -> call .rechunk() on the result', []),
    ('memory-map', 'error', '\\bmemory_map\\s*=',
     'read_ipc/scan_ipc(memory_map=...) was removed', []),
    ('hash-seeds', 'error', '\\bseed_[123]\\s*=|\\.hash\\(\\s*\\w+\\s*,\\s*\\w+',
     'hash(seed, seed_1, seed_2, seed_3) -> hash(seed) (one seed only)', []),
    ('categorical-ordering', 'error', 'pl\\.Categorical\\(\\s*(ordering\\s*=|[\\"\'](physical|lexical)[\\"\'])',
     'pl.Categorical(ordering=...) was removed; categoricals are always lexical now', []),
    ('to-struct-args', 'error', '\\.to_struct\\(\\s*\\)|\\b(upper_bound|n_field_strategy)\\s*=',
     "list.to_struct() needs the field names: list.to_struct(['a', 'b', 'c'])", []),
    ('concat-strict-false', 'error', 'how\\s*=\\s*[\\"\']horizontal[\\"\'][^)]*strict\\s*=\\s*False',
     "concat(how='horizontal', strict=False) now raises -> how='horizontal_extend' to pad", []),
    ('flatten', 'error', '\\.flatten\\(\\s*\\)',
     'Expr.flatten() was removed -> .list.explode(keep_nulls=False, empty_as_null=False)', []),
    ('top-k-descending', 'error', '(top|bottom)_k\\([^)]*\\bdescending\\s*=',
     'top_k/bottom_k(descending=...) -> reverse=...', []),
    ('pivot-columns', 'error', '\\.pivot\\([^)]*\\bcolumns\\s*=',
     'pivot(columns=...) -> pivot(on=...)', []),
    ('string-to-temporal-cast', 'warn', '\\.cast\\(\\s*pl\\.(Date|Datetime|Time)\\b',
     'casting a String column to Date/Datetime/Time now raises -> str.to_date() / str.to_datetime() / str.to_time()', []),
    ('concat-horizontal', 'warn', 'how\\s*=\\s*[\\"\']horizontal[\\"\'](?![^)]*_extend)',
     "horizontal concat now raises on different heights; use how='horizontal_extend' only if padding with nulls is intended", []),
    ('explode-empty', 'warn', '\\.explode\\((?![^)]*empty_as_null)',
     'explode(): empty lists now give zero rows, not one null row; pass empty_as_null=True if the old count matters', []),
    ('join-order', 'warn', '\\.join\\((?![^)]*maintain_order)(?=[^)]*\\bon\\s*=)',
     "lazy joins run on the streaming engine and don't keep row order; pass maintain_order='left' or sort if order matters", []),
    ('datetime-name', 'warn', 'pl\\.(datetime|repeat)\\((?!.*\\.alias\\()',
     "pl.datetime()/pl.repeat() are now named after their first argument; add .alias('...')", []),
    ('bytesio-seek', 'warn', 'BytesIO\\(\\s*\\)',
     'reading a BytesIO no longer rewinds it; call buf.seek(0) between write and read', []),
    ('cut-qcut', 'warn', '\\.q?cut\\(',
     'cut()/qcut() are deprecated -> bin_intervals() / bin_quantiles() (labels required, pass right_closed=True)', []),
    ('column-1', 'warn', '[\\"\']column_1[\\"\']',
     'headerless CSV columns now start at column_0, not column_1', []),
]
FILES = ['*.py', '*.ipynb']
COMMENT = '#'
SKIP_DIRS = {".git", ".venv", "venv", "node_modules", "__pycache__", ".agents", ".claude", ".cursor", "dist", "build", "target", ".godot"}
COMPILED = [(rid, lvl, re.compile(rx), msg, globs) for rid, lvl, rx, msg, globs in RULES]


def strip_comment(line):
    """Drop a trailing comment, leaving string literals alone."""
    q = None
    i = 0
    while i < len(line):
        c = line[i]
        if q:
            if c == "\\":
                i += 2
                continue
            if c == q:
                q = None
        elif c in "\"'`":
            q = c
        elif line.startswith(COMMENT, i):
            return line[:i]
        i += 1
    return line


def wanted(name):
    return any(fnmatch.fnmatch(name, g) for g in FILES)


def walk(paths):
    for p in paths:
        if os.path.isfile(p):
            yield p
            continue
        for root, dirs, files in os.walk(p):
            dirs[:] = sorted(d for d in dirs if d not in SKIP_DIRS)
            for f in sorted(files):
                if wanted(f):
                    yield os.path.join(root, f)


def scan(path):
    out = []
    name = os.path.basename(path)
    try:
        with open(path, encoding="utf-8", errors="replace") as fh:
            lines = fh.read().splitlines()
    except OSError as e:
        return [(path, 0, "error", "unreadable", str(e))]
    for n, raw in enumerate(lines, 1):
        code = strip_comment(raw)
        if not code.strip():
            continue
        accepted = "guard: ok" in raw  # a person accepted the warnings on this line
        for rid, lvl, rx, msg, globs in COMPILED:
            if accepted and lvl == "warn":
                continue
            if globs and not any(fnmatch.fnmatch(name, g) for g in globs):
                continue
            if rx.search(code):
                out.append((path, n, lvl, rid, msg))
    return out


def main(argv):
    if "--list" in argv:
        for rid, lvl, _, msg, globs in RULES:
            print(f"{lvl:5} [{rid}] {msg}" + (f"  (only {', '.join(globs)})" if globs else ""))
        return 0
    if any(a.startswith("-") for a in argv):
        print(__doc__)
        return 2
    paths = argv or ["."]
    for p in paths:
        if not os.path.exists(p):
            print(f"no such path: {p}", file=sys.stderr)
            return 2
    files = list(walk(paths))
    findings = []
    for f in files:
        findings.extend(scan(f))
    for path, n, lvl, rid, msg in findings:
        print(f"{os.path.relpath(path)}:{n}: {lvl} [{rid}] {msg}")
    errors = sum(1 for x in findings if x[2] == "error")
    warns = sum(1 for x in findings if x[2] == "warn")
    print(f"polars2_guard: {len(files)} file(s), {errors} error(s), {warns} warning(s)")
    return 1 if errors else 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

# Polars 1 → Polars 2 table

From the Polars 2.0 upgrade guide (https://docs.pola.rs/releases/upgrade/2/). Checked against polars 2.1.0.

## Removed: raises in Polars 2

| Polars 1.x (or older) | Polars 2 |
|---|---|
| `df.melt(id_vars="a", value_vars=["b"])` | `df.unpivot(index="a", on=["b"])` |
| `df.with_row_count()` (column `row_nr`) | `df.with_row_index()` (column `index`) |
| `df.group_by("k").count()` | `df.group_by("k").len()` |
| `df.groupby("k")` | `df.group_by("k")` |
| `join(how="outer")` / `"outer_coalesce"` | `join(how="full")` / `how="full", coalesce=True` |
| `join(join_nulls=True)` | `join(nulls_equal=True)` |
| `pl.col("s").str.concat("-")` | `pl.col("s").str.join("-")` (default delimiter is now `""`) |
| `lf.fetch(5)` | `lf.head(5).collect()` |
| `lf.profile()` | none; time `collect()` instead |
| `collect(streaming=True)` | `collect(engine="streaming")` |
| `read_csv(dtypes={...})` | `read_csv(schema_overrides={...})` |
| `rolling_mean(3, min_periods=1)` | `rolling_mean(3, min_samples=1)` |
| `lf.with_context(other)` | `pl.concat([lf, other], how="horizontal")` |
| `pl.threadpool_size()` | `pl.thread_pool_size()` |
| `pl.col("a").where(cond)` | `pl.col("a").filter(cond)` |
| `.apply(f)` (removed in 1.0) | a native expression; `map_elements(f)` as a last resort |
| `.map_dict(d)` | `.replace(d)` or `.replace_strict(d)` |
| `.replace(d, default=x)` | `.replace_strict(d, default=x)` |
| `read_parquet(allow_missing_columns=True)` | `read_parquet(missing_columns="insert")` |
| `read_*(rechunk=True)` | `read_*(...).rechunk()` |
| `read_ipc(memory_map=True)` | `read_ipc(...)` |
| `hash(1, 2, 3, 4)` | `hash(1)` |
| `pl.Categorical(ordering="lexical")` | `pl.Categorical()` (always lexical) |
| `list.to_struct(upper_bound=3)` | `list.to_struct(["a", "b", "c"])` |
| `concat(how="horizontal", strict=False)` | `concat(how="horizontal_extend")` |
| `Expr.flatten()` | `list.explode(keep_nulls=False, empty_as_null=False)` |
| `top_k(k, by=..., descending=True)` | `top_k(k, by=..., reverse=True)` |
| `pivot(columns="c")` | `pivot(on="c")` |
| `pl.Series(["2026-10-01"]).cast(pl.Date)` | `pl.Series(["2026-10-01"]).str.to_date()` |

## Changed silently: same code, different answer

| Change in 2.0 | What to do |
|---|---|
| Lazy `collect()` runs on the streaming engine; joins, `group_by`, `unpivot` don't keep row order | `.sort(...)`, or `join(..., maintain_order="left")` |
| `explode()` turns an empty list into zero rows (was one null row) | `explode(..., empty_as_null=True)` for the old count |
| `concat(how="horizontal")` raises on different heights (used to pad) | pad on purpose with `how="horizontal_extend"` |
| `pl.datetime(...)` / `pl.repeat(...)` named after their first argument | `.alias("datetime")` |
| Reading a `BytesIO` starts at its current position | `buf.seek(0)` after writing |
| Signed int + `UInt64` gives `Int128`, not `Float64` | cast if a float was expected |
| Headerless CSV columns are `column_0, column_1, ...` | update names that started at `column_1` |
| Map columns from Arrow/Parquet load as `pl.Map` | `.map.entries()` for the old list-of-structs shape |
| `cut()` / `qcut()` deprecated | `bin_intervals(...)` / `bin_quantiles(...)`, `right_closed=True`, labels required |

To restore the in-memory engine for one query: `lf.collect(engine="in-memory")`. Ask before doing it project-wide.

# A Polars 1.x report script, used to check the guard. Every marked line fails or changes in Polars 2.
import io

import polars as pl

sales = pl.read_csv("sales.csv", dtypes={"store": pl.Utf8})                  # csv-dtypes
sales = sales.with_row_count()                                               # with-row-count
long = sales.melt(id_vars="store", value_vars=["q1", "q2"])                  # melt
per_store = sales.group_by("store").count()                                  # groupby-count
joined = sales.join(per_store, on="store", how="outer")                      # join-outer
labels = sales.select(pl.col("store").str.concat(", "))                      # str-concat
smooth = sales.select(pl.col("q1").rolling_mean(3, min_periods=1))           # min-periods
big = sales.lazy().filter(pl.col("q1") > 10).collect(streaming=True)         # streaming-flag
days = sales.with_columns(pl.col("day").cast(pl.Date))                       # string-to-temporal-cast
names = sales.with_columns(pl.col("store").map_dict({"A": "Leeds"}))         # map-dict
buf = io.BytesIO()                                                           # bytesio-seek
sales.write_parquet(buf)
again = pl.read_parquet(buf)