Back to Devoured - September 08, 2026
DEVOURED
Ruff, mypy, pytest, and then what?

Ruff, mypy, pytest, and then what?

Tech CodeScan
Codebases written by AI agents suffer from unique structural issues like hidden duplication that standard linters fail to catch.
What: The author shares experience using structural analysis tools to manage agent-written code. Standard CI (Ruff/mypy/pytest) misses duplication across modules and dependency violations. Implementing a structural analyzer (pyscn) allowed an AI agent to catch and collapse duplicated helper functions during the development process.
Why it matters: As agents write more code, the human burden shifts from catching syntax errors to architectural review, making automated structural enforcement mandatory for maintainability.
Takeaway: If using AI agents to write code, treat structural analyzers like pyscn or import-linter as essential tools; provide these to the agent as MCP tools so it can check itself before producing the final code.
Deep dive
  • Agents naturally duplicate logic when they don't have visibility into previous sessions, creating 'code rot' that slows down future AI iterations.
  • Standard CI tools (Ruff/mypy) only validate isolated file validity, not cross-module integrity or conceptual duplication.
  • Structural analysis (like pyscn or radon) tracks cyclomatic complexity and clone groups.
  • The most effective fix is providing the agent with the structural tool as an MCP tool to use during the generation phase.
  • Tracking the average complexity of a codebase is a better predictor of technical debt than the maximum complexity of a single file.
Decoder
  • Cyclomatic Complexity: A metric quantifying the number of linearly independent paths through a program's source code; higher numbers indicate more complex logic.
  • MCP (Model Context Protocol): A standard for connecting AI assistants to data and tools.
  • Clone detection: The process of identifying redundant blocks of code across a repository to prevent duplication.
Original article

The moment I started caring about this was not a code review. It was watching an agent get lost in its own code.

It had been working on a project for a while. Every time it needed a helper, it wrote one. Some of them were near-copies of helpers it had written a few sessions earlier, with a slightly different name and one extra parameter. Nothing broke. Ruff was happy, mypy was happy, the tests were green. Then one day I asked it to change how a value was normalized, and it stalled. There were four functions that did roughly that. It read all four, picked one, changed it, ran the tests, found that two other call sites still used a different copy, patched those, and ended up modifying three of the four. It was not sure which one was canonical. Neither was I.

Duplicated code in an agent-written codebase does not only cost the humans. It costs the agent, on every later turn, because it has to re-read all the copies and guess.

What the usual pipeline does and does not check

Most Python CI in 2026 looks roughly like this:

- run: uv run ruff check src tests
- run: uv run ruff format --check src tests
- run: uv run mypy src            # strict = true
- run: uv run pytest -n auto

Ruff checks conventions: unused imports, shadowed names, bad comprehensions. It can also check function size and complexity if you turn on C901 or the PLR09 rules, though most projects I see do not, and the one below did not either. mypy checks that values flowing between functions have the shapes the signatures claim. pytest checks that behavior matches whatever tests somebody wrote.

What none of them do is look across functions. Ruff has no rule for "this block also exists in another file." Nothing in the pipeline knows which modules are allowed to import which. A helper that nobody calls anymore passes lint because it parses and passes mypy because there are no call sites to disagree with it. Before agents, a human reviewer caught these things by reading the diff. Agents produce diffs faster than humans read them, and that part of review quietly stopped happening.

Measuring a real repository

I ran a structural analysis over one repository at five points in its history. It is a few thousand lines of NumPy and SciPy with a CLI and an MCP server, mostly agent-written, with the CI shown above. The tool was pyscn, which I maintain. For complexity and dead code you can get equivalent numbers from radon and vulture, and import-linter covers dependency rules. Clone detection is the piece that does not have an obvious standalone answer in the Python ecosystem, which is a large part of why pyscn exists.

pyscn classifies a function's cyclomatic complexity as low at 9 or below, medium from 10 to 19, and high at 20 or above. The check command, the one meant for CI, fails by default on anything above 10.

Commit Date Source lines Health Avg. complexity Functions ≥ 10 Duplication Cycles
Initial Feb 1 5,387 A (92) 6.9 — 1.6% 0
Phase 2 Feb 3 6,027 B (84) 7.7 — 0.0% 0
Phase 5 Feb 12 8,861 A (90) 8.3 — 0.2% 0
v0.5.0 Feb 21 9,173 A (92) 8.5 — 0.2% 0
HEAD Jun 18 ~9,200 A (91) 8.4 8 of 32 0.4% 0

I should be honest about this table. It is not a horror story, and I did not expect one. This repository was developed with pyscn available to the agent as an MCP tool, so it was getting structural feedback while it wrote. An A is roughly what you would hope for under those conditions.

What the table does show is that the average crept up, from 6.9 to 8.5 across the snapshots, while the grade stayed flat. No function reached the high band, so the summary never changed color. Eight functions at HEAD sit in the medium band, and pyscn check with its default threshold would have failed on all of them, but this project never ran check in CI. It only ever looked at the grade. Five snapshots are not enough to say the code got denser with every merge. They are enough to say that a grade is not a trend line, and that if you only look at the grade you will miss the trend.

The two most complex functions at HEAD are both at 19. One is a shape-validation ladder:

def _validate_dimensions(y, T, Z, R, Q, H, a0=None, P0=None) -> None:
    if y.ndim != 2:
        raise ValidationError(...)
    if T.shape != (n_state, n_state):
        raise ValidationError(...)
    if Z.shape != (n_obs, n_state):
        raise ValidationError(...)
    # ... six more of these

I looked at it and left it alone. It is flat, the error messages are good, and splitting it would not help anyone. A metric is a reason to look, not a verdict.

The other one I did not leave alone. In the CSV loader:

elif obs.transform == "level":
    if drop_first_row:
        transformed.append(series[1:])
    else:
        transformed.append(series)
elif obs.transform == "rate":
    if drop_first_row:
        transformed.append(series[1:])
    else:
        transformed.append(series)

Two branches, same body. This is the same habit as the four normalization helpers, just smaller: the agent writes the branch it needs right now and does not check whether an identical one is already sitting three lines up. Ruff says nothing, mypy says nothing, the tests pass.

The dependency check turned up one more. The package that exposes the tool to the AI assistant imports directly from the deepest numerical modules, skipping the layer in between. Whether that is a violation depends on rules I had never written down. I had an architecture in my head and nothing in the repository enforcing it, so the agent drew the graph however was convenient that day.

What actually helped

In this project the recurring problem was duplication. The agent would add a helper rather than check whether an earlier session had already introduced one, and it would write a branch rather than check whether the branch above it was identical.

The thing that reduced it was not a CI gate. It was giving the agent the checker as a tool it could call itself, in the same session, before I saw the diff. When clone detection reported a group, the agent could collapse it while it still had the context of why both copies existed. By the time a human reviewer sees a clone, that context is gone and the fix is a chore.

Beyond that, a few boring things:

Run pyscn check or your equivalent in CI, but gate on the delta. Fail when a PR introduces a function over the threshold or adds a clone group, not when the whole repository is imperfect. Otherwise the first run reports 200 findings and the check is disabled by lunchtime.

Track the average complexity over time, not just the maximum. The table above is why.

Write the dependency rules down, in import-linter contracts or whatever your tool reads. Agents do read prose architecture notes, but a contract that fails their build is harder to drift away from than a paragraph in CLAUDE.md.

Why bother

Code that is structured well is easier to maintain. Everyone knows that. What I had not appreciated until I watched an agent lose track of its own helpers is that it is also easier for the agent. One way to do each thing, small functions, and a dependency graph that points in one direction is a codebase an agent can read quickly and change safely. That is not a tax on AI-assisted development. It is what keeps the next thousand lines cheap.

Explore all 46 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi