Pre-commit hooks are the feedback loop AI coding was missing

A coding assistant writing code inside a repository is a control loop. It proposes a change, something evaluates that change, and the evaluation determines the next proposal. In most setups the evaluation step is a person: they run the formatter, notice the type error, spot the failing test, and type a summary of what went wrong back into the chat. That person is not reviewing. They are acting as a sensor — manually carrying a machine-readable fact from one machine to another.

Count the prompts in a typical task and most of them are that relay. “You forgot the import.” “Line 40 fails mypy.” “Reformat it.” None of those messages carry human judgment; each one is information a tool already produced in a structured form. The move is not making the model better at guessing, but removing the human from the middle of the wire.

The original note

“Documented the pre-commit system that took our AI-assisted workflow from 10+ prompts per task to 1–2.”

The write-up covered a phased rollout strategy, a team adoption playbook, and quantified metrics — “Not AI hacks—just amplified engineering principles.”

“Who else is measuring prompts-per-task as a dev metric?”

The loop, not the model

A control loop needs three things: an action, a measurement of that action, and a path from the measurement back to the next action. An assistant editing files has the action. A repository with tests, a linter, and a type checker has the measurement. What is usually missing is the path — the measurement lives behind a command nobody told the assistant to run, in a terminal it cannot see.

Pre-commit hooks close that path. pre-commit is a framework that runs a configured set of checks against staged files before a commit is created; if any check fails, the commit is rejected and the check’s output is printed. The critical property is not the blocking. It is that the output is produced on a fixed trigger, in a fixed format, without anyone deciding to ask for it.

That reframes what an extra prompt means. Each round trip in a task is a place where the loop opened up and a human had to close it by hand. Reduce the number of facts that require human relay and the round trips fall with them. The note above reports a large drop on exactly this mechanism; the mechanism is the transferable part, not the figure.

There is a second reason this works that has nothing to do with intelligence. Deterministic tools are correct about their own domain: a formatter is the authority on formatting, a type checker on whether a call signature matches. Wire those authorities into the loop and the assistant stops guessing at questions that have exact answers.

What makes a hook set readable by a machine

Most hook sets are designed for humans who will read them once and move on. A hook set that an agent consumes has stricter requirements.

Fast. Target the whole run under ten seconds on a normal change. This is not aesthetic. A slow hook set trains humans to reach for --no-verify and trains agents (or the people driving them) to skip the verification step entirely. Speed is the difference between a check that runs on every iteration and a check that runs once at the end, which is the same as no loop at all.

Deterministic. The same input must produce the same output, every time, on every machine. A check that depends on wall-clock time, network availability, or file ordering produces noise the assistant will try to “fix” by changing unrelated code. Pin tool versions; avoid checks that hit a network service.

Auto-fixing where possible. A hook that repairs the code itself costs zero round trips. A hook that reports a problem costs at least one. Formatting, import sorting, trailing whitespace, and a large share of lint rules can be applied rather than reported, and every one that is applied is a message nobody has to write. Reserve reporting for the things that genuinely need a decision.

Precise in its errors. The message should name the file, the line, and the rule identifier. E501 line too long at src/api.py:44 is actionable without any further context. “Style issues found” is not — it forces another round trip just to locate the problem, which is the exact cost the hook was supposed to remove.

Those four properties are why the Python stack here is ruff rather than the older black plus flake8 plus isort arrangement. Ruff covers formatting, import sorting, and most of the lint rule set in one tool and one pass. Three tools means three process startups, three configuration files, and three chances for the rules to disagree. Speed and determinism are the point of this section, and consolidating buys both.

# .pre-commit-config.yaml
# Every hook here is fast, deterministic, and names file+line on failure.
repos:
  # Guards: cheap checks that catch mistakes no reviewer should have to.
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v5.0.0
    hooks:
      # A large file in git history is effectively permanent — block it
      # at the only moment it is still cheap to stop.
      - id: check-added-large-files
        args: ['--maxkb=512']
      # Malformed config breaks tooling in confusing, indirect ways.
      - id: check-yaml
      - id: check-toml
      # Auto-fixing: costs zero round trips.
      - id: end-of-file-fixer
      - id: trailing-whitespace

  # Secrets: the one failure that cannot be undone by a later commit.
  - repo: https://github.com/Yelp/detect-secrets
    rev: v1.5.0
    hooks:
      - id: detect-secrets
        args: ['--baseline', '.secrets.baseline']

  # Lint + format in one pass. Replaces black + flake8 + isort.
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.9.2
    hooks:
      # --fix repairs what it can; only the remainder is reported.
      - id: ruff
        args: ['--fix', '--exit-non-zero-on-fix']
      - id: ruff-format

  # Types: the check that catches contract breakage across files,
  # which a single-file reader (human or model) cannot see.
  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.14.1
    hooks:
      - id: mypy
        additional_dependencies: ['pydantic>=2.0']
        # Start narrow. Widen the path as annotations land.
        files: ^src/core/

  # Tests: a local hook so it runs in the project's own environment.
  # Scoped and time-boxed so the run stays inside the budget.
  - repo: local
    hooks:
      - id: pytest-fast
        name: pytest (fast subset)
        entry: pytest -q -m "not slow" --timeout=20 -x
        language: system
        types: [python]
        pass_filenames: false

Rolling it out without stopping the work

Turning every check on at once in an existing codebase produces a wall of pre-existing violations, and the rational response to a wall is to bypass it. The rollout has to be ordered so each phase lands cleanly before the next one starts.

Phase	What lands	What it unblocks
1	Formatter only	Diff noise lands once
2	Lint, changed files	Real defects visible
3	Types, scoped dirs	Cross-file contracts
4	Fast test subset	Behavior regressions

Phase 1 — format only. Run the formatter across the whole repository in a single commit, then turn the hook on. Every subsequent diff is now signal. Formatting churn otherwise contaminates every later phase: a reviewer, or an assistant reading the diff, cannot tell a reindent from a logic change.

Phase 2 — lint on changed files. Pre-commit scopes to staged files by default, so pre-existing violations elsewhere block nobody. New code meets the standard; old code is fixed as it is touched.

Phase 3 — types, scoped by path. Type checking is the highest-value check for an assistant because it catches breakage across files — exactly what a model editing one file at a time is worst at anticipating. It is also the most disruptive to enable everywhere at once. Scope it to a directory, make that directory strict, and widen the glob as annotations land.

Phase 4 — a fast test subset. Not the whole suite. A marked subset that runs in seconds, so the loop stays inside its time budget.

# pyproject.toml — strictness that an existing codebase can adopt.

[tool.ruff]
line-length = 88
target-version = "py311"

[tool.ruff.lint]
# Start with a defensible core, not the full rule catalogue.
# E/F: pycodestyle + pyflakes. I: import sorting (replaces isort).
# B: bugbear (real defects). UP: modernizations that are auto-fixable.
select = ["E", "F", "I", "B", "UP"]
ignore = ["E501"]  # the formatter owns line length; don't double-report

[tool.ruff.lint.per-file-ignores]
"tests/*" = ["B011"]

[tool.mypy]
python_version = "3.11"
# Global posture: useful, not blocking. Strictness is opt-in per module.
warn_unused_ignores = true
warn_redundant_casts = true
ignore_missing_imports = true

# New and refactored code is held to the real standard.
[[tool.mypy.overrides]]
module = "src.core.*"
disallow_untyped_defs = true
disallow_any_generics = true
strict_equality = true

# Legacy modules are documented debt, not silent exceptions.
[[tool.mypy.overrides]]
module = "src.legacy.*"
ignore_errors = true

[tool.pytest.ini_options]
markers = ["slow: excluded from the pre-commit subset"]

Adoption: bypasses and the CI mirror

Two rules keep the system from decaying.

A --no-verify bypass is a bug report about the hook, not about the developer. Treat every bypass as evidence that a check is too slow, too noisy, or wrong, because that is almost always what it is. Someone reaching for the escape hatch under deadline pressure is behaving rationally against the incentives in front of them. Fix the incentive.

CI must run the same checks, from the same config. Local hooks are advisory by construction: they can be skipped, and a fresh clone may have them uninstalled. If the identical check set does not also run in continuous integration, the hooks are theatre — they inconvenience the compliant and stop nothing. Running pre-commit run --all-files in CI guarantees one definition of “correct” rather than two that drift apart.

That single command is also the interface an assistant should use. Rather than being told what is wrong, it runs the check set and reads the result.

#!/usr/bin/env bash
# verify.sh — one command that produces the whole error signal.
# Hooks auto-fix in place, so a second pass shows what remains.

set -uo pipefail

pre-commit run --all-files > .verify.log 2>&1
status=$?

if [ $status -eq 0 ]; then
  echo "PASS: no remaining findings"
  exit 0
fi

# Surface only failing hooks and their file:line:rule output.
echo "FAIL: unresolved findings below"
grep -E '^(.*\.py:[0-9]+|.*\.\.\.*(Failed))' .verify.log
exit $status

The loop is then mechanical: make a change, run verify.sh, read the output, make the next change. No human is needed to notice or describe the error. The human is needed for what a checker cannot evaluate — whether the design is right, whether the abstraction earns its cost, whether the feature is the correct feature.

On prompts-per-task as a metric

The source note ends with an open question, and it deserves to be treated as one rather than answered too quickly.

What it proxies. Prompts-per-task measures round trips to convergence — how many exchanges it took to reach a state the author accepted. That is a reasonable proxy for friction in the loop, and friction is a real cost.

Why it is attractive. It is observable without instrumentation, it is intuitive, and it moves in the right direction when the underlying system genuinely improves. Metrics with those three properties are uncommon.

Where it breaks. Three failure modes, and all of them are serious:

It is trivially gameable. One long prompt containing what would have been six short ones scores a six-fold improvement while changing nothing about the work. Any metric counting messages rewards message consolidation first.
It is not comparable across tasks. A config tweak and a schema migration are not the same unit. Aggregate prompts-per-task moves with the task mix as much as with the tooling, which makes period-over-period comparisons unreliable unless difficulty is held constant.
It says nothing about output quality. Converging in one prompt to code that fails in production is worse than converging in six to code that holds. Optimized alone, the metric rewards accepting the first plausible answer.

How to read it safely. Pair it with a quality measure — change-failure rate (the share of changes that require a fix after merge) or review-rework rate (the share sent back for changes) — and treat movement in the pair, not in either alone. A drop in prompts alongside a flat or falling failure rate is evidence the loop tightened. A drop in prompts alongside a rising failure rate means corners are being cut, and the metric is now actively harmful. The honest position is that prompts-per-task is a useful diagnostic and a dangerous target, which is the usual fate of any metric that is easy to count.

Failure modes

The slow hook set. Past roughly ten seconds, verification stops being part of the iteration and becomes a gate at the end. The loop reopens.
The flaky check. A hook that fails intermittently teaches everyone to re-run rather than read, and sends assistants chasing phantom defects.
The vague message. A failure that does not name a file and line costs an extra round trip to locate — reintroducing the cost the hook removed.
Local-only enforcement. Without a CI mirror, the checks constrain only the people who were already careful.
Big-bang enablement. Turning everything on across a legacy codebase produces a violation wall, and the wall produces bypasses, and the bypasses become permanent.
Confusing the checkable with the important. Passing hooks means the code is well-formed, typed, and not obviously broken — nothing about whether the design is sound. Human review is not what got automated here.

Closing

Nothing described above is new. Deterministic formatting, fast static analysis, typed interfaces, and a quick test suite were good practice long before language models could write code — the argument for each was always that a fast, unambiguous error signal is cheaper than a slow, ambiguous one. That argument has not changed. What changed is the return on it.

When a human is the only consumer of the error signal, the value of tightening it is bounded by how often that human reads it. When a tireless agent consumes the same signal on every iteration, the same investment compounds across every loop it runs. The engineering principle is identical. The multiplier is not.