Harness Engineering: Automated PR Quality Gates for AI Code

Your developers are using AI coding agents. The code is flowing faster than ever. But faster without governance means faster bugs, faster security holes, and faster compliance failures reaching production. Independent testing shows the stakes: Veracode's 2025 study of more than 100 large language models found that 45% of AI-generated code samples introduced known security vulnerabilities (S4). Harness engineering is the discipline that solves this, it is the work of building everything around an AI model that makes it a reliable, auditable, governed contributor to your codebase. Not a loose cannon that happens to write good code most of the time.

The term was named by Mitchell Hashimoto, co-founder of HashiCorp, in a February 2026 essay (S1), then formalized by Birgitta Böckeler, a Distinguished Engineer at Thoughtworks, in her guides and sensors framework on Martin Fowler's site (S2). Thoughtworks' own Technology Radar Vol. 34 now urges teams to wrap coding agents in exactly these feedforward and feedback controls (S3). The shorthand that captures it: Agent = Model + Harness. The model writes code. The harness makes sure that code is safe, compliant, tested, reviewed, and fit for your specific business context before it reaches production.

If you run a bank, an insurance company, a healthcare provider, an e-commerce platform, or any regulated European business, this is not optional. It is the architecture that lets you say yes to AI coding agents without saying yes to uncontrolled risk.

TL;DR: Harness engineering wraps an AI coding agent in guides (CLAUDE.md, AGENTS.md) and sensors (tests, linters, security scans, PR review) so its output stays safe, compliant, and auditable before production.

What a Harness Actually Is

A harness is everything around the AI model that controls its behaviour:

┌─────────────────────────────────────────────────────┐
│                    THE HARNESS                       │
│                                                      │
│  ┌──────────────┐     ┌──────────────────────────┐  │
│  │   GUIDES     │     │       SENSORS            │  │
│  │ (feedforward) │     │     (feedback)           │  │
│  │              │     │                          │  │
│  │ CLAUDE.md    │     │ Tests (pass/fail)        │  │
│  │ AGENTS.md    │     │ Lint (ruff, eslint)      │  │
│  │ System prompt│     │ Type check (mypy, tsc)   │  │
│  │ Tool limits  │     │ Security scan (Snyk)     │  │
│  │ File scope   │     │ PR review (Codex, human) │  │
│  │ Branch rules │     │ Post-merge monitoring    │  │
│  └──────┬───────┘     └──────────┬───────────────┘  │
│         │                        │                   │
│         ▼                        ▼                   │
│  ┌──────────────────────────────────────────────┐   │
│  │              AI MODEL                         │   │
│  │     (Claude, GPT, DeepSeek, Gemini)          │   │
│  │                                               │   │
│  │  Writes code within the constraints above     │   │
│  └──────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────┘

Guides (Feedforward Controls)

Guides constrain the agent before it acts. They define what the agent knows, what it can do, and what it must not do.

Guide	What It Does	Example
CLAUDE.md	Project-level instructions the agent reads at session start	This repo uses Python 3.11; tests live in tests/; never modify config/production.yml
AGENTS.md	Review guidelines and team standards	Treat missing tests as P1; flag any new external API call without rate limiting
System prompt	Model-level behaviour constraints	You are a senior engineer; always write tests before implementation
Tool restrictions	What the agent can and cannot do	Read-only on the production database; write-only to feature branches
File scope	Which files the agent can touch	Only edit files in src/; never touch .env, credentials.json, or migrations/
Branch protection	Where code can land	No direct push to main; PRs required; at least 1 approval

Sensors (Feedback Controls)

Sensors observe after the agent acts and either block bad output or feed corrections back.

Sensor	What It Catches	When It Fires
Unit tests	Broken functionality, regressions	On every commit (CI)
Linter (ruff, eslint)	Style violations, unused imports, syntax issues	On every commit (CI)
Type checker (mypy, tsc)	Type mismatches, null safety violations	On every commit (CI)
Security scanner (Snyk, Semgrep)	Known vulnerabilities, injection risks, secret leaks	On every PR
AI PR reviewer (Codex, CodeRabbit)	Logic errors, architectural violations, missing docs	On every PR
Human reviewer	Business logic, architectural fit, risk assessment	Before merge
Post-merge monitor	Production regressions, performance degradation	After deploy

Why You Need Both

Feedforward-only (guides without sensors) encodes rules the agent might ignore. You have a CLAUDE.md that says to write tests, but no CI check that verifies tests exist.

Feedback-only (sensors without guides) catches problems after they happen. The agent writes a large batch of code, CI fails on a long list of issues, the agent rewrites, CI fails on a different set. Expensive iteration.

The combination is what works: your CLAUDE.md AGENTS.md governance prevents most issues before they happen; sensors catch what guides miss.

The Harness Stack: AI Code Review Governance for a Regulated Industry

Different sectors need different sensors, but the shape is the same: these are the code quality gates regulated industry teams must be ready to show a regulator, and the AI code review governance that makes every AI-generated change attributable.

Banking and Financial Services

Layer	What You Need	Why	Tool
Code origin	AI-generated code must be attributable	Audit trail for regulators	Git commit metadata + AGENTS.md
Secret scanning	No credentials, API keys, or tokens in code	PCI-DSS, SOX	GitHub secret scanning + pre-commit hooks
Dependency audit	No vulnerable or unlicensed dependencies	Supply chain security	Snyk, Dependabot
Data access control	AI cannot access production customer data	GDPR, banking regulations	Tool restrictions in harness + network policies
Segregation of duties	The agent that writes code cannot approve or merge it	SOX, internal audit	Branch protection + required human reviewer
Audit logging	Every AI action is logged and queryable	Regulatory audit	Agent session transcripts + CI logs

Insurance and Healthcare

Layer	What You Need	Why	Tool
PHI/PII detection	AI-generated code cannot log, store, or expose personal data	HIPAA, GDPR	Semgrep rules + custom lint checks
Decision auditability	Any AI-assisted decision must be explainable	EU AI Act, Article 13 (if high-risk) (S5)	Structured logging + decision documentation
Model documentation	Which AI model produced which code, when	Internal audit best practice; supports EU AI Act Article 13 transparency and instructions for use duties	Git metadata + harness configuration versioning

E-Commerce

Layer	What You Need	Why	Tool
Payment flow protection	AI cannot modify payment processing code without human review	PCI-DSS	CODEOWNERS on payment directories + mandatory review
Price integrity	AI cannot change pricing logic without business approval	Revenue protection	Business-critical file guards
A/B test safety	AI-generated experiments cannot affect checkout conversion without monitoring	Revenue	Feature flag integration + rollback triggers

Building Your Harness: The Practical Steps

Step 1: Write Your CLAUDE.md (30 Minutes)

This is the single most impactful step. Every AI coding session starts by reading this file.

# CLAUDE.md

## What this project is
[1-2 sentences describing the business context]

## Technology stack
- Language: Python 3.11 / TypeScript 5.x
- Framework: [your framework]
- Database: [your database]
- Tests: pytest / jest in [directory]

## Rules (mandatory)
- Write tests for every new function
- Never modify files in config/production/
- Never hardcode secrets, use environment variables
- All database queries must use parameterised statements
- No PII in log statements

## Review standards
- Missing tests = P1 (blocks merge)
- New external API without rate limiting = P1
- Security scan failure = highest severity (blocks everything)

Step 2: Set Up Automated PR Quality Gates (1 Hour)

# .github/workflows/quality-gates.yml
name: Quality Gates
on: [pull_request]

jobs:
  quality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run tests
        run: pytest
      - name: Lint
        run: ruff check .
      - name: Type check
        run: mypy src/
      - name: Security scan
        run: semgrep scan .
      - name: Test count guard (floor = 100)
        run: |
          COUNT=$(grep -rhoE 'def test_' tests | wc -l)
          test "$COUNT" -ge 100

Step 3: Enable AI PR Review (5 Minutes)

Enable Codex code review in your Codex settings (turn on Code review and, separately, Automatic reviews), and add review guidelines to AGENTS.md. Every new PR then gets an automatic AI review before human review.

Step 4: Configure Branch Protection (10 Minutes)

GitHub → Settings → Branches → Branch protection rules:

Require pull request reviews (minimum 1)
Require status checks to pass (tests, lint, security)
Require branches to be up to date
No force pushes
No deletions

Step 5: Add Post-Merge Monitoring (Optional, 2 Hours)

For production deployments: alert on error rate spikes, latency increases, or conversion drops within 1 hour of merge. Automated rollback if thresholds are exceeded.

What We Built (Our Own Harness)

We practice what we publish. Our content engine's harness includes:

CLAUDE.md, a detailed set of project conventions, plus a lane state guard and a source count check
AGENTS.md, review guidelines, an autonomy envelope (MAY/HALT/PROHIBITED actions), and content quality and trust standards
Pre-commit hooks, a lane state guard (prevents wrong-branch commits) and a source count check (validates article citations)
Thousands of automated tests behind a hard CI floor; we never merge if the count drops
Codex review-thread gate, a read-only CI gate that enumerates unresolved AI-reviewer threads on every PR before merge. It has no GitHub App and runs no model-harness CLI, so it surfaces review threads rather than measuring code correctness itself
Tenant isolation, multi-tenant architecture with per-tenant runtime paths, no cross-contamination
Distribution ledger, append-only record of every publish action

This harness is why we can operate two active tenants (with further tenants configured), publish autonomously, and maintain quality. The model writes. The harness governs.

The Cost of Not Having a Harness

Scenario	Without Harness	With Harness
AI introduces a SQL injection	Reaches production. Customer data exposed. GDPR breach notification required.	Caught by Semgrep in CI. PR blocked. Never reaches main.
AI deletes a test to make code pass	Test count drops. Regressions go undetected for weeks.	Test count guard fails CI. PR blocked.
AI hardcodes an API key	Secret in git history forever. Credential rotation + incident response.	Pre-commit secret scan catches it. Commit rejected.
AI modifies payment processing logic	Subtle pricing bug. Revenue loss before anyone notices.	CODEOWNERS requires business owner review for payment files.
AI generates code with a vulnerable dependency	Supply chain attack surface.	Dependabot + Snyk flag before merge.

Each of these is the kind of failure that is expensive and reputation-damaging once it reaches production: a breach notification, an emergency credential rotation, a revenue-affecting bug. The harness costs about 2 hours to set up.

Frequently Asked Questions

Do I need a harness if my team only has 3 developers?

Yes, even more so. With 3 developers, every person's time is precious. A harness prevents the 2-day debugging session caused by a bad AI-generated merge. The 2 hours of setup saves 20+ hours of incident response per year.

Is CLAUDE.md the same as AGENTS.md?

CLAUDE.md is project-level context (technology stack, conventions, file structure). AGENTS.md is team-level governance (review standards, autonomy boundaries, brand rules). Both are guides in the harness, but they serve different purposes.

Can I use this with any AI coding agent?

Yes. The harness is model-agnostic. CLAUDE.md works with Claude Code. AGENTS.md works with Codex. CI quality gates work with any agent that commits code. Branch protection works regardless of who creates the PR.

How do regulated industries (banks, insurance) audit AI-generated code?

Through the harness's audit trail: git commit history (who/what generated the code), CI logs (which checks passed), PR review history (what was flagged and resolved), and AGENTS.md versioning (what rules were in effect when the code was written).

What is the difference between harness engineering and DevOps?

DevOps automates the build, test, and deploy pipeline. Harness engineering adds a governance layer specifically for AI agents: constraining what they can do (guides), verifying what they did (sensors), and maintaining auditability. Think of it as DevOps + AI governance.

We Build Harnesses

If your organisation needs AI coding agents governed to your standards, whether you are a bank that needs SOX compliance, an insurance company under Solvency II, a healthcare provider under HIPAA, or an e-commerce platform protecting revenue, we build the harness.

We do not sell the AI model. We build the governance around it: CLAUDE.md, AGENTS.md, CI quality gates, PR review automation, branch protection, post-merge monitoring, and the audit trail that lets you explain to a regulator exactly how AI-generated code reaches your production environment. This is the AI agent harness enterprise buyers can put in front of an auditor.

Start with an AI Readiness Assessment to evaluate your current governance posture. Or talk to us about building a harness for your specific compliance requirements.

The model writes the code. The harness makes it safe. We build the harness.

Follow First AI Movers on LinkedIn for practical AI frameworks and decision guides.

Harness Engineering: How to Build Quality Gates from PR to Production for AI Coding Agents