• 7 min • Agent Xero

The Coder-in-the-Loop Checklist

How to work with AI tools without accumulating technical debt or shipping vulnerabilities.

What is Coder-in-the-Loop?

Coder-in-the-loop means using AI as a tool—not a replacement. The human remains accountable for architecture, security, and maintainability.

The Problem with “Vibe Coding”

AI can generate working code fast. But:

  • It doesn’t validate edge cases.
  • It doesn’t consider security implications.
  • It doesn’t design for maintainability.
  • It doesn’t understand your specific constraints.

If you ship AI-generated code without review, you’re accumulating invisible debt.

The Checklist

Use this checklist every time you merge AI-generated (or AI-assisted) code.

1. Security Review

  • Input validation on all user-facing endpoints
  • Authentication and authorization implemented correctly
  • No secrets or API keys hardcoded
  • SQL injection protection (parameterized queries)
  • XSS protection (sanitize outputs)
  • CSRF tokens on state-changing requests

Red flag: If AI generated auth code, assume it’s wrong until proven otherwise.

2. Error Handling

  • All async operations have error handling
  • User-facing errors don’t leak internal details
  • Errors are logged with sufficient context
  • Edge cases (network failure, timeout, invalid state) are handled

Red flag: Code that assumes “happy path” only.

3. Type Safety

  • TypeScript strict mode enabled (if applicable)
  • Zod (or similar) validation on external inputs
  • No any types (or explicitly justified)
  • Database schema matches application types

Red flag: Liberal use of as unknown as T or @ts-ignore.

4. Performance

  • No N+1 queries
  • Database indexes on queried fields
  • Pagination on list endpoints
  • Rate limiting on public APIs

Red flag: AI code often doesn’t optimize for scale.

5. Testing

  • Critical paths have test coverage
  • Auth flows are tested
  • Error cases are tested (not just success paths)

Red flag: AI generates code, not tests. You write those.

6. Observability

  • Structured logging (JSON, not console.log)
  • Key operations are instrumented (auth, payments, errors)
  • Alerts configured for critical failures

Red flag: No visibility into production behavior.

7. Deployment

  • Environment variables documented
  • Rollback plan exists
  • Database migrations are reversible
  • Secrets are rotated post-deployment (if exposed during development)

When to Get Human Review

Use this heuristic:

  • Ship without review: UI tweaks, copy changes, non-critical refactors
  • Peer review: New features, architectural changes, database migrations
  • Expert review: Auth, payments, compliance-sensitive code

Agent Xero sessions are designed for that “expert review” layer.

Tools We Recommend

  • Zod: Runtime validation
  • Sentry: Error tracking
  • Playwright: End-to-end testing
  • OpenTelemetry: Observability
  • Dependabot: Dependency updates

Need a second pair of eyes? Book a code review session →