Your Refactor Needs an Oracle: Characterization Tests vs. AI Diffs

AI-generated refactors fail silently. The code looks clean. The tests pass. Then a production edge case breaks. Characterization tests catch that break before it ships. This workflow locks current behavior first, then lets a free model propose changes, then uses tests as the oracle.

The Problem

Legacy code has undocumented quirks. Humans miss them. Models miss them too. A refactor that changes a single boundary condition is a regression waiting to happen. You need a way to say "this diff preserves behavior" with evidence.

Characterization tests provide that evidence. They capture what the code does today, not what it should do. They freeze the behavior you are about to touch.

The Workflow

Step 1: Lock the Current Behavior

Write tests that document the current output for known inputs. Include weird cases: zero, negative, undefined, boundary values.

Here is a legacy function with nested conditions:

// legacy.js
export function applyDiscount(total, user) {
  if (user.type === 'vip') {
    return total * 0.9;
  } else {
    if (total > 100) {
      return total * 0.95;
    } else {
      return total;
    }
  }
}

Now write tests that lock its actual behavior:

// applyDiscount.test.js
import { describe, it, expect } from 'vitest';
import { applyDiscount } from './legacy.js';

describe('applyDiscount current behavior', () => {
  it('applies 10% for vip regardless of total', () => {
    expect(applyDiscount(99, { type: 'vip' })).toBe(89.1);
  });

  it('applies 5% for non-vip over 100', () => {
    expect(applyDiscount(101, { type: 'guest' })).toBe(95.95);
  });

  it('no discount at 100 or below', () => {
    expect(applyDiscount(100, { type: 'guest' })).toBe(100);
  });

  it('no discount for zero total', () => {
    expect(applyDiscount(0, { type: 'guest' })).toBe(0);
  });
});

Run it. Make sure it fails on an unmodified repo? No. It should pass and lock the behavior.

Step 2: Ask a Model for a Refactor

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I used MonkeyCode's free model access to request a refactor. The prompt was simple: "Refactor applyDiscount to reduce nesting. Keep the exact same observable behavior."

The model returned guard clauses:

export function applyDiscount(total, user) {
  if (user.type === 'vip') return total * 0.9;
  if (total > 100) return total * 0.95;
  return total;
}

This looks fine. But you are not going to trust the look. You are going to run the tests.

Step 3: Run Tests on a Free Server

Run the suite on an isolated server instead of your laptop. MonkeyCode's free server option can execute the test command. That keeps your local environment clean and reproducible.

Create a small verification script:

#!/usr/bin/env bash
set -euo pipefail

git fetch origin
CHANGED_FILES=$(git diff --name-only origin/main)

echo "Changed files:"
echo "$CHANGED_FILES"

npm install
npm test

Save it as verify_refactor.sh and run it on the server. The script does three things: fetch latest, show the diff surface, and run the full test suite.

Step 4: Apply a Decision Table

The test result plus diff size drives the decision.

If tests fail, do not debug the model output. Instead, find which characterization test broke. That tells you exactly which behavior changed. Go back to Step 2 with a more specific prompt.

Limitations

Characterization tests are not a proof. They only cover inputs you thought about. The model can change behavior on untested inputs. You still need human review of the diff.

This approach is not for safety-critical code. It is not for code requiring formal verification. It is also not for teams that cannot run a test suite in a clean environment.

Who Should Not Use This

Do not use this workflow if you lack any test runner. Do not use it if your legacy module is too tangled to import. And do not use it to skip code review. The oracle saves you from silent regressions, not from thinking.

The Bottom Line

AI accelerates refactoring. Acceleration is not a correctness guarantee. Characterization tests turn "the diff looks safe" into "the diff is safe for the inputs we know." Free model access and a free server make this workflow cost-effective. But nothing happens until you run the tests.

Run them.