Valid JSON, Broken Contract: Layered Checks for Payload Regressions

A response can arrive with HTTP 200, parse without a single error and still break your application. Syntax validation only proves that text is JSON; it says nothing about whether the payload still matches what your code expects. This walkthrough builds a small, layered regression check (semantic comparison, a focused schema and explicit business rules) so that a contract break surfaces as a precise message instead of a strange bug downstream.

A payload that parses and still fails

Consider an order service. The endpoint responds successfully, the JSON parses, and then the code that adds up item quantities produces "021" rather than 3. The scenario is invented, but the failure mode is common: in a crowded payload, the string "2" and the number 2 are easy to miss, both are legal JSON, and only one of them honors the contract.

The fixture below represents the incoming order. Look closely at the first item: its quantity is a quoted string, while the second item uses a plain number. Everything else is plausible order data, including a null shipping instruction and a precomputed totalCents.

{
  "orderId": "ORD-1042",
  "createdAt": "2026-09-21T09:30:00Z",
  "customer": {"id": "C-017", "name": "Avery Example", "vip": false},
  "items": [
    {"sku": "MUG-BLUE", "quantity": "2", "unitPriceCents": 1250},
    {"sku": "NOTE-A5", "quantity": 1, "unitPriceCents": 650}
  ],
  "shipping": {"city": "Exampleton", "instructions": null},
  "paid": true,
  "totalCents": 3150
}

A strict syntax check accepts this document without complaint, and that is a legitimate result: the text conforms to the JSON grammar in RFC 8259, where strings, numbers, booleans and null are all valid values. What the grammar cannot know are the rules your application layers on top:

• quantity has to be an integer greater than zero
• paid has to be a boolean
• the sum of each quantity multiplied by its unit price has to equal totalCents

Assume the previous version of this order differed in exactly two places: quantity was the number 2 and paid was false. The payment flag flipping to true is an expected state change. The quantity turning from a number into a string is the regression.

Diff values by path, not lines of text

There are two very different questions you can ask when comparing payloads. A semantic comparison parses both documents and compares values at each path. It treats object key order as irrelevant, so swapping the position of customer.id and customer.name is not a change, while it keeps array order significant, so moving an item to another index can be. For this fixture, a semantic diff reports exactly two paths: $.items[0].quantity and $.paid. That answers the question "what data changed?"

A text diff answers a different question: "which characters or lines changed?" It is still useful evidence, and normalizing indentation and formatting on both sides first removes much of the noise. But matching or differing text is not proof that two documents do or do not represent the same JSON value, because whitespace, key order and number formatting can all vary without changing meaning.

Reproducing the checks in plain Node

You do not need any external service to verify these facts. The script below reads order.json from its own directory (via import.meta.url), reconstructs the previous version with structuredClone, and asserts the differences using node:assert/strict. It then demonstrates the bug, fixes the fixture on a copy and checks the business rule that the computed total matches totalCents. Save it as check-order.mjs next to the fixture and run node check-order.mjs.

import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
const incoming = JSON.parse(
  readFileSync(new URL('./order.json', import.meta.url), 'utf8')
);
const before = structuredClone(incoming);
before.items[0].quantity = 2;
before.paid = false;
assert.equal(before.items[0].quantity, 2);
assert.equal(incoming.items[0].quantity, '2');
assert.equal(before.paid, false);
assert.equal(incoming.paid, true);
const naiveCount = incoming.items.reduce(
  (sum, item) => sum + item.quantity,
  0
);
assert.equal(naiveCount, '021');
const corrected = structuredClone(incoming);
corrected.items[0].quantity = 2;
const count = corrected.items.reduce((sum, item) => sum + item.quantity, 0);
const computedTotal = corrected.items.reduce(
  (sum, item) => sum + item.quantity * item.unitPriceCents,
  0
);
assert.equal(count, 3);
assert.equal(computedTotal, corrected.totalCents);
console.log('Contract checks passed');

The naiveCount assertion is the interesting one. The reduction starts from the number 0, adds the string "2" and gets the string "02", because + switches to concatenation whenever either operand is a string. The next step appends 1, yielding "021". A numeric initial value does not coerce later operands; once a string enters the accumulator, everything after it is concatenated.

Note what the script does not recommend. Patching the fixture to a number is a way of expressing the correct contract, not a hint to wrap every incoming field in Number(). Silent coercion hides producer bugs. Either fix the service that emits the string or define an explicit, documented normalization rule at the boundary where data enters your system.

Turning the type rule into a schema

Prose rules are easy to forget; executable ones are not. The Draft 2020-12 schema below encodes the expectations that matter for this regression. It requires orderId, items, paid and totalCents, demands at least one item, and constrains quantity to an integer with a minimum of 1 and unitPriceCents to a non-negative integer.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "required": ["orderId", "items", "paid", "totalCents"],
  "properties": {
    "orderId": {"type": "string"},
    "items": {
      "type": "array",
      "minItems": 1,
      "items": {
        "type": "object",
        "required": ["sku", "quantity", "unitPriceCents"],
        "properties": {
          "sku": {"type": "string"},
          "quantity": {"type": "integer", "minimum": 1},
          "unitPriceCents": {"type": "integer", "minimum": 0}
        }
      }
    },
    "paid": {"type": "boolean"},
    "totalCents": {"type": "integer", "minimum": 0}
  }
}

Validating the incoming fixture against this schema fails with a message along the lines of $.items[0].quantity: expected type integer. Changing that single value to the number 2 makes validation pass. That pinpointed path is exactly what you want from a regression check.

Know what your validator actually covers

Validators differ in how much of the specification they implement. Some deliberately skip resolving remote $ref targets, evaluating pattern regular expressions or asserting format keywords, and by default the specification treats format as an annotation rather than an assertion. Make sure the validator you use in tests covers the same keywords your production contract relies on, otherwise a schema can look stricter than the checks that enforce it.

Boundary cases worth a test each

A good contract rejects bad inputs for distinct reasons. Each of these should fail on its own:

• items missing entirely (a required violation)
• items as an empty array (minItems)
• a quantity of zero (minimum)
• paid sent as the string "true" (type)

Equally important is what should pass: an unpaid order with paid: false. The required keyword only checks that the property is present, not that its value is truthy. The official JSON Schema guides explain how required relates to object properties and how each array element is checked against the items subschema.

What the schema intentionally leaves out

This is a targeted schema, not a full order specification. Customer details, shipping, timestamp semantics and extra fields are unconstrained. The arithmetic rule is also out of scope on purpose: whether line items add up to totalCents is application logic that standard JSON Schema cannot express, which is why the Node script asserts it separately. Layering the two keeps each tool doing what it is good at.

Generated types describe one sample

It is tempting to paste a payload into a JSON-to-TypeScript generator and treat the output as the contract. Run on the corrected fixture, such a generator yields things like quantity: number, createdAt: string and instructions: null inside the nested types. Those are accurate observations of this one sample and nothing more:

• JSON has a single number type, so number cannot say "positive integer".
• A single example cannot reveal which fields are optional.
• A field that happens to be null today may legitimately hold a string tomorrow.
• TypeScript types vanish at runtime and never inspect incoming bytes; the TypeScript handbook on type assertions states plainly that assertions perform no runtime checking.

If you want a single definition that drives both runtime validation and static types, see our article on sharing one Zod schema across frontend and backend.

Precision belongs at the boundary too

Numbers raise one more trap. JavaScript parses {"id":9007199254740993} into 9007199254740992, because the value exceeds Number.MAX_SAFE_INTEGER and is rounded to the nearest representable double. If an identifier must keep every digit, the producer's contract should define it as a string. Trying to repair it after JSON.parse is too late; the digits are already gone.

Key takeaways

• Keep a sanitized fixture of a known-good payload and never use real credentials or personal data in test data.
• Treat checks as layers: strict syntax, semantic value diff, schema validation, then business rules the schema cannot express.
• Generate TypeScript types only after the contract is settled, and never mistake them for runtime validation.
• Aim for failures that name a path and a cause, such as "$.items[0].quantity changed from integer to string, so validation fails and addition concatenates", rather than "the order looks wrong".