Catalog › Behavioral › Contract Tests

Contract Tests

Behavioral High oracle Predictive

Does service A continue satisfying the assumptions of service B? Contract tests are a sensor of boundary assumptions — the implicit agreements between independent components about what the interface between them promises.

Contracts measure boundary assumptions

A contract is a specification of what one component promises to provide and what another component is allowed to assume. Contract tests verify that both sides of the boundary continue to honor these promises as the system evolves.

# Consumer-driven contract
Service B (consumer) defines:
  "I expect GET /users/{id} to return
   { id: int, name: string, active: bool }"

Service A (provider) must satisfy:
  its response matches this shape
  the field types are correct
  the endpoint exists and responds

The sensor detects breaking changes — when a provider modifies its API in a way that violates the assumptions its consumers depend on. This is particularly powerful in microservice architectures where independent teams deploy independently.

Contracts measure boundary assumptions. Types measure a particular class of structural inconsistency. Contract tests are the boundary-level version of what type checkers are at the module level: both enforce structural promises, but at different scopes and different feedback latencies.

In practice

A reading is a pact verification failure that names the broken clause and the consumer it belongs to:

Verifying a pact between Orders (consumer) and Billing (provider)

  A request for invoice 42
    GET /invoices/42
      returns a response which
        has status code 200
        includes headers
          "Content-Type" with value "application/json"  FAILED
            actual value: "text/html"
        has a matching body                             FAILED
          $.amount: expected number, got string

2 interactions, 2 failures

Reading it well:

  1. Read the clause, not just the red. Status, header, and body failures have different owners and different fixes. The clause names which promise broke.
  2. The failure belongs to the change, not the expectation. The consumer stated its assumption first; the provider moved. The fix direction is usually to hold the provider to the promise, not to relax the expectation.
  3. A passing pact is one-directional evidence. It says the provider honors the recorded expectations. It does not say the expectations are complete: a consumer that never wrote a pact is a gap the sensor cannot see.

How it gets gamed

  • Delete or weaken the pact. Removing an expectation the provider now violates turns red to green without fixing the provider. The consumer still holds the old assumption, so the breakage moves from the report to production.
  • Verify against a stub. Running the provider side of a pact against a mock instead of the real service passes the check and tests nothing.
  • Let pacts go stale. A pact not re-verified after a provider change is an assumption wearing a check mark.

The meta-signal is the pact count per consumer over time. A pact file that shrinks after a failure is the sensor being overridden.

Response playbook

When a contract test fails:

  1. Read the broken clause. Status, header, and body failures have different owners. The clause names which promise broke and which consumer depends on it.
  2. Assume the provider is wrong until shown otherwise. The consumer stated its assumption before the provider changed. The default fix is restoring the promise, not relaxing the pact.
  3. If the consumer is wrong, change both sides together. Update the pact and the consumer in the same change, so the window where neither matches is zero.
  4. Check the other consumers. A broken clause usually breaks every consumer that holds it. The report names one; the pact broker lists the rest.

What it cannot detect

Contract tests verify the shape of communication, not its meaning. A response that has the right fields and types but contains wrong values will pass a contract test. They also cannot detect integration failures that emerge from the interaction of correct components producing incorrect emergent behavior.

Structural Type Checker Invariants Runtime Invariants Family Change Family Structural
Categories: Behavioral Boundary Assumptions

References

Publications

Tooling