The Catalog
Sensor Catalog
A catalog of epistemic sensors — the observable signals that increase our confidence that a system is correct, maintainable, and behaving as intended. Organized into 10 families. Each entry documents what the sensor can detect, what it cannot detect, how easily it can be gamed, and what evidence it produces.
Build Provenance & SBOM
Is the artifact you are about to deploy structurally the one your pipeline built? A software bill of materials plus build provenance attestation (SLSA-style) answers: these sources, these…
Compiler
Does this code compile? The cheapest and most certain sensor in the catalog. A compiler doesn’t just check syntax — it resolves imports, validates module structure, and produces an artifact.
Contract & Refinement Types
Behavioral guarantees checked before the code ever runs. Typestate systems, refinement types, and design-by-contract annotations (Eiffel-style pre/postconditions, Dafny requires/ensures) ask the…
Linter
Catches structural inconsistencies that are syntactically valid but semantically suspect. Lower oracle strength than a type checker, but faster feedback on style and common traps.
Model Checking
Exhaustive exploration of a system’s reachable states against a temporal property. Where a type checker proves a property holds for one step, and statically checked invariants prove a property holds…
Schema Validator
Structural coherence at the boundary of the system. OpenAPI/GraphQL schema validation, Terraform plan, Kubernetes admission validation, SQL parser/type checker — all of these answer: is this a valid…
Static Analysis
Pattern-matching and dataflow analysis over the source without executing it. Covers the space between a linter (style and local patterns) and a type checker (type soundness): null dereferences…
Theorem Proving
A machine-checked proof that a property holds for all inputs, not just the ones a test happened to exercise.
Type Checker
A Rust compiler saying expected Option<Foo>, found Foo is vastly more useful to an agent than “please reconsider whether this is correct.” This is the first and strongest form of computational…
Contract Tests
Does service A continue satisfying the assumptions of service B? Contract tests are a sensor of boundary assumptions — the implicit agreements between independent components about what the interface…
Example-Based Tests
Given X, expect Y. The fundamental behavioral sensor. Fundamentally different from coverage: coverage says “this code executed.” A behavioral assertion says “this code produced the right result.”…
Integration Tests
Does the thing work when connected to its actual dependencies? Catches failures that unit tests structurally cannot see — connection failures, serialization mismatches, timeout behavior.
Smoke Tests
The cheapest behavioral check against a live deployment: hit /health, create one record, read it back, delete it. If any step fails, roll back.
Snapshot Tests
Did observable output change? Not “is it correct” but “did it change” — a sensor for detecting unintended drift in externally observable artifacts.
Synthetic Monitoring
Scripted user flows run against the production system around the clock: log in, search, add to cart, check out.
Branch Coverage
Did we exercise both sides of decisions? Better than line coverage. Path coverage is better still, but usually expensive and impractical at scale.
Diff Coverage
You don’t necessarily care whether some 15-year-old module has 43% coverage. You care: did the code I just changed have evidence attached to it?
Escaped Defect Rate
Of the bugs that reached users, which ones should the test suite have caught? Escaped defect rate is the slowest and most honest measure of test effectiveness: not “would the tests catch a…
Line Coverage
Did we execute this line? Useful but weak. A project can have 90% line coverage while mutation testing finds large numbers of mutations that tests don’t detect. Coverage measures execution.
Mutation Testing
Take if user.is_admin: allow() and mutate it to if not user.is_admin: allow(). If all your tests still pass, your tests did not actually establish the behavior you thought they established.
Business Invariants
A successful payment implies an order eventually becomes paid. order.total == subtotal + tax + shipping - discounts.
Database Invariants
Every foreign key refers to an existing object. Every request has exactly one request_id. created_at <= updated_at.
Pre-Promotion Invariant Gates
Invariants checked at the moment of promotion, before a change can reach users: migrations must be backward-compatible, no PII column may be added without an encryption flag, the new schema must…
Runtime Invariants
You don’t need to know how the payment service works. You can observe: “5,213 payments occurred; 17 have no corresponding order transition.” That’s a sensor of correctness without understanding the…
Statically Checked Invariants
Invariants the compiler refuses to let you violate: Dafny invariant clauses, Frama-C annotations, JML specs, type-level witnesses like NonEmptyList.
Differential Testing
implementation_A(input) == implementation_B(input). You don’t know which is right. But disagreement is an excellent sensor.
Fault Injection
Kill a node. Drop a network connection. Inject latency. A sensor of resilience — does the system continue to satisfy its invariants under partial failure?
Fuzzing
What happens on inputs humans didn’t think of? Fuzzing is a sensor of robustness against the infinite space of inputs the system will actually encounter — including inputs no engineer would ever…
Live Chaos Experiments
Fault injection against the running production system: kill a node, sever a region, corrupt a fraction of messages, and watch whether runtime invariants hold.
Metamorphic Testing
You don’t know the answer, but you know how the answer should change. This is a particularly beautiful sensor because you don’t need an oracle.
Property-Based Testing
You state a property that should hold for every input, and the tool generates inputs trying to break it.
Static Security Analysis
Attacking the code before it runs. Taint tracking, dataflow analysis, and pattern-based scanners (Semgrep, CodeQL) ask: “is there any path through this program where an adversary’s input reaches a…
Continuous Profiling
Where did computation actually go? Not “CPU is 82%” but “this function consumed 40% of the time in these specific requests.” A sensor of resource reality.
Distributed Traces
What path did this particular operation take? A sensor of execution flow across service boundaries — the span tree is itself a signal.
Load Testing
Does the system behave under the traffic it claims to handle? A load test drives generated requests at a target rate and measures what breaks — latency, errors, throughput — before real users do.
Observability Events
Traditional monitoring says: “CPU is 82%.” Observability says: “Show me the requests that are slow, and let me figure out what those requests have in common.” The distinction is fundamental.
Resource Telemetry
CPU, memory, IO, network, GC, queues. Traditional monitoring — useful but limited. Low cardinality, low dimensionality, predetermined questions. Resource telemetry is the weakest runtime sensor.
A/B Testing
Did users actually behave differently? An A/B test splits traffic between two versions and measures whether the treatment changes a user outcome metric — click-through, conversion, retention…
API Compatibility
Did externally observable contracts change? A sensor of boundary stability — can old and new versions coexist?
Canary Analysis
Does the new version behave differently from the old version? A sensor of behavioral drift between deployments, measured on real traffic.
Error-Budget Impact
Did this change consume an abnormal amount of reliability budget? A sensor that directly ties code changes to user-visible impact.
Feature Flag Exposure Telemetry
A change behind a flag is not a change anyone has experienced. Flag evaluation streams answer “who is actually seeing the new behavior, right now?” — the difference between deployed and live, and the…
Incremental Build Correctness
Did the build system actually rebuild everything this change touched? Incremental builds and remote caches save hours, and silently shipping a stale artifact is the price when the dependency graph…
Shadow Traffic
Run the new implementation against real inputs without affecting users. A sensor that produces differential evidence with zero user risk.
Boundary Sensors
“This package must not import that package.” A sensor of encapsulation and module boundaries — computationally enforced, not prose rules.
Dependency Graph
Fan-in, fan-out, cycles, dependency depth, unstable dependencies. A sensor of structural coupling between modules.
Architecture Fitness Functions
frontend -> application -> domain -> infrastructure, and fail if domain -> infrastructure. A sensor of architectural drift.
Hotspot Analysis
Change frequency times complexity. Identifies places where the system is simultaneously difficult and frequently changed — more interesting than “files with the most lines.” Hotspot analysis combines…
Live Service Graph Discovery
The declared architecture says service A never calls service C. The live service graph — discovered from actual traffic via a service mesh, eBPF flow mapping, or trace aggregation — says whether that…
Change Coupling
Which files repeatedly change together? A sensor of hidden coupling — the repository itself becomes a sensor, no code reading required.
DORA Metrics
Change lead time, deployment frequency, failed deployment recovery time, change fail rate, deployment rework rate.
Incident Correlation
Which components correlate with production failures? A sensor of operational risk concentration, measured from observability events.
Revert Rate
How often does this area get reverted? A black-box sensor of maintainability — you don’t need to understand FooManagerFactoryImpl, you can observe: 27 changes in six months, 8 reverts, 4 incidents…
Time-to-Repair
When this component breaks, how long does it take to restore? A sensor of maintainability measured in hours, not in subjective assessment.
Human Comprehension
"Can another observer understand and challenge this?"
Decision Provenance
Can you answer “why is this weird thing here?” A sensor of archaeological accessibility — can you determine why code exists, not just what it does?
Documentation Drift
Does documentation still predict behavior? A sensor of the gap between what the system is documented to do and what it actually does. Documentation drift measures whether the docs match reality.
Independent Review
Can another engineer explain what this does? A sensor of epistemic accessibility — if humans can’t understand the system, that’s itself a correctness risk.
Onboarding Experiment
How long does it take a competent engineer to safely modify this subsystem? A sensor of knowledge concentration measured in onboarding time.
Second-Agent Review
An independent agent reviews the first agent’s output. A sensor that applies adversarial pressure to AI-generated code without human-in-the-loop latency.