Glossary — Software Observatory

Epistemic sensor

A measurement instrument pointed at a failure mode. The term is deliberately not "quality metric": a metric aggregates or scores, while a sensor reduces uncertainty about a specific property of the system. A compiler is a sensor of structural validity; mutation testing is a sensor of test sensitivity; observability events are sensors of what actually happened. Each sensor measures one thing — no single sensor measures correctness. The catalog is organized into 10 families of epistemic sensor, each asking a different question about the system.

Opaque artifact

Software we cannot — or do not want to — fully understand by reading. Code is increasingly produced by agents, by teams we'll never meet, and by systems that span services we don't own. The question for an opaque artifact is not "is this code good?" but "what independent observations would cause us to update our belief that this software is correct?" The Observatory is a catalog of those observations.

Oracle

An oracle is the thing that tells you whether a given behavior is correct. A compiler error is a perfect oracle of structural validity — the implementation cannot argue with it. A test assertion is a strong oracle for the specific case it checks. A complexity metric is a weak oracle — high complexity doesn't prove anything is wrong. Oracle strength is one of the six dimensions every sensor is characterized along.

Oracle strength

How confidently a sensor knows that something is wrong. The scale runs from maximum (a compiler error — the code cannot argue) to low (a complexity metric — it suggests risk but proves nothing). See the framework page for the full ranking.

Independence

Whether the thing being evaluated can manipulate the sensor. A model writing tests/ is allowed to write tests that make itself pass — that's low independence. A compiler is maximum independence — the code cannot talk its way past a type error. Independence is especially important for AI-generated code: the producer and the evaluator should be separated wherever possible. See the framework and second-agent review.

Scope

What level of the system the sensor tells you about: a single line, a function, a module, a service, the whole system, or a user journey. A type checker has function-level scope; observability events have system-level scope. See the framework.

Feedback latency

How long until the sensor tells you something. A compiler reports in milliseconds; an escaped-defect-rate sensor reports in months. Latency determines where in the lifecycle a sensor is useful — you can't gate a merge on a signal that takes weeks. See the framework.

Actionability

Whether a sensor merely flags a problem or tells you what to fix. Three values: blocking — a binary gate that halts the pipeline (compiler error, invariant gate); exploratory — a signal to investigate that narrows where to look but prescribes nothing (hotspot, trace, coverage gap); guiding — the feedback itself directs the next action (mutation report shows the untested mutation, linter names the rule and fix). See the framework.

Evidence label

A label assigned to each publication reference, describing the study rather than the claim: controlled study (controlled experiment or large-N study with a comparison group), observational study (observational study on production data, no control group), case study (single-organization case study or engineering report with numbers), argument (experience report — not measured, but rendered visibly as unmeasured). The label tells you what kind of evidence backs the claim, so you can weigh it accordingly.

Guiding sensor

A sensor whose feedback directs the next action, not just whether something is wrong. A mutation testing report shows the exact untested mutation — the agent knows what to write a test for. A complexity score just says "this is complex" and leaves the agent to figure out what to do. The distinction comes from Birgitta Böckeler's "guides & sensors" framing.

Predictive vs retrospective

Whether the sensor fires before the code ships (predictive — a compiler error before merge) or after (retrospective — revert rate, incident correlation). This is when the signal arrives, not what kind of feedback it gives — that is actionability. The two are correlated but not the same: a retrospective sensor can still gate (build provenance blocks an unattested artifact). See the framework.

Confidence stack

The layers of evidence that accumulate as code moves from authoring to production: compilation, types, tests, mutation, integration, canary, production events, outcomes. No single layer is sufficient; the combination constrains uncertainty from multiple directions. The atlas arranges the stack as a navigational matrix.

Metamorphic testing

A testing technique where you don't know the correct answer, but you know how the answer should change when the input changes. You may not know what sqrt(2) is, but you know sqrt(x * 4) == 2 * sqrt(x) must hold. You don't need a specified oracle; the relation is a partial one. See the entry.

High cardinality

A property of observability events: each event carries enough distinct fields (user_id, cart_id, order_id, deployment, git_sha) that you can slice the data along dimensions you didn't know you'd need. The opposite of pre-aggregated metrics, which answer only predetermined questions. See the entry.

No single sensor measures correctness

A refrain that recurs across the homepage, the framework, and this glossary — deliberately. The repetition is the point: the Observatory's central claim is that correctness is not a scalar any one sensor measures, and stating it once would understate it. Each occurrence links back here so a reader who notices the repetition can verify it is intentional.