About
About the Observatory
An open reference for the signals software emits — not "code quality metrics," but epistemic sensors: measurement instruments pointed at different failure modes.
The problem
Software is increasingly an opaque artifact. We cannot — and increasingly do not want to — fully understand every implementation. Code is produced by agents, by teams we'll never meet, by systems that span services we don't own. The question is no longer "is this code good?" The question is: what independent observations would cause us to update our belief that this software is correct?
Epistemic sensors, not quality metrics
We would not call these "quality metrics." We'd call them epistemic sensors. Their job is to reduce uncertainty about a system that we cannot — or increasingly do not want to — fully understand.
The framework
The catalog is organized into 10 families of sensors, each asking a different question about the system. Every sensor is characterized along six dimensions: oracle strength, independence, scope, feedback latency, actionability, and predictive vs retrospective. The atlas arranges them as a navigational matrix — families on one axis, confidence stack layers on the other.
Inspirations
Birgitta Böckeler's "guides & sensors" framing. Sensors are tools that give an agent feedback about what it has done. The interesting frontier is guiding sensors, where the feedback itself tells the agent what to do next.
Honeycomb's conception of observability. Don't merely collect predetermined health metrics; preserve enough information to ask questions you didn't know you would need to ask.
Maintained by
The Software Observatory is built and maintained by Justin Abrahms — a principal engineer working on agent-assisted software delivery and the sensors that let us assess what agents produce. The catalog grows out of that work: if agents can produce software faster than humans can inspect it, code review cannot remain the primary mechanism for establishing quality, and we need a living catalog of the signals we can use instead.
Reach out via email, Bluesky, or GitHub. Errors and gaps in the catalog are best filed in the issue tracker.
Contributing
The Observatory is meant to be an open reference, and the field should be able to correct it. The catalog applies a producer-evaluator principle to itself: the author shouldn't be the only evaluator. Independent review of the content is welcome and explicitly invited.
The best way to contribute right now:
- File an issue at the GitHub tracker for a factual error, a missing sensor, a taxonomy challenge, or a depth gap.
-
Open a pull request with a new or revised
content/sensors/*.md. Keep the opening paragraph self-contained; match the frontmatter shape in CONTRIBUTING.md; run the build and the link checker before submitting. -
Propose a family change (adding, renumbering, or
reclassifying a family) via an issue first — these touch
FAMILIESinbuild.pyand the color tokens incss/observatory.css, so they're worth discussing before the work.
See CONTRIBUTING.md in the repository for the full guide — frontmatter shape, the relative-link gotcha, build and link-check commands, and style conventions.
License
All content on the Software Observatory is published under CC BY-SA 4.0. You are free to share and adapt the material for any purpose, provided you give appropriate credit and distribute contributions under the same license.