Overseer testing protocol

The standalone home of Overseer testing-control v3. The Overseer names in headers and JSON identify the protocol. Frontend tests, backend tests, curl and CI can all use it without installing a particular test runner or application.

The protocol gives tests a common way to prepare synthetic data, call the application's normal API, and inspect the result. The backend defines the available operations and their business meaning.

The specification, OpenAPI, JSON Schema and fixtures define the wire contract. The implementation guide explains states, spies, probes and sinks. Product capability names and their schemas belong to each backend.

Route relative to the testing base path	Purpose
GET /capabilities	Discover supported setup, observations and limits
PUT /reset	Restore the declared baseline or require a restart
POST /states	Arrange named synthetic preconditions
POST /probes	Read a safe projection, including an internal spy
POST /sinks/query	Read captured fake effects without consuming them

Using the protocol in an application

For most backend developers, the main entry point to this gem is the overseer-testing-conformance command: run it against an implemented testing API in your local tests or CI.

You can implement that API yourself using the specification. A Rails app can instead use the optional overseer-testing-control-rails helper for routing, JSON validation, correlation and bounded storage. The application still owns its setup handlers, read-only observations, fake providers and reset behavior. The helper uses the protocol's Ruby methods internally; application handlers normally do not need to call Overseer::TestingControl::ProtocolV3.

Direct use of those methods is useful when writing your own adapter. They validate incoming and outgoing JSON and construct protocol envelopes. Request validation runs before a handler changes data; the command checks the HTTP behavior exercised by its tests. One does not replace the other.

Adding the protocol gem alone creates no Rails routes, migrations, model hooks or fake providers. A custom backend can follow the specification without loading this gem and install the command only in its test tooling.

The optional Ruby package overseer-testing-protocol requires Ruby 3.2 or newer. Its only runtime dependency is json_schemer. Other languages can use the data artifacts without Ruby or a framework helper. Package version 0.1.0 carries wire version 3; these are separate versions.

See the implementation guide for an example test flow, the command invocation, and its current coverage limits. Passing the command is a baseline check, not proof that every protocol requirement or business operation has been tested.

Working on this package

bundle install
bundle exec rake test
gem build overseer-testing-protocol.gemspec
bin/overseer-testing-conformance --help

The caller owns test-server startup, isolation and cleanup. The command checks an already isolated loopback server, requires explicit expected identity plus --allow-reset, and uses finite timeouts and bounded reads. Release consumers should pin the package or a reviewed source commit. A local package build does not publish a release.

Made with ❤️ by olistik