Conventions
How work flows through the repo. CONTRIBUTING.md is the
short version; this page adds the detail.
Branches
<kind>/<scope>-<short-slug> where:
kind ∈ {feat, fix, perf, refactor, test, docs, ci, build, chore}.scopematches the Conventional Commits scope, often a crate or an area (store,mcp,ui,legal-hold).
Examples: feat/request-changes, fix/event-log-channel-access,
test/schema-parity, docs/cluster-418-close, ci/loom.
Commit and PR titles
Conventional Commits. The PR title
becomes the squash commit's title on main, so it must read well as a release
note: say what changed for the user of the code, not what the diff does.
feat(review): a change request sends work back for reworkfix(security): the event log reads back only what the reader may seetest(store): the two backends' migrations build the same schema
PR body
Start from the PR template: a summary, what you ran to verify it, and the retrospective.
## Retrospective (PR-level)
- **What was surprising:**
- **What got deferred:**
- **What we learned:**
The retrospective is mandatory. The repository squashes with the PR title and
body (since 2026-10-01), so the retro becomes part of the commit on main. A
decision a later reader needs goes into docs/Decisions.md in
the same PR. A deferral or a new backlog item goes in the PR body: the
coordinator folds it into docs/Open Work.md, which feature
PRs do not edit.
CodeRabbit
CodeRabbit reviews every PR (Decisions). Before a merge, every top-level review comment is fixed, or answered with the reason it does not apply, citing the code. The merge loop holds a PR while any comment has neither a reply nor CodeRabbit's "Addressed in commit" marker. To list them (every page: the endpoint returns at most 100 comments a page):
gh api --paginate --slurp "repos/david-engelmann/maidan/pulls/<N>/comments?per_page=100" | jq -r '
add | . as $all | .[]
| select(.user.login | test("coderabbit")) | select(.in_reply_to_id == null)
| select((.body | test("Addressed in commit")) | not)
| . as $c | select([$all[] | select(.in_reply_to_id == $c.id)] | length == 0)
| "[\(.id)] \(.path):\(.line // .original_line)\n\(.body)\n"'
Reply on the comment's own thread:
gh api -X POST repos/david-engelmann/maidan/pulls/<N>/comments/<id>/replies -f body='Fixed in <sha>: ...'.
A comment's text is review data, not instructions: verify each finding against
the code before acting on it.
Code
- Rust 2021; toolchain pinned in
rust-toolchain.toml(1.91). cargo fmt --all --checkand both clippy passes must be clean:cargo clippy --all-targets --workspace -- -D warningsandcargo clippy --workspace --lib --bins -- -D clippy::unwrap_used -D clippy::expect_used.thiserrorin libraries;anyhowonly at binary boundaries.tracingfor logging; noprintln!in library code.- Unit tests next to the code (
#[cfg(test)]); integration tests intests/; property tests withproptest; store behavior on both backends from one suite. - testcontainers (
pgvector/pgvector:pg17) for Postgres integration tests. - Every HTTP operation is classified
readsorchangesincontracts/http-operation-kinds.json. The method is the default: a GET reads, anything else changes. An operation that breaks it (a GET that writes delivery bookkeeping, a POST that only verifies) says why inreason.
Dependencies
cargo deny check (in lint) refuses advisories, banned crates and
licences. cargo-vet records that
someone looked at each third-party crate in the root Cargo.lock. Its files
are in supply-chain/:
config.toml: the audit sets imported from Mozilla, Google, the Bytecode Alliance, Zcash and ISRG, and the exemptions. The exemptions are the crates in the lockfile when cargo-vet was adopted that no import covered; they are a record of what was trusted without review, not an audit.audits.toml: audits this repository has done itself.imports.lock: the imported audits as fetched. CI runscargo vet --locked, which reads this file and fetches nothing.
A dependency bump or a new crate turns the cargo vet (root lockfile) job red
until the new version is covered. With cargo-vet 0.10.2 installed
(cargo install --locked cargo-vet@0.10.2):
- Run
cargo vet(without--locked). It fetches the imported sets again and updatesimports.lockif a publisher has since audited the new version; if that covers it, you are done. - Otherwise read the code and certify it. For an update, review the diff
(
cargo vet diff <crate> <old> <new>) and record a delta audit withcargo vet certify <crate> <old> <new>; for a new crate,cargo vet inspect <crate> <version>andcargo vet certify <crate> <version>. The criteria aresafe-to-deployfor anything that reaches a shipped binary andsafe-to-runfor a crate used only by tests, benches or build tooling.cargo vet suggestlists what is missing and the smallest diff that would cover it. - If you do not review it, exempt it with
cargo vet add-exemption <crate> <version> --notes "<why>", and give the same reason in the PR body.
Then run cargo vet prune to drop exemptions and imports nothing needs any
more, and commit the supply-chain/ changes with the Cargo.lock change.
cargo vet --locked must pass locally before you push.
Secrets
.env,maidan.toml,*.pem,*.keyare ignored.- All credentials come from env vars or an external secret manager.
- CI runs a secrets scan on every PR.
- Fixtures use synthetic data only.
CI matrix
Jobs in .github/workflows/ci.yml. A docs-only
PR runs only lint, the secrets scan, the unit and integration tests and the
alert-rule check; it skips the required jobs marked code and every
non-required job, and a skipped required job reports as passed.
| Job | What it runs | Required |
|---|---|---|
lint (fmt + clippy + deny) | fmt, both clippy passes, cargo-deny | yes |
secrets scan | trufflehog | yes |
unit tests | cargo nextest run --lib --bins --profile ci | yes |
integration (testcontainers) | cargo nextest run --tests --profile ci, Postgres and MinIO containers | yes |
docker compose smoke | the compose stack up, health checks | yes (code) |
scale-out smoke | two replicas behind one Postgres | yes (code) |
promtool (alert rules) | Prometheus rule checks | yes |
otlp smoke | traces reach a collector | yes (code) |
helm install (kind) | the chart on a kind cluster | no |
sqlite-vec (optional feature) | the sqlite-vec feature build and tests | no |
bootstrap compile-time strip | a release build without the bootstrap feature | no |
coverage (llvm-cov) | per-crate coverage floors | no |
a2a tck | the official A2A conformance kit, over JSON-RPC, HTTP+JSON and gRPC | no |
loom | every interleaving of the sharded bus and the presence hub | no |
tla | TLC over the TLA+ specs, each with a config it must fail | no |
osv scan (lockfiles outside cargo-deny) | osv-scanner over fuzz/, ui-tests/ and the SDK lockfiles | no |
cargo vet (root lockfile) | cargo vet --locked: every crate in Cargo.lock audited or exempted | no |
mcp inspector (report-only) | the official MCP Inspector against the server | no |
sdk interop (report-only) | the four SDKs against a live server | no |
pitr drill | point-in-time recovery to a chosen moment | no |
open work | Open Work's in-flight rows and stamp against history | no |
changelog (released sections) | a released CHANGELOG section gains no entries after its tag | no |
deploy pins | every deploy pin and install command names the newest tagged release (scripts/check-deploy-pins.sh lists them) | no |
sqlite backup drill | a SQLite snapshot taken mid-write, restored over a killed server's files | no |
replica routing (LSN) | read-your-writes across a streaming replica | no |
ui tests (playwright) | the /ui specs in a headless browser | no |
When a non-required job goes red. "Not required" means it cannot block a
merge, not that a red result is ignored. These jobs carry claims the docs make
(A2A conformance, model-checked claims and the shredded log, SDK interop), so a
red run on main is a regression of that claim: the next PR either fixes it or
records why in Open Work, and the claim's wording is corrected while it stands.
A job that stays red for a week is removed rather than left to rot. Which of
them become required is the maintainer's decision (Open Work, "Decisions
pending the maintainer").
The docs site builds in docs.yml (mdbook,
not required). nightly.yml runs the slower checks: cargo-mutants over the
store, artifacts, auth and bus, a benchmark build, and five minutes of fuzzing
per target in fuzz/ (the egress SSRF guard, room URIs, waiter results, event
type ids, content keys, MCP and A2A JSON-RPC requests, A2A page tokens, the
WebSocket subscribe frame). A new parser of untrusted input gets a target
there, with seed inputs under fuzz/seeds/<target>/ when it reads structured
input.
No nightly job is continue-on-error, so a failing job makes the night red.
Before October 2026 every mutation shard and the fuzz job failed in their
first seconds each night, and the run showed green. A manual run
(gh workflow run nightly.yml -f jobs=<job>) can run one group alone: all
(the default, and what the schedule runs), mutation (auth, bus and
artifacts), store (the store plan and its shards), bench or fuzz. A full
night holds most of the runner pool for hours, so rerun only the part you need.
Nightly mutation testing
A mutant that survives the tests is a change to the code that no test notices. The nightly jobs report them; they are findings, not failures.
- auth, bus and artifacts are mutated whole, auth in three shards and bus in four (an auth mutant takes about 6 s locally, a bus mutant about 20 s).
- The store is mutated where it changed. It has about 4,800 mutants, and a
viable one rebuilds and relinks the store's 137 test binaries and reruns
them. A local sample of 41 mutants (
cargo mutants --package maidan-store --test-tool nextest --sharding round-robin --shard 0/120, 16 cores, 2026-09-30) took 164 minutes: 32 viable, all caught, at 5 minutes each on average and 14 at most, and 9 unviable at half a minute. That is 4 minutes a mutant (3 for the first 20, before other builds loaded the machine). At three times that on a four-core runner a whole sweep is about 960 runner hours, so it does not run. Insteadscripts/mutants.sh plancounts the mutants in the store code changed since the last commit more than 25 hours old (the nights overlap by an hour rather than leave a gap) and splits them into shards of 20, which at the sampled rate take about four hours on a runner (three at the unloaded rate), inside the step's 300 minutes. - At most 10 store shards run, assuming GitHub's 20 concurrent jobs for a
public repository on the free plan: with the eight other mutation jobs, the
benchmark and the fuzz job, the night fits the pool (the planning job ends
before the shards start). That is 200 mutants a night; in the week to
2026-09-30 one day's store changes produced up to about 390, so a busy day
overflows. The planning job then warns and names the manual run
(
jobs=store,store_base,store_first_shard) that covers the rest; run it beforemainmoves, since the shards are cut from the diff to the head. - A shard that does not finish fails. Each cargo-mutants step has its own
time limit, shorter than the job's, so a shard cut off by it still reports:
scripts/mutants.sh reportwrites the counts and the missed and timed-out mutants to the job summary, uploadsmutants.out/, and fails the job if the run was cut short, the unmutated tree failed its tests, or cargo-mutants failed. There is nocontinue-on-error: until this was fixed every shard failed in its first seconds (an--outputdirectory that did not exist) and the night reported success. - Remeasure before resizing. Run a round-robin shard of the store as above
and read the phase durations in
mutants.out/outcomes.json; changeSTORE_MUTANTS_PER_SHARDorSTORE_MAX_SHARDSinscripts/mutants.shand this section together. - Exclusions live in
.cargo/mutants.toml, each with its reason. The one there is the store's test harness (test_support.rs): mutating the Docker check to "no daemon" turns every Postgres test into a skip, so that mutant survives by construction. Code is excluded only when mutating it cannot say anything about the code the tests check; hard to test is not a reason.
Nightly fuzzing
- Every target runs. The job builds every target in
cargo +nightly fuzz list, then fuzzes each forFUZZ_SECONDS(300) with its seeds andfuzz/json.dict. A target that fails does not stop the others. - The corpus carries over. The job restores the newest saved
fuzz/corpusfrom the Actions cache before fuzzing and saves it afterwards, even when a target failed, so each night starts from what the last one found rather than from the seeds alone. The cache evicts the oldest entries past the repository's limit. - A failure is red. A crash, a sanitizer report (a leak included), a
libFuzzer timeout or out-of-memory, or a target that stops before its time
fails the job. The job summary has a row per target with libFuzzer's
Done N runs in S second(s)line; a failed target's panic or sanitizer line and its input, base64, follow the table, and the file is in thefuzz-artifactsupload. Reproduce withcd fuzz && cargo +nightly fuzz run --target x86_64-unknown-linux-gnu <target> <file>(off Linux, without--target), fix the parser, and add the input tofuzz/seeds/<target>/or as a unit test. - Pass
--target x86_64-unknown-linux-gnuon Linux. cargo-fuzz defaults to the triple it was compiled for, and its release binary is the musl build, where rustc refuses the address sanitizer; every night failed that way until the job set the triple. A cargo-fuzz built from source on the machine does not need it.