Real-time isolated Docker soak
@{ Tooling.Stage6Soak.Protocol [C:4] [TYPE ADR] [SEMANTICS soak,protocol,durable,isolated]
@BRIEF Executable opt-in protocol for the persistent M01 collector and independent auditor. @RELATION DEPENDS_ON -> [Tooling.Stage6Soak.Collector] @RELATION DEPENDS_ON -> [Tooling.Stage6Soak.Controller] @RELATION DEPENDS_ON -> [Tooling.Stage6Soak.Audit] @RATIONALE Host-side Docker control preserves the collector's smaller authority boundary. @REJECTED A container Docker socket, accelerated clock, modified runtime rows or fabricated receipts cannot prove acceptance. @INVARIANT These commands are a protocol, not evidence that a real 72-hour run has completed.
Run from the repository root. Only the isolated ss-tools-full-flow project is
allowed. Keep /tmp/ss-tools-full-flow.env mode 0600 and preserve every volume.
Never copy the regular backend environment. Collector output contains public
identities and typed error names; Docker inspection retains an allowlist only.
Preparation (disabled schedule only)
Before execution freeze the source/image/dirty-diff/migration identity separately
in the evidence packet, and declare the queue/coverage SLO. The commands below
use the existing actual independently audited M01 source run; preparation creates
its own disabled pinned schedule. It cannot enable an old schedule. Set
SOAK_ACCEPTANCE_DIR explicitly to an existing ignored artifact directory or an
external directory containing independent-result-audit.json; Compose refuses
an unset variable or absent bind source and never creates that input directory.
No historical input is restored from Git. For example, after independently
auditing an actual source run into that existing directory:
export SOAK_ACCEPTANCE_DIR="$PWD/artifacts/docker-full-flow-v2"
test -f "$SOAK_ACCEPTANCE_DIR/independent-result-audit.json"
docker compose --env-file /tmp/ss-tools-full-flow.env -f docker-compose.full-flow.yml -f docker-compose.full-flow-soak.yml --profile stage6-soak run --rm soak prepare --source-commit "$(git rev-parse HEAD)" --max-gap 180 --max-queue-age 300
docker compose --env-file /tmp/ss-tools-full-flow.env -f docker-compose.full-flow.yml -f docker-compose.full-flow-soak.yml --profile stage6-soak run --rm soak status
docker compose --env-file /tmp/ss-tools-full-flow.env -f docker-compose.full-flow.yml -f docker-compose.full-flow-soak.yml --profile stage6-soak run --rm soak tabs --tabs 5
docker compose --env-file /tmp/ss-tools-full-flow.env -f docker-compose.full-flow.yml -f docker-compose.full-flow-soak.yml --profile stage6-soak run --rm soak tabs --tabs 15
docker compose --env-file /tmp/ss-tools-full-flow.env -f docker-compose.full-flow.yml -f docker-compose.full-flow-soak.yml --profile stage6-soak run --rm soak tabs --tabs 50
The image needs its existing Playwright Chromium installation and psycopg2.
Preparation validates the input before credentials or operational I/O and reports
SOAK_ACCEPTANCE_INPUT_UNAVAILABLE or SOAK_ACCEPTANCE_INPUT_INVALID on refusal.
This historical M01 recipe still requires its authored metric admission/pin
format; compatibility with a current source run and operational acceptance have
not been proved by these input changes. Unsupported pins remain unavailable.
Read-only database access targets the isolated db/ss_tools; storage is mounted
read-only at /retained-storage. The host evidence directory is
artifacts/docker-stage6/soak, override via
SOAK_EVIDENCE_DIR consistently for every command if needed.
If preparation loses its response after POST, inspect prepare-intent.json and
the actual schedule list. The schedule remains disabled. Reconcile its actual ID
explicitly; do not blindly retry or enable an unidentified orphan.
Start only after the parent confirms all prerequisites
This slice did not execute these start/restart commands. Starting soak enables
only its manifest schedule and records the actual UTC epoch. A single lock
rejects duplicate collectors. restart: on-failure resumes the same epoch after
unexpected process failure; a normally completed run exits instead of restarting.
docker compose --env-file /tmp/ss-tools-full-flow.env -f docker-compose.full-flow.yml -f docker-compose.full-flow-soak.yml --profile stage6-soak up -d --no-deps soak
backend/.venv/bin/python scripts/stage6_soak/controller.py watch --root artifacts/docker-stage6/soak --env-file /tmp/ss-tools-full-flow.env
Run the host controller under the operator's persistent process supervisor. It cannot depend on a chat turn remaining active. Controller lock and journal preserve progress on restart. It issues exactly three planned backend restarts at genuine elapsed 6/30/54 hours, validates Compose labels, records before/after Docker start times, and waits for actual readiness. No database/volume reset or Docker socket is exposed to the observer container.
If the controller crashes between requesting and recording completion, its next
watch refuses to issue a duplicate restart. Inspect actual Docker state, then use
controller.py reconcile --root <same-directory> --index <1|2|3>: this performs
only inspection/readiness and records completion if the actual start time changed.
An unexecuted request requires operator investigation and remains incomplete.
Repeat the three tab commands after restart recovery. They open genuine browser pages against Superset, close their pages/context/browser and retain aggregate results. They do not establish frontend monitor capacity, independent backend provider shutdown, trace evidence, filter correctness or human analyst scores.
Stop, resume, audit
docker compose --env-file /tmp/ss-tools-full-flow.env -f docker-compose.full-flow.yml -f docker-compose.full-flow-soak.yml --profile stage6-soak run --rm soak stop
docker compose --env-file /tmp/ss-tools-full-flow.env -f docker-compose.full-flow.yml -f docker-compose.full-flow-soak.yml --profile stage6-soak run --rm soak audit
stop requests termination and pauses only the owned schedule. The active
collector records stop time, waits at most the declared queue deadline for work
to drain, then audits. An early terminal stop is INCOMPLETE; a normal signal
pause is resumable via the same service run command without resetting start.
Observation gaps persist and can invalidate acceptance. Preserve all evidence.
The auditor queries real runs, latest required steps, owned artifacts, provider receipts and capacity leases independently. It rehashes retained wire bytes, requires strict numeric schema and actual North revenue 16350, and compares every frozen wrapper/pin to the original admitted source. Complete acceptance needs 259200 real elapsed seconds, every due slot except recorded restart downtime, three genuine restart proofs, coverage/queue SLO and 5/15/50 tab canaries. Failures remain explicit; no terminal runtime record is fabricated.
Scheduled keys contain actual fire timestamps; the independent auditor groups by UTC five-minute bucket, exposing duplicates in different seconds. Without a retained scheduler due timestamp, callbacks delayed across a bucket boundary cannot be conclusively attributed to their original due slot. Treat that as an open scheduler instrumentation limit, not complete logical-dedup proof.
Provider effects are audited for the read-only M01 lane. Notification receipts, DLQ faults, shutdown/session/task instrumentation, LLM evaluation/security and five-person UX acceptance require their separate stage-6 packets. An M01 soak PASS does not close those gates or authorize a global GO.