ss-tools/specs/044-dashboard-scenario-execution/data-model.md

#region ScenarioExecution.DataModel [C:5] [TYPE ADR] [SEMANTICS data-model,scenario,execution,run,step,state] @BRIEF Canonical ScenarioRun, ScenarioStepRun, RunnerPlan, executor registry, and lifecycle state model. @RELATION DEPENDS_ON -> [ScenarioExecution.Research] @RATIONALE A typed run/step state model is required for reload during a test, recovery, resume, and reproducibility. Without it, human-checkpoint resume and page reload are impossible. @REJECTED In-memory-only run state — rejected because runs must survive disconnect and be recoverable by run id.

ScenarioRun and PROD approval lifecycle

ScenarioRun is created before dispatch. Fields: id, scenario_id, scenario_revision_id (revision_id UUID), scenario_content_hash, verification_program_hash, action_registry_version, dashboard_id, environment_id, status (pending_approval|queued|running|waiting_human|blocked|cancel_requested|cancelled|passed|failed|inconclusive), phase (preflight|setup|executing|waiting_human|draining|terminal), parameter_bindings (immutable JSON), baselines_pinned (version), target_snapshot, execution_principal_fingerprint, execution_toggles (optional evidence only), trigger_source (server-owned), agent_run_id? (provenance), verification_run_id? (aggregation), idempotency_key (unique), started_at, finished_at, resume_token, error_code, runner_version.

If the selected revision contains a human step, it is derived as manual_run_only=true: it may start only from the authenticated analyst manual-run route. Scheduler, deploy/ETL/API trigger and any background runner are ineligible; no HumanCheckpoint may be skipped or defaulted.

For a PROD request, the service atomically creates ScenarioRun(status=pending_approval) and ActionApprovalGate(owner_type=scenario_run, owner_id=run_id, operation=scenario_execution). Approval transitions only pending_approval → queued; denial/expiry transitions to blocked. Schedulers and external triggers therefore receive a durable run/intent, never an unusable 403. trigger_source is set only by the trusted entry point (manual route, scheduler, deploy connector, or API key), never by a bearer client field. Idempotency uses canonical execution-request hash: same key + same hash returns the existing run; same key + different hash returns 409 IDEMPOTENCY_KEY_REUSED.

ParameterBinding, ExecutionPrincipal, and TargetSnapshot

ParameterBinding: parameter_name, resolved_value, source, resolved_at, validation_fingerprint. It is derived at start from the 038 ParameterDefinition and launch input; it is never embedded in ScenarioRevision or its content_hash.

ExecutionPrincipal: auth_mode, actor_id?, service_identity?, impersonated_user?, effective_roles_hash, rls_context_hash. Its immutable fingerprint is stored on the run and used for every Superset request.

TargetSnapshot: environment_id, dashboard_release_id, dashboard_fingerprint, dataset_lineage_fingerprint, captured_at. It is mandatory even when no release was selected, so environment_id=preprod is never treated as an immutable target.

AnalyticsContextKey is a server-derived SHA-256 over environment_class + compatibility_family + baseline_family + dashboard_release_id + dashboard_fingerprint + dataset_lineage_fingerprint + execution_principal_fingerprint. It is captured on the run and every step result; analytics never groups runs merely by environment or revision id.

Statuses: queued, running, waiting_human, blocked, cancel_requested, cancelled, passed, failed, inconclusive.

Terminal failed, inconclusive and blocked outcomes emit a 036 InvestigationSignal with this immutable execution snapshot. 047 deterministically creates/updates the Queue/Episode from that signal. Signal delivery never changes run status and never auto-starts an agent run; an analyst opens the case explicitly from 045/047.

ScenarioStepRun

Fields: id, run_id FK, logical_step_id (immutable UUID, from #8), step_position (mutable), step_content_hash (mutable), attempt, status (queued|running|waiting_human|passed|failed|inconclusive|blocked|skipped), started_at, finished_at, inputs_snapshot (JSON, no secrets), outputs (JSON), artifact_refs (JSON), error_code, progress, timeout_ms, side_effect_key (nullable), step_outcome (StepOutcome).

StepOutcome { status, reason_codes[], deterministic_evidence_refs[], agent_evaluation_ids[], decision_policy_id?, decision_policy_version?, decided_at } is the authoritative result of one logical step. It is distinct from executor output and from a model verdict.

AgentEvaluation and DecisionPolicy

AgentEvaluation { evaluation_id, scenario_run_id, logical_step_id, attempt, provider_id, model_id, model_version, prompt_template_id, prompt_template_version, input_manifest_hash, evidence_refs, verdict, confidence, findings, reason_codes, raw_response_artifact_ref, started_at, finished_at } is immutable evidence generated only for a declared 038 AgentEvaluationSpec. Its tool/evidence access is bounded by that spec; it cannot mutate program content, invoke mutation actions, change run state or choose downstream scheduling.

DecisionPolicy { policy_id, version, deterministic_hard_failure, high_confidence_failure, low_confidence, disagreement, missing_evidence } is a versioned deterministic mapper. Defaults: deterministic hard failure→failed; high-confidence policy-qualified agent failure→failed; low confidence→inconclusive; evaluation/evidence disagreement→waiting_human via a HumanCheckpoint only for manual runs; missing required evidence→blocked or inconclusive. Scenario aggregation consumes StepOutcome, not AgentEvaluation verdicts directly.

RunnerPlan — deterministic derivation from revision (#2)

RunnerPlan is derived deterministically at run start from the selected immutable ScenarioRevision/Verification Program, NOT read from a stored runner.plan.json. Fields: scenario_revision_id, scenario_content_hash, verification_program_hash, action_registry_version, env targets, resolved params, pinned baselines, topological order, executor mapping per step, retry/timeout policy, decision-policy map, human-checkpoint list. Run refuses if its revision/program/action-registry hashes differ from the selected revision.

The materialized runner.plan.json in git is a reference artifact, never the runtime source of truth; it may be regenerated from any revision.

ScenarioExecutionContext (#11)

Fields: run_id, browser_session (ref, not raw cookies), page/context ref, auth_context ref, current_dashboard, current_filters, artifact_namespace (owner_type=scenario_run), environment client. Browser secrets/cookies live in a separate secure context, NEVER in inputs_snapshot JSON (keeps reproducibility snapshot secret-free). Browser workers resume only by deterministic replay from the last browser-safe checkpoint (dashboard_open, filters_applied, etc.); replay records reconstruction_replay=true and never reclassifies already completed logical steps as rerun. API/XLSX/pure assertion steps may resume directly only when their executor declares retry-safe/idempotent.

Artifact ownership (#4)

Artifacts use a generic owner: Artifact { id, owner_type: agent_run|scenario_run|verification_run|load_run, owner_id, kind, sha256, content_ref, retention_class, ... }. ScenarioRun evidence/screenshots/report/xlsx use owner_type=scenario_run; no artificial AgentRun is created. Retention_class ties to 046 tiers.

Decision gates (#3)

• ActionApprovalGate — authorization approval for PROD execution, baseline approval, repository mutation. Generalized 036 gate with owner_type + owner_id.
• HumanCheckpoint — checkpoint_id, run_id, logical_step_id, checkpoint_type, decision_policy, status, created_at, expires_at, eligible_role?, eligible_actor_ids?, assigned_to?, evidence_refs, decision_version, decided_by?, decided_at?, disposition?, comment?. Status is pending|decided|expired|cancelled; decision is CAS on decision_version, stale/concurrent decision returns 409. finding_review maps confirm→failed, false_positive→passed, inconclusive→inconclusive; manual_assertion maps pass→passed, fail→failed, inconclusive→inconclusive. It is NOT a 036 ApprovalGate decision.

A HumanCheckpoint is never delegated to the agent: it is a manual-run-only analyst decision inside a currently executing run. The agent may explain the evidence in its workspace but cannot consume the checkpoint or convert it into an automated result.

Worker semantics — at-least-once execution (#6)

Runtime primitives: worker lease, heartbeat, run claim, step claim, lease expiration, idempotency key, recovery scheduler. Each executor declares: idempotent? | retry-safe? | side_effect_key? | external_request_id?. POST /scenario-runs requires Idempotency-Key (unique) to prevent double-run on double-click. A crashed step with an external side effect is only re-run if idempotent/retry-safe or keyed.

Retry semantics (#14)

Retry of a failed step invalidates its downstream closure (descendants depending on its output) and re-runs them; retry after the run advanced beyond the step is rejected unless the whole closure re-runs. Bounded attempts per policy.

Result aggregation truth table (#16)

• any step failed → scenario failed
• blocked descendants counted as blocked (NOT failed)
• skipped does not count against pass
• inconclusive → scenario inconclusive unless a failed also present (then failed)
• warning is an evidence-level qualifier, not an execution status (045 maps from evidence)

Lifecycle

pending_approval → queued → running → waiting_human | blocked → passed | failed | inconclusive; cancel_requested → cancelled. Human decision atomically consumes the checkpoint and resumes internally. Public /resume is reserved for a recoverable infrastructure pause and requires a typed resume token/reason; it cannot consume a HumanCheckpoint. Cancel drains in-flight within a bounded window.

ScenarioExecutorRegistry and BrowserExecutor

Mapping tool -> executor:

• browser -> BrowserExecutor → version-pinned 038 ActionRegistry → Playwright/session infrastructure
• superset_api -> 037 metric_executor_async / SupersetClient.ChartData.Execute
• sql_evidence -> SqlEvidenceExecutor → Superset SQL Lab backend/API → configured database connection (no credentials exposed to agent)
• transform -> version-pinned bounded 038 TransformSpec DSL executor
• xlsx -> xlsx parser + 037 normalization
• assertion -> 037 comparison.py + 038 ComparisonSpec/AssertionSpec executor
• agent_evaluation -> bounded provider adapter executing a declared 038 AgentEvaluationSpec and emitting AgentEvaluation; DecisionPolicy owns StepOutcome
• screenshot -> 038 capture.py + ScreenshotService (owner_type=scenario_run)
• report -> report-template render + artifact (owner_type=scenario_run)
• artifact -> generic artifact register (owner_type=scenario_run)
• human -> EXCLUDED (runner-lifecycle HumanCheckpoint control, not an executor)

BrowserExecutor implements only registered actions: open_dashboard, navigate_tab, apply_native_filter, inspect_filter_state, apply_table_filter, extract_table, scroll_to, inspect_columns, click, select_rows, edit_row, bulk_edit, download, refresh, wait_for_state. ScreenshotService is evidence infrastructure only, not the browser action executor. It validates the registry-declared inputs/outputs/risk/timeout/idempotency before dispatch.

Mutation authorization and execution policy

Authorization is evaluated per run and step as environment_class + scenario risk profile + action mutation profile. Read-only PROD steps require the ScenarioExecution approval policy. Every mutation requires the immutable 038 mutation contract, scoped target keys, precondition evidence, side-effect identity, cleanup/reconciliation outcome and retry_safe=false unless the registry proves an idempotent compensating action. Mutating browser steps in PROD are prohibited. Non-PROD test-data mutation may be delegated only inside an authorized fixture lease; this mutation policy is separate from PROD execution approval.

Immutable Execution Snapshot

A ScenarioRun pins scenario_revision_id + scenario_content_hash at start. Results carry provenance: scenario_revision_id, runner_version, template_version, baseline_revision, target_snapshot, parameter_bindings, execution_principal_fingerprint, query fingerprints. Later edits never alter completed runs.

ExecutionCapacityManager

All AgentRun, VerificationRun, LoadRun, and ScenarioRun claims pass through one environment-scoped capacity manager: environment, workload_class, priority, quota, reserved_capacity. A 046 scenario policy is a consumer of this global allocator, not an independent PROD/PREPROD concurrency limit.

#endregion ScenarioExecution.DataModel