ss-tools/specs/039-dashboard-scenario-ui/spec.md

#region DashboardScenarioUi.Spec [C:3] [TYPE ADR] [SEMANTICS spec,requirements,ux,scenario,dashboard-testing] @BRIEF Persistent agent workspace for scenario generation, scenario revision saves, evidence-led remediation, and policy-bound approvals. @RELATION DEPENDS_ON -> [Doc.Adr.ADR0001] @RELATION DEPENDS_ON -> [Doc.Adr.ADR0005] @RELATION DEPENDS_ON -> [Doc.Adr.ADR0006] @RELATION DEPENDS_ON -> [AgentTestStabilization.Spec] @RELATION DEPENDS_ON -> [SupersetBaselineEngine.Spec] @RELATION DEPENDS_ON -> [DashboardScenarioModel.Spec] @RATIONALE Users should approve a business-level dashboard test scenario and required parameters; low-level tool chains are visible for trust but selected by the agent and scenario validator. @REJECTED Dropdowns such as "Playwright UI tests" vs "SQL checks" vs "XLSX checks" — rejected because each dashboard requires a unique cross-tool program. Validated immutable SQL evidence is visible in the program, not chosen as a loose UI mode.

Navigation (DSA Indexer keywords)

@SEMANTICS: spec, requirements, feature, ux, agent, scenario, dashboard-testing, artifacts, baseline

Feature Branch: 039-dashboard-scenario-ui
Created: 2026-07-07 | Status: Ready for Implementation Input: "Provide the user-facing persistent agent workspace. From a dashboard page the analyst creates or remediates a scenario; the agent analyzes context, proposes a graph, collects authoring context, previews artifacts/evidence, saves delegated revisions, and uses inline approval only where policy requires it."

User Scenarios

Story 1 — Start Scenario From Dashboard Page (P1)

Why P1: The feature must be discoverable from the dashboard the user wants to test and must carry the correct context into the agent.

Independent Test: On a dashboard page, click "Создать сценарий тестирования" and verify /agent opens with dashboard context and scenario intent.

Acceptance:

1. Given a user is viewing a dashboard When they click "Создать сценарий тестирования" Then /agent opens in Dashboard Test Scenario Agent mode with dashboard id, name, env, route, and intent visible.
2. Given environment context is missing When the user starts the flow Then the UI prompts for environment selection before scenario analysis.
3. Given the user lacks scenario-generation permission When they click the action Then a permission-denied recovery path appears and no agent run starts.

---

Story 2 — Review Proposed Unique Scenario (P1)

Why P1: The user must see the proposed business flow and tool chain before artifact generation.

Independent Test: Feed fixture scenario graph data and verify the UI renders phases, steps, tools, outputs, warnings, blockers, and coverage.

Acceptance:

1. Given the agent analyzes a dashboard When scenario graph is ready Then the UI shows goal, phases, step table, dependency graph, tool categories, expected results, coverage, warnings, and blockers.
2. Given some checklist cases are not automatable When scenario preview renders Then they appear as human checkpoints, unsupported, or needs-context with rationale.
3. Given the scenario includes source-mart evidence When preview renders Then the UI shows its validated SqlEvidenceSpec (relation/connection identity, hash, typed parameters, limits and output schema) and distinguishes it from Superset chart API evidence.

---

Story 3 — Define Parameters and Baseline Constraints (P1)

Why P1: Reusable dashboard tests need typed parameter definitions and baseline constraints; actual launch values belong to 045 RunPreflight.

Independent Test: Scenario preview renders parameter definitions; user edits default/validation/source and sees affected steps, while a required no-default scenario remains save-eligible.

Acceptance:

1. Given a scenario declares parameters When preview renders Then each ParameterDefinition has label, type, validation, default/source if available, and affected steps; it contains no runtime value/status.
2. Given an approved baseline exists for metric+filters When the analyst defines binding constraints Then baseline compatibility is shown without creating a runtime binding.
3. Given no approved baseline exists When the user proceeds Then the UI offers discovery candidate flow and marks baseline approval as separate HITL action.

---

Story 4 — Preview Generated Artifacts (P2)

Why P2: Users need to inspect generated files, report templates, and warnings before saving.

Independent Test: Generate draft artifacts from a fixture scenario and verify file tree, validation status, warnings, and content preview render.

Acceptance:

1. Given draft artifacts are generated When preview opens Then the UI lists scenario file, generated runner/check files, report template, baseline candidate files, and validation status.
2. Given an artifact has unresolved markers When preview renders Then save is warning-gated and unresolved markers are visible.
3. Given user downloads draft artifacts When download completes Then repository state remains unchanged.

---

Story 5 — Save or Approve Through Delegated Policy (P2)

Why P2: Durable actions need attributable policy decisions; the agent may save validated scenario revisions while high-risk baseline/publish actions remain gated.

Independent Test: Save a validated revision under delegated policy and trigger baseline approval; verify immutable provenance and inline gate information where required.

Acceptance:

1. Given a validated scenario revision is ready When delegated policy permits Then the agent saves it with immutable revision, delegator, action and evidence provenance.
2. Given baseline approval or another non-delegated action is requested When policy requires it Then an inline card shows target, risk, provenance/diff and a required reason.
3. Given a gate is denied When denial is submitted Then no gated side effect occurs and the run records cancellation.

---

Edge Cases

• Agent analysis fails due to Superset API error → UI shows retry, switch environment, or manual-only scenario recovery.
• Scenario has unresolved blockers → generation can continue only for safe preview; executable save is blocked until resolved or converted to human checkpoint.
• Baseline is stale → UI warns and offers discovery candidate flow, not silent update.
• XLSX export is unavailable → UI shows coverage gap and alternate Superset API/UI checks.
• User navigates away during generation → run id allows returning to latest draft state from 036.

Requirements

Functional — Agent Workspace (persistent chat and work surfaces)

These requirements cover the persistent agent workspace where the analyst creates, refines, investigates, and remediates scenarios through chat plus structured work surfaces.

• AGUI-FR-001: Dashboard pages MUST expose a single business-level action labeled "Создать сценарий тестирования"; UI MUST NOT present low-level artifact/tool choices as the primary entry point.
• AGUI-FR-002: /agent MUST show Dashboard Test Scenario Agent mode when opened with scenario intent from 036.
• AGUI-FR-003: The UI MUST render structured progress stages from agent metadata: context, inspect, scenario, parameters, generate, validate, save.
• AGUI-FR-004: The scenario preview MUST show business goal, phases, step list, tool category per step, dependency graph, coverage, warnings, blockers, expected results, and the visible Verification Program (navigation, evidence, transforms, assertions and declared agentic evaluations).
• AGUI-FR-005: Authoring MUST support typed ParameterDefinition/default/validation/source edits and show affected steps without restarting the flow. Required parameters without defaults remain save-eligible; 045 alone collects runtime ParameterBindings.
• AGUI-FR-006: Baseline UI MUST show approved baseline matches, stale warnings, missing baselines, and draft candidate approval paths.
• AGUI-FR-007: Generated artifact preview MUST show file tree, file content preview, validation status, unresolved markers, and warnings before save.
• AGUI-FR-008: The agent MAY save a validated executable scenario revision under delegated policy. Baseline approval and every non-delegated risky action MUST use a 036 inline ActionApprovalGate; baseline approval requires a reason.
• AGUI-FR-009: The UI MUST distinguish Superset chart API evidence from source-mart SqlEvidenceSpec executed through the Superset SQL Lab adapter. It MUST show SQL as an immutable validated program artifact, never as runtime/chat-editable text.
• AGUI-FR-009a: The UI MUST collect and display ChangeRequestContext (request, affected dashboards/fields, technical detail, relations, acceptance criteria and control totals). Missing needed context is explicit needs_context, not agent inference.
• AGUI-FR-010: All new UI state MUST follow Svelte 5 runes/model-first conventions and remain accessible by keyboard for inline action cards, parameter forms, and previews; modal/dialog interaction MUST NOT be required to complete a workflow.
• AGUI-FR-011: Screenshot evidence artifacts MUST render in a dedicated EvidencePanel showing the captured image, capture metadata (viewport, filters_hash, timestamp), and any linked VLM findings with severity, region, and confidence.
• AGUI-FR-012: VLM findings MUST be reviewable with typed disposition controls: confirm (accepts finding as valid), dismiss (marks as false positive), or inconclusive (defers to human checkpoint). Disposition changes MUST be auditable and MUST NOT alter the scenario graph.
• AGUI-FR-013: The artifact preview file tree MUST include an evidence/ branch listing screenshot artifacts and their associated VLM finding files.

These requirements are satisfied through the DashboardTesting.AgentWorkspace components. The agent is the interaction surface; structured panels (Parameters, ArtifactPreview, EvidencePanel, ActionTimeline) provide data entry and review within the workspace. The agent may perform delegated durable actions, while policy-gated actions remain inline cards in the same thread.

Functional — Pipeline Verification Views (automated, no agent interaction)

These requirements cover views that display the results of automatically triggered verification runs during the release pipeline. These views DO NOT require starting an AgentRun or conversing with the agent. They consume VerificationRun records (037) produced by pipeline hooks.

• AGUI-FR-014: The PREPROD deployment status page MUST display a dedicated Verification tab showing: StructureDiff summary, metric comparison results, visual findings status, and overall VerificationRun status. CRITICAL structural changes MUST block the «Validate» action. The tab is populated from the VerificationRun created automatically by the deploy_to_preprod hook — no agent interaction required.
• AGUI-FR-015: The release detail page MUST show verification status inline: StructureDiff summary between this release and the previous one, metric pass/warn/fail counts, immutability violation alerts for closed periods. Release approval MUST be gated on verification status. All data comes from the VerificationRun attached to the release.
• AGUI-FR-016: The dashboard page MUST show verification history: a chronological list of VerificationRun entries with trigger, environment, overall status, and a link to the full run. The latest run status MUST be visible as a badge next to the release version. This is a read-only projection of the verification log.

These requirements are satisfied through the ReleaseVerification components. They bind to VerificationRun data directly (via DashboardTesting.ApiClient), not to AgentRun or WorkspaceModel. No conversation, no parameter entry, no draft artifacts — the analyst reviews structured results and takes pipeline decisions (validate, approve, publish) through dedicated buttons.

Architectural boundary

Agent Workspace (AGUI-FR-001..013)          Pipeline Views (AGUI-FR-014..016)
──────────────────────────────────          ──────────────────────────────
Entry:  «Создать сценарий» → /agent        Entry:  deployment page, release page,
Interaction:  диалог с агентом                      dashboard page
State:  WorkspaceModel + AgentRunModel      State:  VerificationRun[] + StructureDiff
Trigger:  manual (аналитик)                 Trigger:  deploy_to_preprod, release_create,
Data:  038 ScenarioResponse, 036 drafts                scheduled, etl_completed (автоматически)
Save:  delegated policy / inline gate       Save:  validate/approve/publish (pipeline gates)

LLM Verification Tooling Reuse (039)

The EvidencePanel and VlmFindingReviewCard components consume DTOs only — they never call a provider, capture a screenshot, or parse LLM prose. All LLM verification tooling is reused from upstream specs:

UI surface	Reused contract / DTO	Producer
EvidencePanel screenshot viewer + capture metadata	evidence_captured event, ScreenshotEvidence DraftArtifact (capture_meta)	036 Evidence adapter + Plugin.Service.ScreenshotService
VlmFindingReviewCard (severity, region, confidence, provenance)	VlmFinding / VlmAnalysis DTOs	038 ScenarioGraph.Vlm.Analyze over Plugin.Service.LLMClient
Disposition controls (confirm/dismiss/inconclusive)	HumanDisposition DTO, 036 audit events	038 ScenarioGraph.Human.Disposition
Visual baseline comparison badges	ComparisonResult (ssim/pass/stale_visual_baseline)	037 BaselineEngine.Visual.Compare over BaselineEngine.Visual.SSIM

@REJECTED Frontend-side LLM/VLM calls or screenshot rendering heuristics — the UI renders typed findings and evidence only; provider calls stay on the backend in reused plugin modules.

Key Entities — Agent Workspace

• ScenarioEntryAction: Dashboard-page action that launches /agent with dashboard scenario intent. Only valid in manual (ad-hoc) flow.
• ScenarioWorkspaceState: Frontend state machine for agent-driven creation and remediation: context, progress, scenario preview, parameter collection, artifact preview, evidence and inline action states.
• ScenarioPreviewCard: UI representation of DashboardTestScenario from 038.
• ParameterPanel: Form for scenario business parameters and validation feedback. Text-input based, with typed validation.
• BaselineImpactPanel: UI section showing approved, stale, missing, and candidate baseline statuses within the agent workspace.
• ArtifactPreviewPanel: UI file tree/content preview for draft generated artifacts. Download is side-effect-free.
• ScenarioActionCard: Inline 036 policy/gate card for baseline approval and other non-delegated actions.
• EvidencePanel: Panel for viewing captured screenshots, reviewing VLM findings, recording dispositions. Composes screenshot viewer, finding list, finding detail, and disposition controls.
• VlmFindingReviewCard: Card per VLM finding showing severity badge, confidence bar, region highlight, model/prompt provenance, and confirm/dismiss/inconclusive controls.

Key Entities — Pipeline Verification Views

These entities are independent of the agent workspace. They consume VerificationRun data directly — no AgentRun, no conversation, no parameter entry.

• VerificationStatusBadge: Compact badge showing VerificationRun overall_status (pass/warn/fail/blocked/pending). Appears on dashboard page, release list, and PREPROD deployment page. Pure presentational; no mutation.
• StructureDiffPanel: Side-by-side or list view of structural changes between releases. Groups by severity (critical/warning/info), shows affected artifacts, supports confirm/dismiss per change. Confirm/dismiss are local audit actions — they record analyst acknowledgement, not agent state.
• VerificationHistoryList: Chronological table of VerificationRun entries with trigger icon, environment, overall_status badge, summary, and link to full run. Read-only projection.

Success Criteria

• SC-001: Users can launch scenario generation from a dashboard page in one click and see correct context in /agent in frontend tests.
• SC-002: Scenario preview renders at least 15-step fixture graphs with phases, tools, warnings, and blockers without layout collapse at 1366px width.
• SC-003: Parameter updates resolve dependent scenario readiness within 200ms in model tests.
• SC-004: Artifact preview blocks save when unresolved markers are present in fixture data.
• SC-005: 100% of durable actions have immutable delegated-policy provenance; baseline approval cannot submit without a reason and any required gate.
• SC-006: UX exposes every SqlEvidenceSpec as a validated immutable authoring artifact and never as a runtime-editable or agent-generated-on-run validation path.
• SC-007: Pipeline verification views render StructureDiff, metric results, and VerificationRun data without initializing an AgentRun or WorkspaceModel.
• SC-008: VerificationStatusBadge appears on dashboard, release, and PREPROD pages within 200ms of data load.

Implementation Status & MVP Debt (audit 2026-08-07)

Facts (code check, not tasks.md):

• ✅ 13 компонентов dashboard-testing (ScenarioWorkspace, EvidencePanel, ParameterPanel, ArtifactPreview, VlmFindingReviewCard и др.) реализованы и отрендерены в frontend/src/routes/agent/+page.svelte.
• ✅ Pipeline-компоненты VerificationStatusBadge, StructureDiffPanel, VerificationHistoryList и API-клиент getVerificationHistory()/getVerificationRun() существуют (frontend/src/lib/components/dashboards/verification/, frontend/src/lib/api/dashboard-testing.ts).
• 🔴 Прямой API-слой сценариев не задействован: dashboard-testing.ts (scenario compile/validate/resolve) нигде не импортируется; DashboardScenarioWorkspaceModel не выполняет fetch — данные только через агентские события (036). Без запущенного агента сценарий не строится.
• 🔴 Pipeline views не привязаны к страницам: badge/history/diff-компоненты нигде не отрендерены (dashboard hub / release / PREPROD страницы не используют их), а их GET-эндпоинты (/verification/history, /verification/{runId}) отсутствуют на backend (037 Gap B, только POST /verification-runs существует). Проверка «Validate» на PREPROD деплое не создаёт VerificationRun.

Закрытие: задачи T054–T056 (REST-binding, автономный evidence/VLM-путь) + T057–T058 (привязка pipeline views к страницам, verify-action) в tasks.md Phase 9. T057–T058 зависят от 037 T080–T081 (deploy-hook triggers + GET read-API). Если продукт сознательно остаётся agent-only, зафиксировать это решение в UX-контракте вместо имплементации.

Runtime Closure Status (2026-08-07, resolved)

• ✅ T054/T056: REST-путь сценариев готов — api/dashboard-testing.ts (compile/validate/resolve) + WorkspaceModel.compileFromRest/validateFromRest/resolveFromRest; DashboardScenarioWorkspaceModel.rest.test.ts (3 теста), 79 vitest passed, build OK.
• ✅ T057: DashboardDetailModel.loadVerificationRuns() + VerificationHistoryList привязан на /dashboards/[id] (потребляет 037 T081 GET /verification/history); DashboardDetailModel.test.ts = 67 passed.
• 🟡 T058 — открыт (deferred): verify-action на PREPROD требует repository_id, которого нет в dashboard metadata, и отдельной deployment-страницы, которой нет во frontend. Требует dashboard→git-repository linkage + deployment surface. Зафиксировано в tasks.md как известный blocker.

#endregion DashboardScenarioUi.Spec