ss-tools/specs/038-dashboard-scenario-model/research.md

#region DashboardScenarioModel.Research [C:5] [TYPE ADR] [SEMANTICS research,scenario,graph,checklist,determinism] @BRIEF Phase 0 decisions for deterministic scenario compilation, validation, checklist mapping, and safe draft-pack generation. @RELATION DEPENDS_ON -> [DashboardScenarioModel.Spec] @RELATION DEPENDS_ON -> [SupersetBaselineEngine.DataModel] @RATIONALE The agent may explain and propose intent, but a deterministic compiler/validator must own the graph and generated pack. @REJECTED Direct LLM-to-code or LLM-owned graph identifiers — rejected because refs, safety, and repeatability cannot be guaranteed.

1. Source Checklist Audit

The source research/Чеклист 29.05 (1).pdf is 21 pages and contains 19 cases:

• basic B01–B09;
• complex C01–C07;
• technical T01–T03.

The PDF mixes reusable behavior, FI-0080-specific data, historic pass/fail notes, screenshots, and source-evidence SQL instructions. Historic result text is evidence, not a reusable expected result. SQL instructions become a validated immutable SqlEvidenceSpec when source-mart evidence is required; otherwise they map to Superset API verification or human checkpoints.

2. Deterministic Compiler Boundary

• Decision: Agent produces a bounded ScenarioIntentDraft (objective, selected case ids, user rationale). Backend compiler combines it with DashboardQueryModel, checklist catalog, baseline summaries, capability model, and parameters.
• Determinism: Same canonical inputs and compiler/template versions produce byte-stable graph and draft manifest.
• IDs: Stable ids derive from phase rank, checklist case id, action, and collision-safe ordinal; never random UUIDs inside the canonical graph.
• Ordering: Topological order with stable phase/tool/action tie-breakers.
• Alternative rejected: temperature=0 as the only determinism mechanism — model/provider behavior is not a serialization contract.

3. Capability Mapping

Capability tags include native_filters, text_filter, table_filter, pagination, row_edit, bulk_edit, persistence_refresh, time_rollover, xlsx_export, cross_dashboard, superset_metric, dataset_field_read, screenshot, repository_write.

Each ChecklistCase has:

• required and optional capabilities;
• parameter requirements;
• candidate tool-chain templates;
• automation policy;
• expected evidence;
• fallback classification.

Classification is one of automated, human_checkpoint, unsupported, needs_context. No case is silently dropped.

4. Technical PDF Cases Without SQL

T01–T03 refer to SQL Lab/database fields. Mapping rule:

1. if an authoritative saved dataset/chart result exposes the fields, use superset_api;
2. otherwise create a human checkpoint describing the required evidence;
3. never emit SQL text, SQL tool, or an expected value invented from the PDF.

This preserves coverage intent without violating 037.

5. Graph Model

• DAG phases: setup, interact, observe, assert, evidence, report.
• Every step declares tool, action, inputs, outputs, expected result, dependencies, automation status, checklist refs, and risk.
• Inputs reference parameters, context, baseline refs, or earlier output refs.
• Assertions require baseline_id/candidate_id or structural expectation; raw numeric truth is forbidden.
• Unknown selectors become NEEDS_SELECTOR; missing values become NEEDS_CONTEXT; stale/missing baseline becomes NEEDS_BASELINE.

6. Validator

Validation is pure and returns all findings:

• schema/type/enum violations;
• cycles and dependency order;
• missing/duplicate refs;
• parameter type/resolution;
• tool/action compatibility;
• selector and baseline requirements;
• unsupported/dangerous actions;
• checklist coverage and unreachable steps;
• direct SQL/embedded baseline literals.

Errors block draft-pack compilation. Warnings may allow preview but are repeated in 036 save gate.

7. Safe Draft-Pack Compiler

• Decision: Compile a valid graph through versioned repository-owned templates.
• MVP outputs: scenario.yaml, runner.plan.json, report_template.md, evidence_manifest.json, and optional template-based browser/XLSX assertion modules.
• Invariant: LLM text may populate bounded descriptions only; it cannot provide executable code bodies, paths, imports, or shell commands.
• Drafts: Registered via 036 outside target repository.
• Alternative rejected: Agent writes scripts and later validator scans them — unsafe constructs are already materialized and scanning is incomplete.

8. Resolution Semantics

Editing ParameterDefinitions/defaults may produce a new executable revision with parent_revision_id; supplying runtime ParameterBindings occurs only at 044 launch and never changes content identity. Unrelated steps retain logical_step_id and serialization. Manual conversion and selector hints are explicit resolution operations with audit reason.

9. LLM Verification Tooling — Module Reuse

Capture and VLM analysis are not new implementations; they are thin orchestration over the existing llm_analysis plugin infrastructure.

Concern	Reused module (real contract ID)	File
Playwright capture (login/CSRF, tab traversal, stabilization, chunk capture, webp)	Plugin.Service.ScreenshotService	backend/src/plugins/llm_analysis/service.py
VLM/LLM provider submission (multimodal, JSON mode, retries, image optimization, payload sizing)	Plugin.Service.LLMClient	backend/src/plugins/llm_analysis/service.py
Provider resolution (multimodal check, encrypted API key, token config)	Services.LlmProvider.LLMProviderService	backend/src/services/llm_provider.py
Artifact registration + masking	Services.AgentRuns.Evidence (register_screenshot_draft / register_masked_derivative)	backend/src/services/agent_runs/evidence.py
Redaction of logs/raw responses	Plugin.Service.RedactionService	backend/src/plugins/llm_analysis/service.py

Decision: ScenarioGraph.Capture.Dispatch (038) registers evidence through the 036 bridge AND delegates the actual browser capture to Plugin.Service.ScreenshotService.capture_dashboard_chunks at runtime. Registering artifacts without real capture bytes is a stub, not a capture. Decision: ScenarioGraph.Vlm.Analyze (038) parses/validates typed VlmFinding[]; the actual provider call is injected as submit= and MUST be backed by Plugin.Service.LLMClient + Services.LlmProvider.LLMProviderService. The _default_submit placeholder (returns empty findings) is a test seam only and must be replaced by a real provider submit in production wiring. Decision: Provider selection for VLM reuses the multimodal-required validation semantics of Services.ValidationService.ValidationTaskService._validate_provider; a non-multimodal provider is rejected before analysis. Alternative rejected: Building a second VlmProviderClient or a second Playwright capture path inside dashboard_testing/ — both would fork retry/SSL/masking behavior from the established plugin and double maintenance. @REJECTED: VLM output as deterministic assertion truth; capture without Plugin.Service.ScreenshotService; provider calls without Plugin.Service.LLMClient/Services.LlmProvider.LLMProviderService.

#endregion DashboardScenarioModel.Research