Compare commits
111 Commits
33433c3173
...
021-llm-pr
| Author | SHA1 | Date | |
|---|---|---|---|
| 6e9f4642db | |||
| 64b7ab8703 | |||
| 0100ed88dd | |||
| 0f9df3715f | |||
| c8ef49f067 | |||
| 24cb95ebe2 | |||
| 473c81d9ba | |||
| ce3bc1e671 | |||
| c3299f8bdf | |||
| bd52e25ff3 | |||
| 2ef946f141 | |||
| 2b16851026 | |||
| 33179ce4c2 | |||
| 4106542da2 | |||
| f0831d5d28 | |||
| e432915ec3 | |||
| 7e09ecde25 | |||
| 787445398f | |||
| 47cffcc35f | |||
| c30272fe8b | |||
| 11e8c8e132 | |||
| 40c2e2414d | |||
| 066ef5eab5 | |||
| 2946ee9b42 | |||
| 5f70a239a7 | |||
| d67d24e7e6 | |||
| 01efc9dae1 | |||
| 43814511ee | |||
| db47e4ce55 | |||
| d5a5c3b902 | |||
| 066c37087d | |||
| b40649b9ed | |||
| 197647d97a | |||
| e9e529e322 | |||
| bc3ff29d2f | |||
| eb8ed5da59 | |||
| b6ae41d576 | |||
| cf42de3060 | |||
| 6062712a92 | |||
| 7790a2dc51 | |||
| a58bef5c73 | |||
| 232dd947d8 | |||
| 33966548d7 | |||
| cad6e97464 | |||
| 47a3213fb9 | |||
| 303d7272f8 | |||
| 0711ded532 | |||
| 495857bbee | |||
| df7582a8db | |||
| 3802b0af8c | |||
| 1702f3a5e9 | |||
| 83c24d4b85 | |||
| dd596698e5 | |||
| 0fee26a846 | |||
| 35096b5e23 | |||
| 0299728d72 | |||
| de6ff0d41b | |||
| 260a90aac5 | |||
| 56a1508b38 | |||
| 7c0a601499 | |||
| a5b1bba226 | |||
| 8f13ed3031 | |||
| 305b07bf8b | |||
| 4e1992f489 | |||
| ac7a6cfadc | |||
| 29daebd628 | |||
| 71873b7bb3 | |||
| 68b25c90a8 | |||
| e9b8794f1a | |||
| 6d94d26e40 | |||
| 598dd50d1d | |||
| eacb88a0e3 | |||
| 10676b7029 | |||
| 2023f6c211 | |||
| 2111c12d0a | |||
| b46133e4c1 | |||
| 6cc2fb4c9b | |||
| c406f71988 | |||
| 55bdd981b1 | |||
| 15843a4607 | |||
| 8b81bb9f1f | |||
| 7f244a8252 | |||
| c0505b4d4f | |||
| 1b863bea1b | |||
| 7c6c959774 | |||
| 554e1128b8 | |||
| 55ca476972 | |||
| 4b4d23e671 | |||
| e80369c8b5 | |||
| ffe942c9dd | |||
| 19744796e4 | |||
| a6bebe295c | |||
| e2ce346b7b | |||
| 789e5a90e3 | |||
| 163d03e6f5 | |||
| 169237b31b | |||
| 45bb8c5429 | |||
| 17c28433bd | |||
| 077daa0245 | |||
| d38cda09dd | |||
| 1a893c0bc0 | |||
| 40ed375aa4 | |||
| 5fdc92fcdf | |||
| e83328b4ff | |||
| 687f4ce565 | |||
| dc9e9e0588 | |||
| 2de3e53ab2 | |||
| 40ea0580d9 | |||
| 8da906738b | |||
| d5a1c0e091 | |||
| ef7a0fcf92 |
@@ -1,35 +0,0 @@
|
||||
# ss-tools Development Guidelines
|
||||
|
||||
Auto-generated from all feature plans. Last updated: 2026-02-25
|
||||
|
||||
## Knowledge Graph (GRACE)
|
||||
**CRITICAL**: This project uses a GRACE Knowledge Graph for context. Always load the root map first:
|
||||
- **Root Map**: `.ai/ROOT.md` -> `[DEF:Project_Knowledge_Map:Root]`
|
||||
- **Project Map**: `.ai/PROJECT_MAP.md` -> `[DEF:Project_Map]`
|
||||
- **Standards**: Read `.ai/standards/` for architecture and style rules.
|
||||
|
||||
## Active Technologies
|
||||
|
||||
- (022-sync-id-cross-filters)
|
||||
|
||||
## Project Structure
|
||||
|
||||
```text
|
||||
src/
|
||||
tests/
|
||||
```
|
||||
|
||||
## Commands
|
||||
|
||||
# Add commands for
|
||||
|
||||
## Code Style
|
||||
|
||||
: Follow standard conventions
|
||||
|
||||
## Recent Changes
|
||||
|
||||
- 022-sync-id-cross-filters: Added
|
||||
|
||||
<!-- MANUAL ADDITIONS START -->
|
||||
<!-- MANUAL ADDITIONS END -->
|
||||
@@ -1,4 +0,0 @@
|
||||
---
|
||||
description: USE SEMANTIC
|
||||
---
|
||||
Прочитай .specify/memory/semantics.md (или .ai/standards/semantics.md, если не найден). ОБЯЗАТЕЛЬНО используй его при разработке
|
||||
@@ -1,184 +0,0 @@
|
||||
---
|
||||
description: Perform a non-destructive cross-artifact consistency and quality analysis across spec.md, plan.md, and tasks.md after task generation.
|
||||
---
|
||||
|
||||
## User Input
|
||||
|
||||
```text
|
||||
$ARGUMENTS
|
||||
```
|
||||
|
||||
You **MUST** consider the user input before proceeding (if not empty).
|
||||
|
||||
## Goal
|
||||
|
||||
Identify inconsistencies, duplications, ambiguities, and underspecified items across the three core artifacts (`spec.md`, `plan.md`, `tasks.md`) before implementation. This command MUST run only after `/speckit.tasks` has successfully produced a complete `tasks.md`.
|
||||
|
||||
## Operating Constraints
|
||||
|
||||
**STRICTLY READ-ONLY**: Do **not** modify any files. Output a structured analysis report. Offer an optional remediation plan (user must explicitly approve before any follow-up editing commands would be invoked manually).
|
||||
|
||||
**Constitution Authority**: The project constitution (`.specify/memory/constitution.md`) is **non-negotiable** within this analysis scope. Constitution conflicts are automatically CRITICAL and require adjustment of the spec, plan, or tasks—not dilution, reinterpretation, or silent ignoring of the principle. If a principle itself needs to change, that must occur in a separate, explicit constitution update outside `/speckit.analyze`.
|
||||
|
||||
## Execution Steps
|
||||
|
||||
### 1. Initialize Analysis Context
|
||||
|
||||
Run `.specify/scripts/bash/check-prerequisites.sh --json --require-tasks --include-tasks` once from repo root and parse JSON for FEATURE_DIR and AVAILABLE_DOCS. Derive absolute paths:
|
||||
|
||||
- SPEC = FEATURE_DIR/spec.md
|
||||
- PLAN = FEATURE_DIR/plan.md
|
||||
- TASKS = FEATURE_DIR/tasks.md
|
||||
|
||||
Abort with an error message if any required file is missing (instruct the user to run missing prerequisite command).
|
||||
For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||
|
||||
### 2. Load Artifacts (Progressive Disclosure)
|
||||
|
||||
Load only the minimal necessary context from each artifact:
|
||||
|
||||
**From spec.md:**
|
||||
|
||||
- Overview/Context
|
||||
- Functional Requirements
|
||||
- Non-Functional Requirements
|
||||
- User Stories
|
||||
- Edge Cases (if present)
|
||||
|
||||
**From plan.md:**
|
||||
|
||||
- Architecture/stack choices
|
||||
- Data Model references
|
||||
- Phases
|
||||
- Technical constraints
|
||||
|
||||
**From tasks.md:**
|
||||
|
||||
- Task IDs
|
||||
- Descriptions
|
||||
- Phase grouping
|
||||
- Parallel markers [P]
|
||||
- Referenced file paths
|
||||
|
||||
**From constitution:**
|
||||
|
||||
- Load `.specify/memory/constitution.md` for principle validation
|
||||
|
||||
### 3. Build Semantic Models
|
||||
|
||||
Create internal representations (do not include raw artifacts in output):
|
||||
|
||||
- **Requirements inventory**: Each functional + non-functional requirement with a stable key (derive slug based on imperative phrase; e.g., "User can upload file" → `user-can-upload-file`)
|
||||
- **User story/action inventory**: Discrete user actions with acceptance criteria
|
||||
- **Task coverage mapping**: Map each task to one or more requirements or stories (inference by keyword / explicit reference patterns like IDs or key phrases)
|
||||
- **Constitution rule set**: Extract principle names and MUST/SHOULD normative statements
|
||||
|
||||
### 4. Detection Passes (Token-Efficient Analysis)
|
||||
|
||||
Focus on high-signal findings. Limit to 50 findings total; aggregate remainder in overflow summary.
|
||||
|
||||
#### A. Duplication Detection
|
||||
|
||||
- Identify near-duplicate requirements
|
||||
- Mark lower-quality phrasing for consolidation
|
||||
|
||||
#### B. Ambiguity Detection
|
||||
|
||||
- Flag vague adjectives (fast, scalable, secure, intuitive, robust) lacking measurable criteria
|
||||
- Flag unresolved placeholders (TODO, TKTK, ???, `<placeholder>`, etc.)
|
||||
|
||||
#### C. Underspecification
|
||||
|
||||
- Requirements with verbs but missing object or measurable outcome
|
||||
- User stories missing acceptance criteria alignment
|
||||
- Tasks referencing files or components not defined in spec/plan
|
||||
|
||||
#### D. Constitution Alignment
|
||||
|
||||
- Any requirement or plan element conflicting with a MUST principle
|
||||
- Missing mandated sections or quality gates from constitution
|
||||
|
||||
#### E. Coverage Gaps
|
||||
|
||||
- Requirements with zero associated tasks
|
||||
- Tasks with no mapped requirement/story
|
||||
- Non-functional requirements not reflected in tasks (e.g., performance, security)
|
||||
|
||||
#### F. Inconsistency
|
||||
|
||||
- Terminology drift (same concept named differently across files)
|
||||
- Data entities referenced in plan but absent in spec (or vice versa)
|
||||
- Task ordering contradictions (e.g., integration tasks before foundational setup tasks without dependency note)
|
||||
- Conflicting requirements (e.g., one requires Next.js while other specifies Vue)
|
||||
|
||||
### 5. Severity Assignment
|
||||
|
||||
Use this heuristic to prioritize findings:
|
||||
|
||||
- **CRITICAL**: Violates constitution MUST, missing core spec artifact, or requirement with zero coverage that blocks baseline functionality
|
||||
- **HIGH**: Duplicate or conflicting requirement, ambiguous security/performance attribute, untestable acceptance criterion
|
||||
- **MEDIUM**: Terminology drift, missing non-functional task coverage, underspecified edge case
|
||||
- **LOW**: Style/wording improvements, minor redundancy not affecting execution order
|
||||
|
||||
### 6. Produce Compact Analysis Report
|
||||
|
||||
Output a Markdown report (no file writes) with the following structure:
|
||||
|
||||
## Specification Analysis Report
|
||||
|
||||
| ID | Category | Severity | Location(s) | Summary | Recommendation |
|
||||
|----|----------|----------|-------------|---------|----------------|
|
||||
| A1 | Duplication | HIGH | spec.md:L120-134 | Two similar requirements ... | Merge phrasing; keep clearer version |
|
||||
|
||||
(Add one row per finding; generate stable IDs prefixed by category initial.)
|
||||
|
||||
**Coverage Summary Table:**
|
||||
|
||||
| Requirement Key | Has Task? | Task IDs | Notes |
|
||||
|-----------------|-----------|----------|-------|
|
||||
|
||||
**Constitution Alignment Issues:** (if any)
|
||||
|
||||
**Unmapped Tasks:** (if any)
|
||||
|
||||
**Metrics:**
|
||||
|
||||
- Total Requirements
|
||||
- Total Tasks
|
||||
- Coverage % (requirements with >=1 task)
|
||||
- Ambiguity Count
|
||||
- Duplication Count
|
||||
- Critical Issues Count
|
||||
|
||||
### 7. Provide Next Actions
|
||||
|
||||
At end of report, output a concise Next Actions block:
|
||||
|
||||
- If CRITICAL issues exist: Recommend resolving before `/speckit.implement`
|
||||
- If only LOW/MEDIUM: User may proceed, but provide improvement suggestions
|
||||
- Provide explicit command suggestions: e.g., "Run /speckit.specify with refinement", "Run /speckit.plan to adjust architecture", "Manually edit tasks.md to add coverage for 'performance-metrics'"
|
||||
|
||||
### 8. Offer Remediation
|
||||
|
||||
Ask the user: "Would you like me to suggest concrete remediation edits for the top N issues?" (Do NOT apply them automatically.)
|
||||
|
||||
## Operating Principles
|
||||
|
||||
### Context Efficiency
|
||||
|
||||
- **Minimal high-signal tokens**: Focus on actionable findings, not exhaustive documentation
|
||||
- **Progressive disclosure**: Load artifacts incrementally; don't dump all content into analysis
|
||||
- **Token-efficient output**: Limit findings table to 50 rows; summarize overflow
|
||||
- **Deterministic results**: Rerunning without changes should produce consistent IDs and counts
|
||||
|
||||
### Analysis Guidelines
|
||||
|
||||
- **NEVER modify files** (this is read-only analysis)
|
||||
- **NEVER hallucinate missing sections** (if absent, report them accurately)
|
||||
- **Prioritize constitution violations** (these are always CRITICAL)
|
||||
- **Use examples over exhaustive rules** (cite specific instances, not generic patterns)
|
||||
- **Report zero issues gracefully** (emit success report with coverage statistics)
|
||||
|
||||
## Context
|
||||
|
||||
$ARGUMENTS
|
||||
@@ -1,294 +0,0 @@
|
||||
---
|
||||
description: Generate a custom checklist for the current feature based on user requirements.
|
||||
---
|
||||
|
||||
## Checklist Purpose: "Unit Tests for English"
|
||||
|
||||
**CRITICAL CONCEPT**: Checklists are **UNIT TESTS FOR REQUIREMENTS WRITING** - they validate the quality, clarity, and completeness of requirements in a given domain.
|
||||
|
||||
**NOT for verification/testing**:
|
||||
|
||||
- ❌ NOT "Verify the button clicks correctly"
|
||||
- ❌ NOT "Test error handling works"
|
||||
- ❌ NOT "Confirm the API returns 200"
|
||||
- ❌ NOT checking if code/implementation matches the spec
|
||||
|
||||
**FOR requirements quality validation**:
|
||||
|
||||
- ✅ "Are visual hierarchy requirements defined for all card types?" (completeness)
|
||||
- ✅ "Is 'prominent display' quantified with specific sizing/positioning?" (clarity)
|
||||
- ✅ "Are hover state requirements consistent across all interactive elements?" (consistency)
|
||||
- ✅ "Are accessibility requirements defined for keyboard navigation?" (coverage)
|
||||
- ✅ "Does the spec define what happens when logo image fails to load?" (edge cases)
|
||||
|
||||
**Metaphor**: If your spec is code written in English, the checklist is its unit test suite. You're testing whether the requirements are well-written, complete, unambiguous, and ready for implementation - NOT whether the implementation works.
|
||||
|
||||
## User Input
|
||||
|
||||
```text
|
||||
$ARGUMENTS
|
||||
```
|
||||
|
||||
You **MUST** consider the user input before proceeding (if not empty).
|
||||
|
||||
## Execution Steps
|
||||
|
||||
1. **Setup**: Run `.specify/scripts/bash/check-prerequisites.sh --json` from repo root and parse JSON for FEATURE_DIR and AVAILABLE_DOCS list.
|
||||
- All file paths must be absolute.
|
||||
- For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||
|
||||
2. **Clarify intent (dynamic)**: Derive up to THREE initial contextual clarifying questions (no pre-baked catalog). They MUST:
|
||||
- Be generated from the user's phrasing + extracted signals from spec/plan/tasks
|
||||
- Only ask about information that materially changes checklist content
|
||||
- Be skipped individually if already unambiguous in `$ARGUMENTS`
|
||||
- Prefer precision over breadth
|
||||
|
||||
Generation algorithm:
|
||||
1. Extract signals: feature domain keywords (e.g., auth, latency, UX, API), risk indicators ("critical", "must", "compliance"), stakeholder hints ("QA", "review", "security team"), and explicit deliverables ("a11y", "rollback", "contracts").
|
||||
2. Cluster signals into candidate focus areas (max 4) ranked by relevance.
|
||||
3. Identify probable audience & timing (author, reviewer, QA, release) if not explicit.
|
||||
4. Detect missing dimensions: scope breadth, depth/rigor, risk emphasis, exclusion boundaries, measurable acceptance criteria.
|
||||
5. Formulate questions chosen from these archetypes:
|
||||
- Scope refinement (e.g., "Should this include integration touchpoints with X and Y or stay limited to local module correctness?")
|
||||
- Risk prioritization (e.g., "Which of these potential risk areas should receive mandatory gating checks?")
|
||||
- Depth calibration (e.g., "Is this a lightweight pre-commit sanity list or a formal release gate?")
|
||||
- Audience framing (e.g., "Will this be used by the author only or peers during PR review?")
|
||||
- Boundary exclusion (e.g., "Should we explicitly exclude performance tuning items this round?")
|
||||
- Scenario class gap (e.g., "No recovery flows detected—are rollback / partial failure paths in scope?")
|
||||
|
||||
Question formatting rules:
|
||||
- If presenting options, generate a compact table with columns: Option | Candidate | Why It Matters
|
||||
- Limit to A–E options maximum; omit table if a free-form answer is clearer
|
||||
- Never ask the user to restate what they already said
|
||||
- Avoid speculative categories (no hallucination). If uncertain, ask explicitly: "Confirm whether X belongs in scope."
|
||||
|
||||
Defaults when interaction impossible:
|
||||
- Depth: Standard
|
||||
- Audience: Reviewer (PR) if code-related; Author otherwise
|
||||
- Focus: Top 2 relevance clusters
|
||||
|
||||
Output the questions (label Q1/Q2/Q3). After answers: if ≥2 scenario classes (Alternate / Exception / Recovery / Non-Functional domain) remain unclear, you MAY ask up to TWO more targeted follow‑ups (Q4/Q5) with a one-line justification each (e.g., "Unresolved recovery path risk"). Do not exceed five total questions. Skip escalation if user explicitly declines more.
|
||||
|
||||
3. **Understand user request**: Combine `$ARGUMENTS` + clarifying answers:
|
||||
- Derive checklist theme (e.g., security, review, deploy, ux)
|
||||
- Consolidate explicit must-have items mentioned by user
|
||||
- Map focus selections to category scaffolding
|
||||
- Infer any missing context from spec/plan/tasks (do NOT hallucinate)
|
||||
|
||||
4. **Load feature context**: Read from FEATURE_DIR:
|
||||
- spec.md: Feature requirements and scope
|
||||
- plan.md (if exists): Technical details, dependencies
|
||||
- tasks.md (if exists): Implementation tasks
|
||||
|
||||
**Context Loading Strategy**:
|
||||
- Load only necessary portions relevant to active focus areas (avoid full-file dumping)
|
||||
- Prefer summarizing long sections into concise scenario/requirement bullets
|
||||
- Use progressive disclosure: add follow-on retrieval only if gaps detected
|
||||
- If source docs are large, generate interim summary items instead of embedding raw text
|
||||
|
||||
5. **Generate checklist** - Create "Unit Tests for Requirements":
|
||||
- Create `FEATURE_DIR/checklists/` directory if it doesn't exist
|
||||
- Generate unique checklist filename:
|
||||
- Use short, descriptive name based on domain (e.g., `ux.md`, `api.md`, `security.md`)
|
||||
- Format: `[domain].md`
|
||||
- If file exists, append to existing file
|
||||
- Number items sequentially starting from CHK001
|
||||
- Each `/speckit.checklist` run creates a NEW file (never overwrites existing checklists)
|
||||
|
||||
**CORE PRINCIPLE - Test the Requirements, Not the Implementation**:
|
||||
Every checklist item MUST evaluate the REQUIREMENTS THEMSELVES for:
|
||||
- **Completeness**: Are all necessary requirements present?
|
||||
- **Clarity**: Are requirements unambiguous and specific?
|
||||
- **Consistency**: Do requirements align with each other?
|
||||
- **Measurability**: Can requirements be objectively verified?
|
||||
- **Coverage**: Are all scenarios/edge cases addressed?
|
||||
|
||||
**Category Structure** - Group items by requirement quality dimensions:
|
||||
- **Requirement Completeness** (Are all necessary requirements documented?)
|
||||
- **Requirement Clarity** (Are requirements specific and unambiguous?)
|
||||
- **Requirement Consistency** (Do requirements align without conflicts?)
|
||||
- **Acceptance Criteria Quality** (Are success criteria measurable?)
|
||||
- **Scenario Coverage** (Are all flows/cases addressed?)
|
||||
- **Edge Case Coverage** (Are boundary conditions defined?)
|
||||
- **Non-Functional Requirements** (Performance, Security, Accessibility, etc. - are they specified?)
|
||||
- **Dependencies & Assumptions** (Are they documented and validated?)
|
||||
- **Ambiguities & Conflicts** (What needs clarification?)
|
||||
|
||||
**HOW TO WRITE CHECKLIST ITEMS - "Unit Tests for English"**:
|
||||
|
||||
❌ **WRONG** (Testing implementation):
|
||||
- "Verify landing page displays 3 episode cards"
|
||||
- "Test hover states work on desktop"
|
||||
- "Confirm logo click navigates home"
|
||||
|
||||
✅ **CORRECT** (Testing requirements quality):
|
||||
- "Are the exact number and layout of featured episodes specified?" [Completeness]
|
||||
- "Is 'prominent display' quantified with specific sizing/positioning?" [Clarity]
|
||||
- "Are hover state requirements consistent across all interactive elements?" [Consistency]
|
||||
- "Are keyboard navigation requirements defined for all interactive UI?" [Coverage]
|
||||
- "Is the fallback behavior specified when logo image fails to load?" [Edge Cases]
|
||||
- "Are loading states defined for asynchronous episode data?" [Completeness]
|
||||
- "Does the spec define visual hierarchy for competing UI elements?" [Clarity]
|
||||
|
||||
**ITEM STRUCTURE**:
|
||||
Each item should follow this pattern:
|
||||
- Question format asking about requirement quality
|
||||
- Focus on what's WRITTEN (or not written) in the spec/plan
|
||||
- Include quality dimension in brackets [Completeness/Clarity/Consistency/etc.]
|
||||
- Reference spec section `[Spec §X.Y]` when checking existing requirements
|
||||
- Use `[Gap]` marker when checking for missing requirements
|
||||
|
||||
**EXAMPLES BY QUALITY DIMENSION**:
|
||||
|
||||
Completeness:
|
||||
- "Are error handling requirements defined for all API failure modes? [Gap]"
|
||||
- "Are accessibility requirements specified for all interactive elements? [Completeness]"
|
||||
- "Are mobile breakpoint requirements defined for responsive layouts? [Gap]"
|
||||
|
||||
Clarity:
|
||||
- "Is 'fast loading' quantified with specific timing thresholds? [Clarity, Spec §NFR-2]"
|
||||
- "Are 'related episodes' selection criteria explicitly defined? [Clarity, Spec §FR-5]"
|
||||
- "Is 'prominent' defined with measurable visual properties? [Ambiguity, Spec §FR-4]"
|
||||
|
||||
Consistency:
|
||||
- "Do navigation requirements align across all pages? [Consistency, Spec §FR-10]"
|
||||
- "Are card component requirements consistent between landing and detail pages? [Consistency]"
|
||||
|
||||
Coverage:
|
||||
- "Are requirements defined for zero-state scenarios (no episodes)? [Coverage, Edge Case]"
|
||||
- "Are concurrent user interaction scenarios addressed? [Coverage, Gap]"
|
||||
- "Are requirements specified for partial data loading failures? [Coverage, Exception Flow]"
|
||||
|
||||
Measurability:
|
||||
- "Are visual hierarchy requirements measurable/testable? [Acceptance Criteria, Spec §FR-1]"
|
||||
- "Can 'balanced visual weight' be objectively verified? [Measurability, Spec §FR-2]"
|
||||
|
||||
**Scenario Classification & Coverage** (Requirements Quality Focus):
|
||||
- Check if requirements exist for: Primary, Alternate, Exception/Error, Recovery, Non-Functional scenarios
|
||||
- For each scenario class, ask: "Are [scenario type] requirements complete, clear, and consistent?"
|
||||
- If scenario class missing: "Are [scenario type] requirements intentionally excluded or missing? [Gap]"
|
||||
- Include resilience/rollback when state mutation occurs: "Are rollback requirements defined for migration failures? [Gap]"
|
||||
|
||||
**Traceability Requirements**:
|
||||
- MINIMUM: ≥80% of items MUST include at least one traceability reference
|
||||
- Each item should reference: spec section `[Spec §X.Y]`, or use markers: `[Gap]`, `[Ambiguity]`, `[Conflict]`, `[Assumption]`
|
||||
- If no ID system exists: "Is a requirement & acceptance criteria ID scheme established? [Traceability]"
|
||||
|
||||
**Surface & Resolve Issues** (Requirements Quality Problems):
|
||||
Ask questions about the requirements themselves:
|
||||
- Ambiguities: "Is the term 'fast' quantified with specific metrics? [Ambiguity, Spec §NFR-1]"
|
||||
- Conflicts: "Do navigation requirements conflict between §FR-10 and §FR-10a? [Conflict]"
|
||||
- Assumptions: "Is the assumption of 'always available podcast API' validated? [Assumption]"
|
||||
- Dependencies: "Are external podcast API requirements documented? [Dependency, Gap]"
|
||||
- Missing definitions: "Is 'visual hierarchy' defined with measurable criteria? [Gap]"
|
||||
|
||||
**Content Consolidation**:
|
||||
- Soft cap: If raw candidate items > 40, prioritize by risk/impact
|
||||
- Merge near-duplicates checking the same requirement aspect
|
||||
- If >5 low-impact edge cases, create one item: "Are edge cases X, Y, Z addressed in requirements? [Coverage]"
|
||||
|
||||
**🚫 ABSOLUTELY PROHIBITED** - These make it an implementation test, not a requirements test:
|
||||
- ❌ Any item starting with "Verify", "Test", "Confirm", "Check" + implementation behavior
|
||||
- ❌ References to code execution, user actions, system behavior
|
||||
- ❌ "Displays correctly", "works properly", "functions as expected"
|
||||
- ❌ "Click", "navigate", "render", "load", "execute"
|
||||
- ❌ Test cases, test plans, QA procedures
|
||||
- ❌ Implementation details (frameworks, APIs, algorithms)
|
||||
|
||||
**✅ REQUIRED PATTERNS** - These test requirements quality:
|
||||
- ✅ "Are [requirement type] defined/specified/documented for [scenario]?"
|
||||
- ✅ "Is [vague term] quantified/clarified with specific criteria?"
|
||||
- ✅ "Are requirements consistent between [section A] and [section B]?"
|
||||
- ✅ "Can [requirement] be objectively measured/verified?"
|
||||
- ✅ "Are [edge cases/scenarios] addressed in requirements?"
|
||||
- ✅ "Does the spec define [missing aspect]?"
|
||||
|
||||
6. **Structure Reference**: Generate the checklist following the canonical template in `.specify/templates/checklist-template.md` for title, meta section, category headings, and ID formatting. If template is unavailable, use: H1 title, purpose/created meta lines, `##` category sections containing `- [ ] CHK### <requirement item>` lines with globally incrementing IDs starting at CHK001.
|
||||
|
||||
7. **Report**: Output full path to created checklist, item count, and remind user that each run creates a new file. Summarize:
|
||||
- Focus areas selected
|
||||
- Depth level
|
||||
- Actor/timing
|
||||
- Any explicit user-specified must-have items incorporated
|
||||
|
||||
**Important**: Each `/speckit.checklist` command invocation creates a checklist file using short, descriptive names unless file already exists. This allows:
|
||||
|
||||
- Multiple checklists of different types (e.g., `ux.md`, `test.md`, `security.md`)
|
||||
- Simple, memorable filenames that indicate checklist purpose
|
||||
- Easy identification and navigation in the `checklists/` folder
|
||||
|
||||
To avoid clutter, use descriptive types and clean up obsolete checklists when done.
|
||||
|
||||
## Example Checklist Types & Sample Items
|
||||
|
||||
**UX Requirements Quality:** `ux.md`
|
||||
|
||||
Sample items (testing the requirements, NOT the implementation):
|
||||
|
||||
- "Are visual hierarchy requirements defined with measurable criteria? [Clarity, Spec §FR-1]"
|
||||
- "Is the number and positioning of UI elements explicitly specified? [Completeness, Spec §FR-1]"
|
||||
- "Are interaction state requirements (hover, focus, active) consistently defined? [Consistency]"
|
||||
- "Are accessibility requirements specified for all interactive elements? [Coverage, Gap]"
|
||||
- "Is fallback behavior defined when images fail to load? [Edge Case, Gap]"
|
||||
- "Can 'prominent display' be objectively measured? [Measurability, Spec §FR-4]"
|
||||
|
||||
**API Requirements Quality:** `api.md`
|
||||
|
||||
Sample items:
|
||||
|
||||
- "Are error response formats specified for all failure scenarios? [Completeness]"
|
||||
- "Are rate limiting requirements quantified with specific thresholds? [Clarity]"
|
||||
- "Are authentication requirements consistent across all endpoints? [Consistency]"
|
||||
- "Are retry/timeout requirements defined for external dependencies? [Coverage, Gap]"
|
||||
- "Is versioning strategy documented in requirements? [Gap]"
|
||||
|
||||
**Performance Requirements Quality:** `performance.md`
|
||||
|
||||
Sample items:
|
||||
|
||||
- "Are performance requirements quantified with specific metrics? [Clarity]"
|
||||
- "Are performance targets defined for all critical user journeys? [Coverage]"
|
||||
- "Are performance requirements under different load conditions specified? [Completeness]"
|
||||
- "Can performance requirements be objectively measured? [Measurability]"
|
||||
- "Are degradation requirements defined for high-load scenarios? [Edge Case, Gap]"
|
||||
|
||||
**Security Requirements Quality:** `security.md`
|
||||
|
||||
Sample items:
|
||||
|
||||
- "Are authentication requirements specified for all protected resources? [Coverage]"
|
||||
- "Are data protection requirements defined for sensitive information? [Completeness]"
|
||||
- "Is the threat model documented and requirements aligned to it? [Traceability]"
|
||||
- "Are security requirements consistent with compliance obligations? [Consistency]"
|
||||
- "Are security failure/breach response requirements defined? [Gap, Exception Flow]"
|
||||
|
||||
## Anti-Examples: What NOT To Do
|
||||
|
||||
**❌ WRONG - These test implementation, not requirements:**
|
||||
|
||||
```markdown
|
||||
- [ ] CHK001 - Verify landing page displays 3 episode cards [Spec §FR-001]
|
||||
- [ ] CHK002 - Test hover states work correctly on desktop [Spec §FR-003]
|
||||
- [ ] CHK003 - Confirm logo click navigates to home page [Spec §FR-010]
|
||||
- [ ] CHK004 - Check that related episodes section shows 3-5 items [Spec §FR-005]
|
||||
```
|
||||
|
||||
**✅ CORRECT - These test requirements quality:**
|
||||
|
||||
```markdown
|
||||
- [ ] CHK001 - Are the number and layout of featured episodes explicitly specified? [Completeness, Spec §FR-001]
|
||||
- [ ] CHK002 - Are hover state requirements consistently defined for all interactive elements? [Consistency, Spec §FR-003]
|
||||
- [ ] CHK003 - Are navigation requirements clear for all clickable brand elements? [Clarity, Spec §FR-010]
|
||||
- [ ] CHK004 - Is the selection criteria for related episodes documented? [Gap, Spec §FR-005]
|
||||
- [ ] CHK005 - Are loading state requirements defined for asynchronous episode data? [Gap]
|
||||
- [ ] CHK006 - Can "visual hierarchy" requirements be objectively measured? [Measurability, Spec §FR-001]
|
||||
```
|
||||
|
||||
**Key Differences:**
|
||||
|
||||
- Wrong: Tests if the system works correctly
|
||||
- Correct: Tests if the requirements are written correctly
|
||||
- Wrong: Verification of behavior
|
||||
- Correct: Validation of requirement quality
|
||||
- Wrong: "Does it do X?"
|
||||
- Correct: "Is X clearly specified?"
|
||||
@@ -1,181 +0,0 @@
|
||||
---
|
||||
description: Identify underspecified areas in the current feature spec by asking up to 5 highly targeted clarification questions and encoding answers back into the spec.
|
||||
handoffs:
|
||||
- label: Build Technical Plan
|
||||
agent: speckit.plan
|
||||
prompt: Create a plan for the spec. I am building with...
|
||||
---
|
||||
|
||||
## User Input
|
||||
|
||||
```text
|
||||
$ARGUMENTS
|
||||
```
|
||||
|
||||
You **MUST** consider the user input before proceeding (if not empty).
|
||||
|
||||
## Outline
|
||||
|
||||
Goal: Detect and reduce ambiguity or missing decision points in the active feature specification and record the clarifications directly in the spec file.
|
||||
|
||||
Note: This clarification workflow is expected to run (and be completed) BEFORE invoking `/speckit.plan`. If the user explicitly states they are skipping clarification (e.g., exploratory spike), you may proceed, but must warn that downstream rework risk increases.
|
||||
|
||||
Execution steps:
|
||||
|
||||
1. Run `.specify/scripts/bash/check-prerequisites.sh --json --paths-only` from repo root **once** (combined `--json --paths-only` mode / `-Json -PathsOnly`). Parse minimal JSON payload fields:
|
||||
- `FEATURE_DIR`
|
||||
- `FEATURE_SPEC`
|
||||
- (Optionally capture `IMPL_PLAN`, `TASKS` for future chained flows.)
|
||||
- If JSON parsing fails, abort and instruct user to re-run `/speckit.specify` or verify feature branch environment.
|
||||
- For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||
|
||||
2. Load the current spec file. Perform a structured ambiguity & coverage scan using this taxonomy. For each category, mark status: Clear / Partial / Missing. Produce an internal coverage map used for prioritization (do not output raw map unless no questions will be asked).
|
||||
|
||||
Functional Scope & Behavior:
|
||||
- Core user goals & success criteria
|
||||
- Explicit out-of-scope declarations
|
||||
- User roles / personas differentiation
|
||||
|
||||
Domain & Data Model:
|
||||
- Entities, attributes, relationships
|
||||
- Identity & uniqueness rules
|
||||
- Lifecycle/state transitions
|
||||
- Data volume / scale assumptions
|
||||
|
||||
Interaction & UX Flow:
|
||||
- Critical user journeys / sequences
|
||||
- Error/empty/loading states
|
||||
- Accessibility or localization notes
|
||||
|
||||
Non-Functional Quality Attributes:
|
||||
- Performance (latency, throughput targets)
|
||||
- Scalability (horizontal/vertical, limits)
|
||||
- Reliability & availability (uptime, recovery expectations)
|
||||
- Observability (logging, metrics, tracing signals)
|
||||
- Security & privacy (authN/Z, data protection, threat assumptions)
|
||||
- Compliance / regulatory constraints (if any)
|
||||
|
||||
Integration & External Dependencies:
|
||||
- External services/APIs and failure modes
|
||||
- Data import/export formats
|
||||
- Protocol/versioning assumptions
|
||||
|
||||
Edge Cases & Failure Handling:
|
||||
- Negative scenarios
|
||||
- Rate limiting / throttling
|
||||
- Conflict resolution (e.g., concurrent edits)
|
||||
|
||||
Constraints & Tradeoffs:
|
||||
- Technical constraints (language, storage, hosting)
|
||||
- Explicit tradeoffs or rejected alternatives
|
||||
|
||||
Terminology & Consistency:
|
||||
- Canonical glossary terms
|
||||
- Avoided synonyms / deprecated terms
|
||||
|
||||
Completion Signals:
|
||||
- Acceptance criteria testability
|
||||
- Measurable Definition of Done style indicators
|
||||
|
||||
Misc / Placeholders:
|
||||
- TODO markers / unresolved decisions
|
||||
- Ambiguous adjectives ("robust", "intuitive") lacking quantification
|
||||
|
||||
For each category with Partial or Missing status, add a candidate question opportunity unless:
|
||||
- Clarification would not materially change implementation or validation strategy
|
||||
- Information is better deferred to planning phase (note internally)
|
||||
|
||||
3. Generate (internally) a prioritized queue of candidate clarification questions (maximum 5). Do NOT output them all at once. Apply these constraints:
|
||||
- Maximum of 10 total questions across the whole session.
|
||||
- Each question must be answerable with EITHER:
|
||||
- A short multiple‑choice selection (2–5 distinct, mutually exclusive options), OR
|
||||
- A one-word / short‑phrase answer (explicitly constrain: "Answer in <=5 words").
|
||||
- Only include questions whose answers materially impact architecture, data modeling, task decomposition, test design, UX behavior, operational readiness, or compliance validation.
|
||||
- Ensure category coverage balance: attempt to cover the highest impact unresolved categories first; avoid asking two low-impact questions when a single high-impact area (e.g., security posture) is unresolved.
|
||||
- Exclude questions already answered, trivial stylistic preferences, or plan-level execution details (unless blocking correctness).
|
||||
- Favor clarifications that reduce downstream rework risk or prevent misaligned acceptance tests.
|
||||
- If more than 5 categories remain unresolved, select the top 5 by (Impact * Uncertainty) heuristic.
|
||||
|
||||
4. Sequential questioning loop (interactive):
|
||||
- Present EXACTLY ONE question at a time.
|
||||
- For multiple‑choice questions:
|
||||
- **Analyze all options** and determine the **most suitable option** based on:
|
||||
- Best practices for the project type
|
||||
- Common patterns in similar implementations
|
||||
- Risk reduction (security, performance, maintainability)
|
||||
- Alignment with any explicit project goals or constraints visible in the spec
|
||||
- Present your **recommended option prominently** at the top with clear reasoning (1-2 sentences explaining why this is the best choice).
|
||||
- Format as: `**Recommended:** Option [X] - <reasoning>`
|
||||
- Then render all options as a Markdown table:
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| A | <Option A description> |
|
||||
| B | <Option B description> |
|
||||
| C | <Option C description> (add D/E as needed up to 5) |
|
||||
| Short | Provide a different short answer (<=5 words) (Include only if free-form alternative is appropriate) |
|
||||
|
||||
- After the table, add: `You can reply with the option letter (e.g., "A"), accept the recommendation by saying "yes" or "recommended", or provide your own short answer.`
|
||||
- For short‑answer style (no meaningful discrete options):
|
||||
- Provide your **suggested answer** based on best practices and context.
|
||||
- Format as: `**Suggested:** <your proposed answer> - <brief reasoning>`
|
||||
- Then output: `Format: Short answer (<=5 words). You can accept the suggestion by saying "yes" or "suggested", or provide your own answer.`
|
||||
- After the user answers:
|
||||
- If the user replies with "yes", "recommended", or "suggested", use your previously stated recommendation/suggestion as the answer.
|
||||
- Otherwise, validate the answer maps to one option or fits the <=5 word constraint.
|
||||
- If ambiguous, ask for a quick disambiguation (count still belongs to same question; do not advance).
|
||||
- Once satisfactory, record it in working memory (do not yet write to disk) and move to the next queued question.
|
||||
- Stop asking further questions when:
|
||||
- All critical ambiguities resolved early (remaining queued items become unnecessary), OR
|
||||
- User signals completion ("done", "good", "no more"), OR
|
||||
- You reach 5 asked questions.
|
||||
- Never reveal future queued questions in advance.
|
||||
- If no valid questions exist at start, immediately report no critical ambiguities.
|
||||
|
||||
5. Integration after EACH accepted answer (incremental update approach):
|
||||
- Maintain in-memory representation of the spec (loaded once at start) plus the raw file contents.
|
||||
- For the first integrated answer in this session:
|
||||
- Ensure a `## Clarifications` section exists (create it just after the highest-level contextual/overview section per the spec template if missing).
|
||||
- Under it, create (if not present) a `### Session YYYY-MM-DD` subheading for today.
|
||||
- Append a bullet line immediately after acceptance: `- Q: <question> → A: <final answer>`.
|
||||
- Then immediately apply the clarification to the most appropriate section(s):
|
||||
- Functional ambiguity → Update or add a bullet in Functional Requirements.
|
||||
- User interaction / actor distinction → Update User Stories or Actors subsection (if present) with clarified role, constraint, or scenario.
|
||||
- Data shape / entities → Update Data Model (add fields, types, relationships) preserving ordering; note added constraints succinctly.
|
||||
- Non-functional constraint → Add/modify measurable criteria in Non-Functional / Quality Attributes section (convert vague adjective to metric or explicit target).
|
||||
- Edge case / negative flow → Add a new bullet under Edge Cases / Error Handling (or create such subsection if template provides placeholder for it).
|
||||
- Terminology conflict → Normalize term across spec; retain original only if necessary by adding `(formerly referred to as "X")` once.
|
||||
- If the clarification invalidates an earlier ambiguous statement, replace that statement instead of duplicating; leave no obsolete contradictory text.
|
||||
- Save the spec file AFTER each integration to minimize risk of context loss (atomic overwrite).
|
||||
- Preserve formatting: do not reorder unrelated sections; keep heading hierarchy intact.
|
||||
- Keep each inserted clarification minimal and testable (avoid narrative drift).
|
||||
|
||||
6. Validation (performed after EACH write plus final pass):
|
||||
- Clarifications session contains exactly one bullet per accepted answer (no duplicates).
|
||||
- Total asked (accepted) questions ≤ 5.
|
||||
- Updated sections contain no lingering vague placeholders the new answer was meant to resolve.
|
||||
- No contradictory earlier statement remains (scan for now-invalid alternative choices removed).
|
||||
- Markdown structure valid; only allowed new headings: `## Clarifications`, `### Session YYYY-MM-DD`.
|
||||
- Terminology consistency: same canonical term used across all updated sections.
|
||||
|
||||
7. Write the updated spec back to `FEATURE_SPEC`.
|
||||
|
||||
8. Report completion (after questioning loop ends or early termination):
|
||||
- Number of questions asked & answered.
|
||||
- Path to updated spec.
|
||||
- Sections touched (list names).
|
||||
- Coverage summary table listing each taxonomy category with Status: Resolved (was Partial/Missing and addressed), Deferred (exceeds question quota or better suited for planning), Clear (already sufficient), Outstanding (still Partial/Missing but low impact).
|
||||
- If any Outstanding or Deferred remain, recommend whether to proceed to `/speckit.plan` or run `/speckit.clarify` again later post-plan.
|
||||
- Suggested next command.
|
||||
|
||||
Behavior rules:
|
||||
|
||||
- If no meaningful ambiguities found (or all potential questions would be low-impact), respond: "No critical ambiguities detected worth formal clarification." and suggest proceeding.
|
||||
- If spec file missing, instruct user to run `/speckit.specify` first (do not create a new spec here).
|
||||
- Never exceed 5 total asked questions (clarification retries for a single question do not count as new questions).
|
||||
- Avoid speculative tech stack questions unless the absence blocks functional clarity.
|
||||
- Respect user early termination signals ("stop", "done", "proceed").
|
||||
- If no questions asked due to full coverage, output a compact coverage summary (all categories Clear) then suggest advancing.
|
||||
- If quota reached with unresolved high-impact categories remaining, explicitly flag them under Deferred with rationale.
|
||||
|
||||
Context for prioritization: $ARGUMENTS
|
||||
@@ -1,84 +0,0 @@
|
||||
---
|
||||
description: Create or update the project constitution from interactive or provided principle inputs, ensuring all dependent templates stay in sync.
|
||||
handoffs:
|
||||
- label: Build Specification
|
||||
agent: speckit.specify
|
||||
prompt: Implement the feature specification based on the updated constitution. I want to build...
|
||||
---
|
||||
|
||||
## User Input
|
||||
|
||||
```text
|
||||
$ARGUMENTS
|
||||
```
|
||||
|
||||
You **MUST** consider the user input before proceeding (if not empty).
|
||||
|
||||
## Outline
|
||||
|
||||
You are updating the project constitution at `.specify/memory/constitution.md`. This file is a TEMPLATE containing placeholder tokens in square brackets (e.g. `[PROJECT_NAME]`, `[PRINCIPLE_1_NAME]`). Your job is to (a) collect/derive concrete values, (b) fill the template precisely, and (c) propagate any amendments across dependent artifacts.
|
||||
|
||||
**Note**: If `.specify/memory/constitution.md` does not exist yet, it should have been initialized from `.specify/templates/constitution-template.md` during project setup. If it's missing, copy the template first.
|
||||
|
||||
Follow this execution flow:
|
||||
|
||||
1. Load the existing constitution at `.specify/memory/constitution.md`.
|
||||
- Identify every placeholder token of the form `[ALL_CAPS_IDENTIFIER]`.
|
||||
**IMPORTANT**: The user might require less or more principles than the ones used in the template. If a number is specified, respect that - follow the general template. You will update the doc accordingly.
|
||||
|
||||
2. Collect/derive values for placeholders:
|
||||
- If user input (conversation) supplies a value, use it.
|
||||
- Otherwise infer from existing repo context (README, docs, prior constitution versions if embedded).
|
||||
- For governance dates: `RATIFICATION_DATE` is the original adoption date (if unknown ask or mark TODO), `LAST_AMENDED_DATE` is today if changes are made, otherwise keep previous.
|
||||
- `CONSTITUTION_VERSION` must increment according to semantic versioning rules:
|
||||
- MAJOR: Backward incompatible governance/principle removals or redefinitions.
|
||||
- MINOR: New principle/section added or materially expanded guidance.
|
||||
- PATCH: Clarifications, wording, typo fixes, non-semantic refinements.
|
||||
- If version bump type ambiguous, propose reasoning before finalizing.
|
||||
|
||||
3. Draft the updated constitution content:
|
||||
- Replace every placeholder with concrete text (no bracketed tokens left except intentionally retained template slots that the project has chosen not to define yet—explicitly justify any left).
|
||||
- Preserve heading hierarchy and comments can be removed once replaced unless they still add clarifying guidance.
|
||||
- Ensure each Principle section: succinct name line, paragraph (or bullet list) capturing non‑negotiable rules, explicit rationale if not obvious.
|
||||
- Ensure Governance section lists amendment procedure, versioning policy, and compliance review expectations.
|
||||
|
||||
4. Consistency propagation checklist (convert prior checklist into active validations):
|
||||
- Read `.specify/templates/plan-template.md` and ensure any "Constitution Check" or rules align with updated principles.
|
||||
- Read `.specify/templates/spec-template.md` for scope/requirements alignment—update if constitution adds/removes mandatory sections or constraints.
|
||||
- Read `.specify/templates/tasks-template.md` and ensure task categorization reflects new or removed principle-driven task types (e.g., observability, versioning, testing discipline).
|
||||
- Read each command file in `.specify/templates/commands/*.md` (including this one) to verify no outdated references (agent-specific names like CLAUDE only) remain when generic guidance is required.
|
||||
- Read any runtime guidance docs (e.g., `README.md`, `docs/quickstart.md`, or agent-specific guidance files if present). Update references to principles changed.
|
||||
|
||||
5. Produce a Sync Impact Report (prepend as an HTML comment at top of the constitution file after update):
|
||||
- Version change: old → new
|
||||
- List of modified principles (old title → new title if renamed)
|
||||
- Added sections
|
||||
- Removed sections
|
||||
- Templates requiring updates (✅ updated / ⚠ pending) with file paths
|
||||
- Follow-up TODOs if any placeholders intentionally deferred.
|
||||
|
||||
6. Validation before final output:
|
||||
- No remaining unexplained bracket tokens.
|
||||
- Version line matches report.
|
||||
- Dates ISO format YYYY-MM-DD.
|
||||
- Principles are declarative, testable, and free of vague language ("should" → replace with MUST/SHOULD rationale where appropriate).
|
||||
|
||||
7. Write the completed constitution back to `.specify/memory/constitution.md` (overwrite).
|
||||
|
||||
8. Output a final summary to the user with:
|
||||
- New version and bump rationale.
|
||||
- Any files flagged for manual follow-up.
|
||||
- Suggested commit message (e.g., `docs: amend constitution to vX.Y.Z (principle additions + governance update)`).
|
||||
|
||||
Formatting & Style Requirements:
|
||||
|
||||
- Use Markdown headings exactly as in the template (do not demote/promote levels).
|
||||
- Wrap long rationale lines to keep readability (<100 chars ideally) but do not hard enforce with awkward breaks.
|
||||
- Keep a single blank line between sections.
|
||||
- Avoid trailing whitespace.
|
||||
|
||||
If the user supplies partial updates (e.g., only one principle revision), still perform validation and version decision steps.
|
||||
|
||||
If critical info missing (e.g., ratification date truly unknown), insert `TODO(<FIELD_NAME>): explanation` and include in the Sync Impact Report under deferred items.
|
||||
|
||||
Do not create a new template; always operate on the existing `.specify/memory/constitution.md` file.
|
||||
@@ -1,199 +0,0 @@
|
||||
---
|
||||
|
||||
description: Fix failing tests and implementation issues based on test reports
|
||||
|
||||
---
|
||||
|
||||
## User Input
|
||||
|
||||
```text
|
||||
$ARGUMENTS
|
||||
```
|
||||
|
||||
You **MUST** consider the user input before proceeding (if not empty).
|
||||
|
||||
## Goal
|
||||
|
||||
Analyze test failure reports, identify root causes, and fix implementation issues while preserving semantic protocol compliance.
|
||||
|
||||
## Operating Constraints
|
||||
|
||||
1. **USE CODER MODE**: Always switch to `coder` mode for code fixes
|
||||
2. **SEMANTIC PROTOCOL**: Never remove semantic annotations ([DEF], @TAGS). Only update code logic.
|
||||
3. **TEST DATA**: If tests use @TEST_DATA fixtures, preserve them when fixing
|
||||
4. **NO DELETION**: Never delete existing tests or semantic annotations
|
||||
5. **REPORT FIRST**: Always write a fix report before making changes
|
||||
|
||||
## Execution Steps
|
||||
|
||||
### 1. Load Test Report
|
||||
|
||||
**Required**: Test report file path (e.g., `specs/<feature>/tests/reports/2026-02-19-report.md`)
|
||||
|
||||
**Parse the report for**:
|
||||
- Failed test cases
|
||||
- Error messages
|
||||
- Stack traces
|
||||
- Expected vs actual behavior
|
||||
- Affected modules/files
|
||||
|
||||
### 2. Analyze Root Causes
|
||||
|
||||
For each failed test:
|
||||
|
||||
1. **Read the test file** to understand what it's testing
|
||||
2. **Read the implementation file** to find the bug
|
||||
3. **Check semantic protocol compliance**:
|
||||
- Does the implementation have correct [DEF] anchors?
|
||||
- Are @TAGS (@PRE, @POST, @UX_STATE, etc.) present?
|
||||
- Does the code match the TIER requirements?
|
||||
4. **Identify the fix**:
|
||||
- Logic error in implementation
|
||||
- Missing error handling
|
||||
- Incorrect API usage
|
||||
- State management issue
|
||||
|
||||
### 3. Write Fix Report
|
||||
|
||||
Create a structured fix report:
|
||||
|
||||
```markdown
|
||||
# Fix Report: [FEATURE]
|
||||
|
||||
**Date**: [YYYY-MM-DD]
|
||||
**Report**: [Test Report Path]
|
||||
**Fixer**: Coder Agent
|
||||
|
||||
## Summary
|
||||
|
||||
- Total Failed Tests: [X]
|
||||
- Total Fixed: [X]
|
||||
- Total Skipped: [X]
|
||||
|
||||
## Failed Tests Analysis
|
||||
|
||||
### Test: [Test Name]
|
||||
|
||||
**File**: `path/to/test.py`
|
||||
**Error**: [Error message]
|
||||
|
||||
**Root Cause**: [Explanation of why test failed]
|
||||
|
||||
**Fix Required**: [Description of fix]
|
||||
|
||||
**Status**: [Pending/In Progress/Completed]
|
||||
|
||||
## Fixes Applied
|
||||
|
||||
### Fix 1: [Description]
|
||||
|
||||
**Affected File**: `path/to/file.py`
|
||||
**Test Affected**: `[Test Name]`
|
||||
|
||||
**Changes**:
|
||||
```diff
|
||||
<<<<<<< SEARCH
|
||||
[Original Code]
|
||||
=======
|
||||
[Fixed Code]
|
||||
>>>>>>> REPLACE
|
||||
```
|
||||
|
||||
**Verification**: [How to verify fix works]
|
||||
|
||||
**Semantic Integrity**: [Confirmed annotations preserved]
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [ ] Run tests to verify fix: `cd backend && .venv/bin/python3 -m pytest`
|
||||
- [ ] Check for related failing tests
|
||||
- [ ] Update test documentation if needed
|
||||
```
|
||||
|
||||
### 4. Apply Fixes (in Coder Mode)
|
||||
|
||||
Switch to `coder` mode and apply fixes:
|
||||
|
||||
1. **Read the implementation file** to get exact content
|
||||
2. **Apply the fix** using apply_diff
|
||||
3. **Preserve all semantic annotations**:
|
||||
- Keep [DEF:...] and [/DEF:...] anchors
|
||||
- Keep all @TAGS (@PURPOSE, @LAYER, @TIER, @RELATION, @PRE, @POST, @UX_STATE, @UX_FEEDBACK, @UX_RECOVERY)
|
||||
4. **Only update code logic** to fix the bug
|
||||
5. **Run tests** to verify the fix
|
||||
|
||||
### 5. Verification
|
||||
|
||||
After applying fixes:
|
||||
|
||||
1. **Run tests**:
|
||||
```bash
|
||||
cd backend && .venv/bin/python3 -m pytest -v
|
||||
```
|
||||
or
|
||||
```bash
|
||||
cd frontend && npm run test
|
||||
```
|
||||
|
||||
2. **Check test results**:
|
||||
- Failed tests should now pass
|
||||
- No new tests should fail
|
||||
- Coverage should not decrease
|
||||
|
||||
3. **Update fix report** with results:
|
||||
- Mark fixes as completed
|
||||
- Add verification steps
|
||||
- Note any remaining issues
|
||||
|
||||
## Output
|
||||
|
||||
Generate final fix report:
|
||||
|
||||
```markdown
|
||||
# Fix Report: [FEATURE] - COMPLETED
|
||||
|
||||
**Date**: [YYYY-MM-DD]
|
||||
**Report**: [Test Report Path]
|
||||
**Fixer**: Coder Agent
|
||||
|
||||
## Summary
|
||||
|
||||
- Total Failed Tests: [X]
|
||||
- Total Fixed: [X] ✅
|
||||
- Total Skipped: [X]
|
||||
|
||||
## Fixes Applied
|
||||
|
||||
### Fix 1: [Description] ✅
|
||||
|
||||
**Affected File**: `path/to/file.py`
|
||||
**Test Affected**: `[Test Name]`
|
||||
|
||||
**Changes**: [Summary of changes]
|
||||
|
||||
**Verification**: All tests pass ✅
|
||||
|
||||
**Semantic Integrity**: Preserved ✅
|
||||
|
||||
## Test Results
|
||||
|
||||
```
|
||||
[Full test output showing all passing tests]
|
||||
```
|
||||
|
||||
## Recommendations
|
||||
|
||||
- [ ] Monitor for similar issues
|
||||
- [ ] Update documentation if needed
|
||||
- [ ] Consider adding more tests for edge cases
|
||||
|
||||
## Related Files
|
||||
|
||||
- Test Report: [path]
|
||||
- Implementation: [path]
|
||||
- Test File: [path]
|
||||
```
|
||||
|
||||
## Context for Fixing
|
||||
|
||||
$ARGUMENTS
|
||||
@@ -1,135 +0,0 @@
|
||||
---
|
||||
description: Execute the implementation plan by processing and executing all tasks defined in tasks.md
|
||||
---
|
||||
|
||||
## User Input
|
||||
|
||||
```text
|
||||
$ARGUMENTS
|
||||
```
|
||||
|
||||
You **MUST** consider the user input before proceeding (if not empty).
|
||||
|
||||
## Outline
|
||||
|
||||
1. Run `.specify/scripts/bash/check-prerequisites.sh --json --require-tasks --include-tasks` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||
|
||||
2. **Check checklists status** (if FEATURE_DIR/checklists/ exists):
|
||||
- Scan all checklist files in the checklists/ directory
|
||||
- For each checklist, count:
|
||||
- Total items: All lines matching `- [ ]` or `- [X]` or `- [x]`
|
||||
- Completed items: Lines matching `- [X]` or `- [x]`
|
||||
- Incomplete items: Lines matching `- [ ]`
|
||||
- Create a status table:
|
||||
|
||||
```text
|
||||
| Checklist | Total | Completed | Incomplete | Status |
|
||||
|-----------|-------|-----------|------------|--------|
|
||||
| ux.md | 12 | 12 | 0 | ✓ PASS |
|
||||
| test.md | 8 | 5 | 3 | ✗ FAIL |
|
||||
| security.md | 6 | 6 | 0 | ✓ PASS |
|
||||
```
|
||||
|
||||
- Calculate overall status:
|
||||
- **PASS**: All checklists have 0 incomplete items
|
||||
- **FAIL**: One or more checklists have incomplete items
|
||||
|
||||
- **If any checklist is incomplete**:
|
||||
- Display the table with incomplete item counts
|
||||
- **STOP** and ask: "Some checklists are incomplete. Do you want to proceed with implementation anyway? (yes/no)"
|
||||
- Wait for user response before continuing
|
||||
- If user says "no" or "wait" or "stop", halt execution
|
||||
- If user says "yes" or "proceed" or "continue", proceed to step 3
|
||||
|
||||
- **If all checklists are complete**:
|
||||
- Display the table showing all checklists passed
|
||||
- Automatically proceed to step 3
|
||||
|
||||
3. Load and analyze the implementation context:
|
||||
- **REQUIRED**: Read tasks.md for the complete task list and execution plan
|
||||
- **REQUIRED**: Read plan.md for tech stack, architecture, and file structure
|
||||
- **IF EXISTS**: Read data-model.md for entities and relationships
|
||||
- **IF EXISTS**: Read contracts/ for API specifications and test requirements
|
||||
- **IF EXISTS**: Read research.md for technical decisions and constraints
|
||||
- **IF EXISTS**: Read quickstart.md for integration scenarios
|
||||
|
||||
4. **Project Setup Verification**:
|
||||
- **REQUIRED**: Create/verify ignore files based on actual project setup:
|
||||
|
||||
**Detection & Creation Logic**:
|
||||
- Check if the following command succeeds to determine if the repository is a git repo (create/verify .gitignore if so):
|
||||
|
||||
```sh
|
||||
git rev-parse --git-dir 2>/dev/null
|
||||
```
|
||||
|
||||
- Check if Dockerfile* exists or Docker in plan.md → create/verify .dockerignore
|
||||
- Check if .eslintrc* exists → create/verify .eslintignore
|
||||
- Check if eslint.config.* exists → ensure the config's `ignores` entries cover required patterns
|
||||
- Check if .prettierrc* exists → create/verify .prettierignore
|
||||
- Check if .npmrc or package.json exists → create/verify .npmignore (if publishing)
|
||||
- Check if terraform files (*.tf) exist → create/verify .terraformignore
|
||||
- Check if .helmignore needed (helm charts present) → create/verify .helmignore
|
||||
|
||||
**If ignore file already exists**: Verify it contains essential patterns, append missing critical patterns only
|
||||
**If ignore file missing**: Create with full pattern set for detected technology
|
||||
|
||||
**Common Patterns by Technology** (from plan.md tech stack):
|
||||
- **Node.js/JavaScript/TypeScript**: `node_modules/`, `dist/`, `build/`, `*.log`, `.env*`
|
||||
- **Python**: `__pycache__/`, `*.pyc`, `.venv/`, `venv/`, `dist/`, `*.egg-info/`
|
||||
- **Java**: `target/`, `*.class`, `*.jar`, `.gradle/`, `build/`
|
||||
- **C#/.NET**: `bin/`, `obj/`, `*.user`, `*.suo`, `packages/`
|
||||
- **Go**: `*.exe`, `*.test`, `vendor/`, `*.out`
|
||||
- **Ruby**: `.bundle/`, `log/`, `tmp/`, `*.gem`, `vendor/bundle/`
|
||||
- **PHP**: `vendor/`, `*.log`, `*.cache`, `*.env`
|
||||
- **Rust**: `target/`, `debug/`, `release/`, `*.rs.bk`, `*.rlib`, `*.prof*`, `.idea/`, `*.log`, `.env*`
|
||||
- **Kotlin**: `build/`, `out/`, `.gradle/`, `.idea/`, `*.class`, `*.jar`, `*.iml`, `*.log`, `.env*`
|
||||
- **C++**: `build/`, `bin/`, `obj/`, `out/`, `*.o`, `*.so`, `*.a`, `*.exe`, `*.dll`, `.idea/`, `*.log`, `.env*`
|
||||
- **C**: `build/`, `bin/`, `obj/`, `out/`, `*.o`, `*.a`, `*.so`, `*.exe`, `Makefile`, `config.log`, `.idea/`, `*.log`, `.env*`
|
||||
- **Swift**: `.build/`, `DerivedData/`, `*.swiftpm/`, `Packages/`
|
||||
- **R**: `.Rproj.user/`, `.Rhistory`, `.RData`, `.Ruserdata`, `*.Rproj`, `packrat/`, `renv/`
|
||||
- **Universal**: `.DS_Store`, `Thumbs.db`, `*.tmp`, `*.swp`, `.vscode/`, `.idea/`
|
||||
|
||||
**Tool-Specific Patterns**:
|
||||
- **Docker**: `node_modules/`, `.git/`, `Dockerfile*`, `.dockerignore`, `*.log*`, `.env*`, `coverage/`
|
||||
- **ESLint**: `node_modules/`, `dist/`, `build/`, `coverage/`, `*.min.js`
|
||||
- **Prettier**: `node_modules/`, `dist/`, `build/`, `coverage/`, `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`
|
||||
- **Terraform**: `.terraform/`, `*.tfstate*`, `*.tfvars`, `.terraform.lock.hcl`
|
||||
- **Kubernetes/k8s**: `*.secret.yaml`, `secrets/`, `.kube/`, `kubeconfig*`, `*.key`, `*.crt`
|
||||
|
||||
5. Parse tasks.md structure and extract:
|
||||
- **Task phases**: Setup, Tests, Core, Integration, Polish
|
||||
- **Task dependencies**: Sequential vs parallel execution rules
|
||||
- **Task details**: ID, description, file paths, parallel markers [P]
|
||||
- **Execution flow**: Order and dependency requirements
|
||||
|
||||
6. Execute implementation following the task plan:
|
||||
- **Phase-by-phase execution**: Complete each phase before moving to the next
|
||||
- **Respect dependencies**: Run sequential tasks in order, parallel tasks [P] can run together
|
||||
- **Follow TDD approach**: Execute test tasks before their corresponding implementation tasks
|
||||
- **File-based coordination**: Tasks affecting the same files must run sequentially
|
||||
- **Validation checkpoints**: Verify each phase completion before proceeding
|
||||
|
||||
7. Implementation execution rules:
|
||||
- **Setup first**: Initialize project structure, dependencies, configuration
|
||||
- **Tests before code**: If you need to write tests for contracts, entities, and integration scenarios
|
||||
- **Core development**: Implement models, services, CLI commands, endpoints
|
||||
- **Integration work**: Database connections, middleware, logging, external services
|
||||
- **Polish and validation**: Unit tests, performance optimization, documentation
|
||||
|
||||
8. Progress tracking and error handling:
|
||||
- Report progress after each completed task
|
||||
- Halt execution if any non-parallel task fails
|
||||
- For parallel tasks [P], continue with successful tasks, report failed ones
|
||||
- Provide clear error messages with context for debugging
|
||||
- Suggest next steps if implementation cannot proceed
|
||||
- **IMPORTANT** For completed tasks, make sure to mark the task off as [X] in the tasks file.
|
||||
|
||||
9. Completion validation:
|
||||
- Verify all required tasks are completed
|
||||
- Check that implemented features match the original specification
|
||||
- Validate that tests pass and coverage meets requirements
|
||||
- Confirm the implementation follows the technical plan
|
||||
- Report final status with summary of completed work
|
||||
|
||||
Note: This command assumes a complete task breakdown exists in tasks.md. If tasks are incomplete or missing, suggest running `/speckit.tasks` first to regenerate the task list.
|
||||
@@ -1,90 +0,0 @@
|
||||
---
|
||||
description: Execute the implementation planning workflow using the plan template to generate design artifacts.
|
||||
handoffs:
|
||||
- label: Create Tasks
|
||||
agent: speckit.tasks
|
||||
prompt: Break the plan into tasks
|
||||
send: true
|
||||
- label: Create Checklist
|
||||
agent: speckit.checklist
|
||||
prompt: Create a checklist for the following domain...
|
||||
---
|
||||
|
||||
## User Input
|
||||
|
||||
```text
|
||||
$ARGUMENTS
|
||||
```
|
||||
|
||||
You **MUST** consider the user input before proceeding (if not empty).
|
||||
|
||||
## Outline
|
||||
|
||||
1. **Setup**: Run `.specify/scripts/bash/setup-plan.sh --json` from repo root and parse JSON for FEATURE_SPEC, IMPL_PLAN, SPECS_DIR, BRANCH. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||
|
||||
2. **Load context**: Read FEATURE_SPEC and `.specify/memory/constitution.md`. Load IMPL_PLAN template (already copied).
|
||||
|
||||
3. **Execute plan workflow**: Follow the structure in IMPL_PLAN template to:
|
||||
- Fill Technical Context (mark unknowns as "NEEDS CLARIFICATION")
|
||||
- Fill Constitution Check section from constitution
|
||||
- Evaluate gates (ERROR if violations unjustified)
|
||||
- Phase 0: Generate research.md (resolve all NEEDS CLARIFICATION)
|
||||
- Phase 1: Generate data-model.md, contracts/, quickstart.md
|
||||
- Phase 1: Update agent context by running the agent script
|
||||
- Re-evaluate Constitution Check post-design
|
||||
|
||||
4. **Stop and report**: Command ends after Phase 2 planning. Report branch, IMPL_PLAN path, and generated artifacts.
|
||||
|
||||
## Phases
|
||||
|
||||
### Phase 0: Outline & Research
|
||||
|
||||
1. **Extract unknowns from Technical Context** above:
|
||||
- For each NEEDS CLARIFICATION → research task
|
||||
- For each dependency → best practices task
|
||||
- For each integration → patterns task
|
||||
|
||||
2. **Generate and dispatch research agents**:
|
||||
|
||||
```text
|
||||
For each unknown in Technical Context:
|
||||
Task: "Research {unknown} for {feature context}"
|
||||
For each technology choice:
|
||||
Task: "Find best practices for {tech} in {domain}"
|
||||
```
|
||||
|
||||
3. **Consolidate findings** in `research.md` using format:
|
||||
- Decision: [what was chosen]
|
||||
- Rationale: [why chosen]
|
||||
- Alternatives considered: [what else evaluated]
|
||||
|
||||
**Output**: research.md with all NEEDS CLARIFICATION resolved
|
||||
|
||||
### Phase 1: Design & Contracts
|
||||
|
||||
**Prerequisites:** `research.md` complete
|
||||
|
||||
1. **Extract entities from feature spec** → `data-model.md`:
|
||||
- Entity name, fields, relationships
|
||||
- Validation rules from requirements
|
||||
- State transitions if applicable
|
||||
|
||||
2. **Define interface contracts** (if project has external interfaces) → `/contracts/`:
|
||||
- Identify what interfaces the project exposes to users or other systems
|
||||
- Document the contract format appropriate for the project type
|
||||
- Examples: public APIs for libraries, command schemas for CLI tools, endpoints for web services, grammars for parsers, UI contracts for applications
|
||||
- Skip if project is purely internal (build scripts, one-off tools, etc.)
|
||||
|
||||
3. **Agent context update**:
|
||||
- Run `.specify/scripts/bash/update-agent-context.sh agy`
|
||||
- These scripts detect which AI agent is in use
|
||||
- Update the appropriate agent-specific context file
|
||||
- Add only new technology from current plan
|
||||
- Preserve manual additions between markers
|
||||
|
||||
**Output**: data-model.md, /contracts/*, quickstart.md, agent-specific file
|
||||
|
||||
## Key rules
|
||||
|
||||
- Use absolute paths
|
||||
- ERROR on gate failures or unresolved clarifications
|
||||
@@ -1,258 +0,0 @@
|
||||
---
|
||||
description: Create or update the feature specification from a natural language feature description.
|
||||
handoffs:
|
||||
- label: Build Technical Plan
|
||||
agent: speckit.plan
|
||||
prompt: Create a plan for the spec. I am building with...
|
||||
- label: Clarify Spec Requirements
|
||||
agent: speckit.clarify
|
||||
prompt: Clarify specification requirements
|
||||
send: true
|
||||
---
|
||||
|
||||
## User Input
|
||||
|
||||
```text
|
||||
$ARGUMENTS
|
||||
```
|
||||
|
||||
You **MUST** consider the user input before proceeding (if not empty).
|
||||
|
||||
## Outline
|
||||
|
||||
The text the user typed after `/speckit.specify` in the triggering message **is** the feature description. Assume you always have it available in this conversation even if `$ARGUMENTS` appears literally below. Do not ask the user to repeat it unless they provided an empty command.
|
||||
|
||||
Given that feature description, do this:
|
||||
|
||||
1. **Generate a concise short name** (2-4 words) for the branch:
|
||||
- Analyze the feature description and extract the most meaningful keywords
|
||||
- Create a 2-4 word short name that captures the essence of the feature
|
||||
- Use action-noun format when possible (e.g., "add-user-auth", "fix-payment-bug")
|
||||
- Preserve technical terms and acronyms (OAuth2, API, JWT, etc.)
|
||||
- Keep it concise but descriptive enough to understand the feature at a glance
|
||||
- Examples:
|
||||
- "I want to add user authentication" → "user-auth"
|
||||
- "Implement OAuth2 integration for the API" → "oauth2-api-integration"
|
||||
- "Create a dashboard for analytics" → "analytics-dashboard"
|
||||
- "Fix payment processing timeout bug" → "fix-payment-timeout"
|
||||
|
||||
2. **Check for existing branches before creating new one**:
|
||||
|
||||
a. First, fetch all remote branches to ensure we have the latest information:
|
||||
|
||||
```bash
|
||||
git fetch --all --prune
|
||||
```
|
||||
|
||||
b. Find the highest feature number across all sources for the short-name:
|
||||
- Remote branches: `git ls-remote --heads origin | grep -E 'refs/heads/[0-9]+-<short-name>$'`
|
||||
- Local branches: `git branch | grep -E '^[* ]*[0-9]+-<short-name>$'`
|
||||
- Specs directories: Check for directories matching `specs/[0-9]+-<short-name>`
|
||||
|
||||
c. Determine the next available number:
|
||||
- Extract all numbers from all three sources
|
||||
- Find the highest number N
|
||||
- Use N+1 for the new branch number
|
||||
|
||||
d. Run the script `.specify/scripts/bash/create-new-feature.sh --json "$ARGUMENTS"` with the calculated number and short-name:
|
||||
- Pass `--number N+1` and `--short-name "your-short-name"` along with the feature description
|
||||
- Bash example: `.specify/scripts/bash/create-new-feature.sh --json "$ARGUMENTS" --json --number 5 --short-name "user-auth" "Add user authentication"`
|
||||
- PowerShell example: `.specify/scripts/bash/create-new-feature.sh --json "$ARGUMENTS" -Json -Number 5 -ShortName "user-auth" "Add user authentication"`
|
||||
|
||||
**IMPORTANT**:
|
||||
- Check all three sources (remote branches, local branches, specs directories) to find the highest number
|
||||
- Only match branches/directories with the exact short-name pattern
|
||||
- If no existing branches/directories found with this short-name, start with number 1
|
||||
- You must only ever run this script once per feature
|
||||
- The JSON is provided in the terminal as output - always refer to it to get the actual content you're looking for
|
||||
- The JSON output will contain BRANCH_NAME and SPEC_FILE paths
|
||||
- For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot")
|
||||
|
||||
3. Load `.specify/templates/spec-template.md` to understand required sections.
|
||||
|
||||
4. Follow this execution flow:
|
||||
|
||||
1. Parse user description from Input
|
||||
If empty: ERROR "No feature description provided"
|
||||
2. Extract key concepts from description
|
||||
Identify: actors, actions, data, constraints
|
||||
3. For unclear aspects:
|
||||
- Make informed guesses based on context and industry standards
|
||||
- Only mark with [NEEDS CLARIFICATION: specific question] if:
|
||||
- The choice significantly impacts feature scope or user experience
|
||||
- Multiple reasonable interpretations exist with different implications
|
||||
- No reasonable default exists
|
||||
- **LIMIT: Maximum 3 [NEEDS CLARIFICATION] markers total**
|
||||
- Prioritize clarifications by impact: scope > security/privacy > user experience > technical details
|
||||
4. Fill User Scenarios & Testing section
|
||||
If no clear user flow: ERROR "Cannot determine user scenarios"
|
||||
5. Generate Functional Requirements
|
||||
Each requirement must be testable
|
||||
Use reasonable defaults for unspecified details (document assumptions in Assumptions section)
|
||||
6. Define Success Criteria
|
||||
Create measurable, technology-agnostic outcomes
|
||||
Include both quantitative metrics (time, performance, volume) and qualitative measures (user satisfaction, task completion)
|
||||
Each criterion must be verifiable without implementation details
|
||||
7. Identify Key Entities (if data involved)
|
||||
8. Return: SUCCESS (spec ready for planning)
|
||||
|
||||
5. Write the specification to SPEC_FILE using the template structure, replacing placeholders with concrete details derived from the feature description (arguments) while preserving section order and headings.
|
||||
|
||||
6. **Specification Quality Validation**: After writing the initial spec, validate it against quality criteria:
|
||||
|
||||
a. **Create Spec Quality Checklist**: Generate a checklist file at `FEATURE_DIR/checklists/requirements.md` using the checklist template structure with these validation items:
|
||||
|
||||
```markdown
|
||||
# Specification Quality Checklist: [FEATURE NAME]
|
||||
|
||||
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
||||
**Created**: [DATE]
|
||||
**Feature**: [Link to spec.md]
|
||||
|
||||
## Content Quality
|
||||
|
||||
- [ ] No implementation details (languages, frameworks, APIs)
|
||||
- [ ] Focused on user value and business needs
|
||||
- [ ] Written for non-technical stakeholders
|
||||
- [ ] All mandatory sections completed
|
||||
|
||||
## Requirement Completeness
|
||||
|
||||
- [ ] No [NEEDS CLARIFICATION] markers remain
|
||||
- [ ] Requirements are testable and unambiguous
|
||||
- [ ] Success criteria are measurable
|
||||
- [ ] Success criteria are technology-agnostic (no implementation details)
|
||||
- [ ] All acceptance scenarios are defined
|
||||
- [ ] Edge cases are identified
|
||||
- [ ] Scope is clearly bounded
|
||||
- [ ] Dependencies and assumptions identified
|
||||
|
||||
## Feature Readiness
|
||||
|
||||
- [ ] All functional requirements have clear acceptance criteria
|
||||
- [ ] User scenarios cover primary flows
|
||||
- [ ] Feature meets measurable outcomes defined in Success Criteria
|
||||
- [ ] No implementation details leak into specification
|
||||
|
||||
## Notes
|
||||
|
||||
- Items marked incomplete require spec updates before `/speckit.clarify` or `/speckit.plan`
|
||||
```
|
||||
|
||||
b. **Run Validation Check**: Review the spec against each checklist item:
|
||||
- For each item, determine if it passes or fails
|
||||
- Document specific issues found (quote relevant spec sections)
|
||||
|
||||
c. **Handle Validation Results**:
|
||||
|
||||
- **If all items pass**: Mark checklist complete and proceed to step 6
|
||||
|
||||
- **If items fail (excluding [NEEDS CLARIFICATION])**:
|
||||
1. List the failing items and specific issues
|
||||
2. Update the spec to address each issue
|
||||
3. Re-run validation until all items pass (max 3 iterations)
|
||||
4. If still failing after 3 iterations, document remaining issues in checklist notes and warn user
|
||||
|
||||
- **If [NEEDS CLARIFICATION] markers remain**:
|
||||
1. Extract all [NEEDS CLARIFICATION: ...] markers from the spec
|
||||
2. **LIMIT CHECK**: If more than 3 markers exist, keep only the 3 most critical (by scope/security/UX impact) and make informed guesses for the rest
|
||||
3. For each clarification needed (max 3), present options to user in this format:
|
||||
|
||||
```markdown
|
||||
## Question [N]: [Topic]
|
||||
|
||||
**Context**: [Quote relevant spec section]
|
||||
|
||||
**What we need to know**: [Specific question from NEEDS CLARIFICATION marker]
|
||||
|
||||
**Suggested Answers**:
|
||||
|
||||
| Option | Answer | Implications |
|
||||
|--------|--------|--------------|
|
||||
| A | [First suggested answer] | [What this means for the feature] |
|
||||
| B | [Second suggested answer] | [What this means for the feature] |
|
||||
| C | [Third suggested answer] | [What this means for the feature] |
|
||||
| Custom | Provide your own answer | [Explain how to provide custom input] |
|
||||
|
||||
**Your choice**: _[Wait for user response]_
|
||||
```
|
||||
|
||||
4. **CRITICAL - Table Formatting**: Ensure markdown tables are properly formatted:
|
||||
- Use consistent spacing with pipes aligned
|
||||
- Each cell should have spaces around content: `| Content |` not `|Content|`
|
||||
- Header separator must have at least 3 dashes: `|--------|`
|
||||
- Test that the table renders correctly in markdown preview
|
||||
5. Number questions sequentially (Q1, Q2, Q3 - max 3 total)
|
||||
6. Present all questions together before waiting for responses
|
||||
7. Wait for user to respond with their choices for all questions (e.g., "Q1: A, Q2: Custom - [details], Q3: B")
|
||||
8. Update the spec by replacing each [NEEDS CLARIFICATION] marker with the user's selected or provided answer
|
||||
9. Re-run validation after all clarifications are resolved
|
||||
|
||||
d. **Update Checklist**: After each validation iteration, update the checklist file with current pass/fail status
|
||||
|
||||
7. Report completion with branch name, spec file path, checklist results, and readiness for the next phase (`/speckit.clarify` or `/speckit.plan`).
|
||||
|
||||
**NOTE:** The script creates and checks out the new branch and initializes the spec file before writing.
|
||||
|
||||
## General Guidelines
|
||||
|
||||
## Quick Guidelines
|
||||
|
||||
- Focus on **WHAT** users need and **WHY**.
|
||||
- Avoid HOW to implement (no tech stack, APIs, code structure).
|
||||
- Written for business stakeholders, not developers.
|
||||
- DO NOT create any checklists that are embedded in the spec. That will be a separate command.
|
||||
|
||||
### Section Requirements
|
||||
|
||||
- **Mandatory sections**: Must be completed for every feature
|
||||
- **Optional sections**: Include only when relevant to the feature
|
||||
- When a section doesn't apply, remove it entirely (don't leave as "N/A")
|
||||
|
||||
### For AI Generation
|
||||
|
||||
When creating this spec from a user prompt:
|
||||
|
||||
1. **Make informed guesses**: Use context, industry standards, and common patterns to fill gaps
|
||||
2. **Document assumptions**: Record reasonable defaults in the Assumptions section
|
||||
3. **Limit clarifications**: Maximum 3 [NEEDS CLARIFICATION] markers - use only for critical decisions that:
|
||||
- Significantly impact feature scope or user experience
|
||||
- Have multiple reasonable interpretations with different implications
|
||||
- Lack any reasonable default
|
||||
4. **Prioritize clarifications**: scope > security/privacy > user experience > technical details
|
||||
5. **Think like a tester**: Every vague requirement should fail the "testable and unambiguous" checklist item
|
||||
6. **Common areas needing clarification** (only if no reasonable default exists):
|
||||
- Feature scope and boundaries (include/exclude specific use cases)
|
||||
- User types and permissions (if multiple conflicting interpretations possible)
|
||||
- Security/compliance requirements (when legally/financially significant)
|
||||
|
||||
**Examples of reasonable defaults** (don't ask about these):
|
||||
|
||||
- Data retention: Industry-standard practices for the domain
|
||||
- Performance targets: Standard web/mobile app expectations unless specified
|
||||
- Error handling: User-friendly messages with appropriate fallbacks
|
||||
- Authentication method: Standard session-based or OAuth2 for web apps
|
||||
- Integration patterns: Use project-appropriate patterns (REST/GraphQL for web services, function calls for libraries, CLI args for tools, etc.)
|
||||
|
||||
### Success Criteria Guidelines
|
||||
|
||||
Success criteria must be:
|
||||
|
||||
1. **Measurable**: Include specific metrics (time, percentage, count, rate)
|
||||
2. **Technology-agnostic**: No mention of frameworks, languages, databases, or tools
|
||||
3. **User-focused**: Describe outcomes from user/business perspective, not system internals
|
||||
4. **Verifiable**: Can be tested/validated without knowing implementation details
|
||||
|
||||
**Good examples**:
|
||||
|
||||
- "Users can complete checkout in under 3 minutes"
|
||||
- "System supports 10,000 concurrent users"
|
||||
- "95% of searches return results in under 1 second"
|
||||
- "Task completion rate improves by 40%"
|
||||
|
||||
**Bad examples** (implementation-focused):
|
||||
|
||||
- "API response time is under 200ms" (too technical, use "Users see results instantly")
|
||||
- "Database can handle 1000 TPS" (implementation detail, use user-facing metric)
|
||||
- "React components render efficiently" (framework-specific)
|
||||
- "Redis cache hit rate above 80%" (technology-specific)
|
||||
@@ -1,137 +0,0 @@
|
||||
---
|
||||
description: Generate an actionable, dependency-ordered tasks.md for the feature based on available design artifacts.
|
||||
handoffs:
|
||||
- label: Analyze For Consistency
|
||||
agent: speckit.analyze
|
||||
prompt: Run a project analysis for consistency
|
||||
send: true
|
||||
- label: Implement Project
|
||||
agent: speckit.implement
|
||||
prompt: Start the implementation in phases
|
||||
send: true
|
||||
---
|
||||
|
||||
## User Input
|
||||
|
||||
```text
|
||||
$ARGUMENTS
|
||||
```
|
||||
|
||||
You **MUST** consider the user input before proceeding (if not empty).
|
||||
|
||||
## Outline
|
||||
|
||||
1. **Setup**: Run `.specify/scripts/bash/check-prerequisites.sh --json` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||
|
||||
2. **Load design documents**: Read from FEATURE_DIR:
|
||||
- **Required**: plan.md (tech stack, libraries, structure), spec.md (user stories with priorities)
|
||||
- **Optional**: data-model.md (entities), contracts/ (interface contracts), research.md (decisions), quickstart.md (test scenarios)
|
||||
- Note: Not all projects have all documents. Generate tasks based on what's available.
|
||||
|
||||
3. **Execute task generation workflow**:
|
||||
- Load plan.md and extract tech stack, libraries, project structure
|
||||
- Load spec.md and extract user stories with their priorities (P1, P2, P3, etc.)
|
||||
- If data-model.md exists: Extract entities and map to user stories
|
||||
- If contracts/ exists: Map interface contracts to user stories
|
||||
- If research.md exists: Extract decisions for setup tasks
|
||||
- Generate tasks organized by user story (see Task Generation Rules below)
|
||||
- Generate dependency graph showing user story completion order
|
||||
- Create parallel execution examples per user story
|
||||
- Validate task completeness (each user story has all needed tasks, independently testable)
|
||||
|
||||
4. **Generate tasks.md**: Use `.specify/templates/tasks-template.md` as structure, fill with:
|
||||
- Correct feature name from plan.md
|
||||
- Phase 1: Setup tasks (project initialization)
|
||||
- Phase 2: Foundational tasks (blocking prerequisites for all user stories)
|
||||
- Phase 3+: One phase per user story (in priority order from spec.md)
|
||||
- Each phase includes: story goal, independent test criteria, tests (if requested), implementation tasks
|
||||
- Final Phase: Polish & cross-cutting concerns
|
||||
- All tasks must follow the strict checklist format (see Task Generation Rules below)
|
||||
- Clear file paths for each task
|
||||
- Dependencies section showing story completion order
|
||||
- Parallel execution examples per story
|
||||
- Implementation strategy section (MVP first, incremental delivery)
|
||||
|
||||
5. **Report**: Output path to generated tasks.md and summary:
|
||||
- Total task count
|
||||
- Task count per user story
|
||||
- Parallel opportunities identified
|
||||
- Independent test criteria for each story
|
||||
- Suggested MVP scope (typically just User Story 1)
|
||||
- Format validation: Confirm ALL tasks follow the checklist format (checkbox, ID, labels, file paths)
|
||||
|
||||
Context for task generation: $ARGUMENTS
|
||||
|
||||
The tasks.md should be immediately executable - each task must be specific enough that an LLM can complete it without additional context.
|
||||
|
||||
## Task Generation Rules
|
||||
|
||||
**CRITICAL**: Tasks MUST be organized by user story to enable independent implementation and testing.
|
||||
|
||||
**Tests are OPTIONAL**: Only generate test tasks if explicitly requested in the feature specification or if user requests TDD approach.
|
||||
|
||||
### Checklist Format (REQUIRED)
|
||||
|
||||
Every task MUST strictly follow this format:
|
||||
|
||||
```text
|
||||
- [ ] [TaskID] [P?] [Story?] Description with file path
|
||||
```
|
||||
|
||||
**Format Components**:
|
||||
|
||||
1. **Checkbox**: ALWAYS start with `- [ ]` (markdown checkbox)
|
||||
2. **Task ID**: Sequential number (T001, T002, T003...) in execution order
|
||||
3. **[P] marker**: Include ONLY if task is parallelizable (different files, no dependencies on incomplete tasks)
|
||||
4. **[Story] label**: REQUIRED for user story phase tasks only
|
||||
- Format: [US1], [US2], [US3], etc. (maps to user stories from spec.md)
|
||||
- Setup phase: NO story label
|
||||
- Foundational phase: NO story label
|
||||
- User Story phases: MUST have story label
|
||||
- Polish phase: NO story label
|
||||
5. **Description**: Clear action with exact file path
|
||||
|
||||
**Examples**:
|
||||
|
||||
- ✅ CORRECT: `- [ ] T001 Create project structure per implementation plan`
|
||||
- ✅ CORRECT: `- [ ] T005 [P] Implement authentication middleware in src/middleware/auth.py`
|
||||
- ✅ CORRECT: `- [ ] T012 [P] [US1] Create User model in src/models/user.py`
|
||||
- ✅ CORRECT: `- [ ] T014 [US1] Implement UserService in src/services/user_service.py`
|
||||
- ❌ WRONG: `- [ ] Create User model` (missing ID and Story label)
|
||||
- ❌ WRONG: `T001 [US1] Create model` (missing checkbox)
|
||||
- ❌ WRONG: `- [ ] [US1] Create User model` (missing Task ID)
|
||||
- ❌ WRONG: `- [ ] T001 [US1] Create model` (missing file path)
|
||||
|
||||
### Task Organization
|
||||
|
||||
1. **From User Stories (spec.md)** - PRIMARY ORGANIZATION:
|
||||
- Each user story (P1, P2, P3...) gets its own phase
|
||||
- Map all related components to their story:
|
||||
- Models needed for that story
|
||||
- Services needed for that story
|
||||
- Interfaces/UI needed for that story
|
||||
- If tests requested: Tests specific to that story
|
||||
- Mark story dependencies (most stories should be independent)
|
||||
|
||||
2. **From Contracts**:
|
||||
- Map each interface contract → to the user story it serves
|
||||
- If tests requested: Each interface contract → contract test task [P] before implementation in that story's phase
|
||||
|
||||
3. **From Data Model**:
|
||||
- Map each entity to the user story(ies) that need it
|
||||
- If entity serves multiple stories: Put in earliest story or Setup phase
|
||||
- Relationships → service layer tasks in appropriate story phase
|
||||
|
||||
4. **From Setup/Infrastructure**:
|
||||
- Shared infrastructure → Setup phase (Phase 1)
|
||||
- Foundational/blocking tasks → Foundational phase (Phase 2)
|
||||
- Story-specific setup → within that story's phase
|
||||
|
||||
### Phase Structure
|
||||
|
||||
- **Phase 1**: Setup (project initialization)
|
||||
- **Phase 2**: Foundational (blocking prerequisites - MUST complete before user stories)
|
||||
- **Phase 3+**: User Stories in priority order (P1, P2, P3...)
|
||||
- Within each story: Tests (if requested) → Models → Services → Endpoints → Integration
|
||||
- Each phase should be a complete, independently testable increment
|
||||
- **Final Phase**: Polish & Cross-Cutting Concerns
|
||||
@@ -1,30 +0,0 @@
|
||||
---
|
||||
description: Convert existing tasks into actionable, dependency-ordered GitHub issues for the feature based on available design artifacts.
|
||||
tools: ['github/github-mcp-server/issue_write']
|
||||
---
|
||||
|
||||
## User Input
|
||||
|
||||
```text
|
||||
$ARGUMENTS
|
||||
```
|
||||
|
||||
You **MUST** consider the user input before proceeding (if not empty).
|
||||
|
||||
## Outline
|
||||
|
||||
1. Run `.specify/scripts/bash/check-prerequisites.sh --json --require-tasks --include-tasks` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||
1. From the executed script, extract the path to **tasks**.
|
||||
1. Get the Git remote by running:
|
||||
|
||||
```bash
|
||||
git config --get remote.origin.url
|
||||
```
|
||||
|
||||
> [!CAUTION]
|
||||
> ONLY PROCEED TO NEXT STEPS IF THE REMOTE IS A GITHUB URL
|
||||
|
||||
1. For each task in the list, use the GitHub MCP server to create a new issue in the repository that is representative of the Git remote.
|
||||
|
||||
> [!CAUTION]
|
||||
> UNDER NO CIRCUMSTANCES EVER CREATE ISSUES IN REPOSITORIES THAT DO NOT MATCH THE REMOTE URL
|
||||
@@ -1,178 +0,0 @@
|
||||
---
|
||||
|
||||
description: Generate tests, manage test documentation, and ensure maximum code coverage
|
||||
|
||||
---
|
||||
|
||||
## User Input
|
||||
|
||||
```text
|
||||
$ARGUMENTS
|
||||
```
|
||||
|
||||
You **MUST** consider the user input before proceeding (if not empty).
|
||||
|
||||
## Goal
|
||||
|
||||
Execute full testing cycle: analyze code for testable modules, write tests with proper coverage, maintain test documentation, and ensure no test duplication or deletion.
|
||||
|
||||
## Operating Constraints
|
||||
|
||||
1. **NEVER delete existing tests** - Only update if they fail due to bugs in the test or implementation
|
||||
2. **NEVER duplicate tests** - Check existing tests first before creating new ones
|
||||
3. **Use TEST_DATA fixtures** - For CRITICAL tier modules, read @TEST_DATA from .specify/memory/semantics.md
|
||||
4. **Co-location required** - Write tests in `__tests__` directories relative to the code being tested
|
||||
|
||||
## Execution Steps
|
||||
|
||||
### 1. Analyze Context
|
||||
|
||||
Run `.specify/scripts/bash/check-prerequisites.sh --json --require-tasks --include-tasks` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS.
|
||||
|
||||
Determine:
|
||||
- FEATURE_DIR - where the feature is located
|
||||
- TASKS_FILE - path to tasks.md
|
||||
- Which modules need testing based on task status
|
||||
|
||||
### 2. Load Relevant Artifacts
|
||||
|
||||
**From tasks.md:**
|
||||
- Identify completed implementation tasks (not test tasks)
|
||||
- Extract file paths that need tests
|
||||
|
||||
**From .specify/memory/semantics.md:**
|
||||
- Read @TIER annotations for modules
|
||||
- For CRITICAL modules: Read @TEST_DATA fixtures
|
||||
|
||||
**From existing tests:**
|
||||
- Scan `__tests__` directories for existing tests
|
||||
- Identify test patterns and coverage gaps
|
||||
|
||||
### 3. Test Coverage Analysis
|
||||
|
||||
Create coverage matrix:
|
||||
|
||||
| Module | File | Has Tests | TIER | TEST_DATA Available |
|
||||
|--------|------|-----------|------|-------------------|
|
||||
| ... | ... | ... | ... | ... |
|
||||
|
||||
### 4. Write Tests (TDD Approach)
|
||||
|
||||
For each module requiring tests:
|
||||
|
||||
1. **Check existing tests**: Scan `__tests__/` for duplicates
|
||||
2. **Read TEST_DATA**: If CRITICAL tier, read @TEST_DATA from .specify/memory/semantics.md
|
||||
3. **Write test**: Follow co-location strategy
|
||||
- Python: `src/module/__tests__/test_module.py`
|
||||
- Svelte: `src/lib/components/__tests__/test_component.test.js`
|
||||
4. **Use mocks**: Use `unittest.mock.MagicMock` for external dependencies
|
||||
|
||||
### 4a. UX Contract Testing (Frontend Components)
|
||||
|
||||
For Svelte components with `@UX_STATE`, `@UX_FEEDBACK`, `@UX_RECOVERY` tags:
|
||||
|
||||
1. **Parse UX tags**: Read component file and extract all `@UX_*` annotations
|
||||
2. **Generate UX tests**: Create tests for each UX state transition
|
||||
```javascript
|
||||
// Example: Testing @UX_STATE: Idle -> Expanded
|
||||
it('should transition from Idle to Expanded on toggle click', async () => {
|
||||
render(Sidebar);
|
||||
const toggleBtn = screen.getByRole('button', { name: /toggle/i });
|
||||
await fireEvent.click(toggleBtn);
|
||||
expect(screen.getByTestId('sidebar')).toHaveClass('expanded');
|
||||
});
|
||||
```
|
||||
3. **Test @UX_FEEDBACK**: Verify visual feedback (toast, shake, color changes)
|
||||
4. **Test @UX_RECOVERY**: Verify error recovery mechanisms (retry, clear input)
|
||||
5. **Use @UX_TEST fixtures**: If component has `@UX_TEST` tags, use them as test specifications
|
||||
|
||||
**UX Test Template:**
|
||||
```javascript
|
||||
// [DEF:__tests__/test_Component:Module]
|
||||
// @RELATION: VERIFIES -> ../Component.svelte
|
||||
// @PURPOSE: Test UX states and transitions
|
||||
|
||||
describe('Component UX States', () => {
|
||||
// @UX_STATE: Idle -> {action: click, expected: Active}
|
||||
it('should transition Idle -> Active on click', async () => { ... });
|
||||
|
||||
// @UX_FEEDBACK: Toast on success
|
||||
it('should show toast on successful action', async () => { ... });
|
||||
|
||||
// @UX_RECOVERY: Retry on error
|
||||
it('should allow retry on error', async () => { ... });
|
||||
});
|
||||
```
|
||||
|
||||
### 5. Test Documentation
|
||||
|
||||
Create/update documentation in `specs/<feature>/tests/`:
|
||||
|
||||
```
|
||||
tests/
|
||||
├── README.md # Test strategy and overview
|
||||
├── coverage.md # Coverage matrix and reports
|
||||
└── reports/
|
||||
└── YYYY-MM-DD-report.md
|
||||
```
|
||||
|
||||
### 6. Execute Tests
|
||||
|
||||
Run tests and report results:
|
||||
|
||||
**Backend:**
|
||||
```bash
|
||||
cd backend && .venv/bin/python3 -m pytest -v
|
||||
```
|
||||
|
||||
**Frontend:**
|
||||
```bash
|
||||
cd frontend && npm run test
|
||||
```
|
||||
|
||||
### 7. Update Tasks
|
||||
|
||||
Mark test tasks as completed in tasks.md with:
|
||||
- Test file path
|
||||
- Coverage achieved
|
||||
- Any issues found
|
||||
|
||||
## Output
|
||||
|
||||
Generate test execution report:
|
||||
|
||||
```markdown
|
||||
# Test Report: [FEATURE]
|
||||
|
||||
**Date**: [YYYY-MM-DD]
|
||||
**Executed by**: Tester Agent
|
||||
|
||||
## Coverage Summary
|
||||
|
||||
| Module | Tests | Coverage % |
|
||||
|--------|-------|------------|
|
||||
| ... | ... | ... |
|
||||
|
||||
## Test Results
|
||||
|
||||
- Total: [X]
|
||||
- Passed: [X]
|
||||
- Failed: [X]
|
||||
- Skipped: [X]
|
||||
|
||||
## Issues Found
|
||||
|
||||
| Test | Error | Resolution |
|
||||
|------|-------|------------|
|
||||
| ... | ... | ... |
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [ ] Fix failed tests
|
||||
- [ ] Add more coverage for [module]
|
||||
- [ ] Review TEST_DATA fixtures
|
||||
```
|
||||
|
||||
## Context for Testing
|
||||
|
||||
$ARGUMENTS
|
||||
@@ -30,12 +30,12 @@
|
||||
#
|
||||
# 5. Multi-Agent Support
|
||||
# - Handles agent-specific file paths and naming conventions
|
||||
# - Supports: Claude, Gemini, Copilot, Cursor, Qwen, opencode, Codex, Windsurf, Kilo Code, Auggie CLI, Roo Code, CodeBuddy CLI, Qoder CLI, Amp, SHAI, Amazon Q Developer CLI, or Antigravity
|
||||
# - Supports: Claude, Gemini, Copilot, Cursor, Qwen, opencode, Codex, Windsurf, Kilo Code, Auggie CLI, Roo Code, CodeBuddy CLI, Qoder CLI, Amp, SHAI, or Amazon Q Developer CLI
|
||||
# - Can update single agents or all existing agent files
|
||||
# - Creates default Claude file if no agent files exist
|
||||
#
|
||||
# Usage: ./update-agent-context.sh [agent_type]
|
||||
# Agent types: claude|gemini|copilot|cursor-agent|qwen|opencode|codex|windsurf|kilocode|auggie|roo|codebuddy|amp|shai|q|agy|bob|qodercli
|
||||
# Agent types: claude|gemini|copilot|cursor-agent|qwen|opencode|codex|windsurf|kilocode|auggie|shai|q|bob|qoder
|
||||
# Leave empty to update all existing agent files
|
||||
|
||||
set -e
|
||||
@@ -74,7 +74,6 @@ QODER_FILE="$REPO_ROOT/QODER.md"
|
||||
AMP_FILE="$REPO_ROOT/AGENTS.md"
|
||||
SHAI_FILE="$REPO_ROOT/SHAI.md"
|
||||
Q_FILE="$REPO_ROOT/AGENTS.md"
|
||||
AGY_FILE="$REPO_ROOT/.agent/rules/specify-rules.md"
|
||||
BOB_FILE="$REPO_ROOT/AGENTS.md"
|
||||
|
||||
# Template file
|
||||
@@ -619,7 +618,7 @@ update_specific_agent() {
|
||||
codebuddy)
|
||||
update_agent_file "$CODEBUDDY_FILE" "CodeBuddy CLI"
|
||||
;;
|
||||
qodercli)
|
||||
qoder)
|
||||
update_agent_file "$QODER_FILE" "Qoder CLI"
|
||||
;;
|
||||
amp)
|
||||
@@ -631,18 +630,12 @@ update_specific_agent() {
|
||||
q)
|
||||
update_agent_file "$Q_FILE" "Amazon Q Developer CLI"
|
||||
;;
|
||||
agy)
|
||||
update_agent_file "$AGY_FILE" "Antigravity"
|
||||
;;
|
||||
bob)
|
||||
update_agent_file "$BOB_FILE" "IBM Bob"
|
||||
;;
|
||||
generic)
|
||||
log_info "Generic agent: no predefined context file. Use the agent-specific update script for your agent."
|
||||
;;
|
||||
*)
|
||||
log_error "Unknown agent type '$agent_type'"
|
||||
log_error "Expected: claude|gemini|copilot|cursor-agent|qwen|opencode|codex|windsurf|kilocode|auggie|roo|codebuddy|amp|shai|q|agy|bob|qodercli|generic"
|
||||
log_error "Expected: claude|gemini|copilot|cursor-agent|qwen|opencode|codex|windsurf|kilocode|auggie|roo|amp|shai|q|bob|qoder"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
@@ -721,11 +714,7 @@ update_all_existing_agents() {
|
||||
update_agent_file "$Q_FILE" "Amazon Q Developer CLI"
|
||||
found_agent=true
|
||||
fi
|
||||
|
||||
if [[ -f "$AGY_FILE" ]]; then
|
||||
update_agent_file "$AGY_FILE" "Antigravity"
|
||||
found_agent=true
|
||||
fi
|
||||
|
||||
if [[ -f "$BOB_FILE" ]]; then
|
||||
update_agent_file "$BOB_FILE" "IBM Bob"
|
||||
found_agent=true
|
||||
@@ -755,7 +744,7 @@ print_summary() {
|
||||
|
||||
echo
|
||||
|
||||
log_info "Usage: $0 [claude|gemini|copilot|cursor-agent|qwen|opencode|codex|windsurf|kilocode|auggie|roo|codebuddy|amp|shai|q|agy|bob|qodercli]"
|
||||
log_info "Usage: $0 [claude|gemini|copilot|cursor-agent|qwen|opencode|codex|windsurf|kilocode|auggie|codebuddy|shai|q|bob|qoder]"
|
||||
}
|
||||
|
||||
#==============================================================================
|
||||
|
||||
@@ -1,50 +0,0 @@
|
||||
# [PROJECT_NAME] Constitution
|
||||
<!-- Example: Spec Constitution, TaskFlow Constitution, etc. -->
|
||||
|
||||
## Core Principles
|
||||
|
||||
### [PRINCIPLE_1_NAME]
|
||||
<!-- Example: I. Library-First -->
|
||||
[PRINCIPLE_1_DESCRIPTION]
|
||||
<!-- Example: Every feature starts as a standalone library; Libraries must be self-contained, independently testable, documented; Clear purpose required - no organizational-only libraries -->
|
||||
|
||||
### [PRINCIPLE_2_NAME]
|
||||
<!-- Example: II. CLI Interface -->
|
||||
[PRINCIPLE_2_DESCRIPTION]
|
||||
<!-- Example: Every library exposes functionality via CLI; Text in/out protocol: stdin/args → stdout, errors → stderr; Support JSON + human-readable formats -->
|
||||
|
||||
### [PRINCIPLE_3_NAME]
|
||||
<!-- Example: III. Test-First (NON-NEGOTIABLE) -->
|
||||
[PRINCIPLE_3_DESCRIPTION]
|
||||
<!-- Example: TDD mandatory: Tests written → User approved → Tests fail → Then implement; Red-Green-Refactor cycle strictly enforced -->
|
||||
|
||||
### [PRINCIPLE_4_NAME]
|
||||
<!-- Example: IV. Integration Testing -->
|
||||
[PRINCIPLE_4_DESCRIPTION]
|
||||
<!-- Example: Focus areas requiring integration tests: New library contract tests, Contract changes, Inter-service communication, Shared schemas -->
|
||||
|
||||
### [PRINCIPLE_5_NAME]
|
||||
<!-- Example: V. Observability, VI. Versioning & Breaking Changes, VII. Simplicity -->
|
||||
[PRINCIPLE_5_DESCRIPTION]
|
||||
<!-- Example: Text I/O ensures debuggability; Structured logging required; Or: MAJOR.MINOR.BUILD format; Or: Start simple, YAGNI principles -->
|
||||
|
||||
## [SECTION_2_NAME]
|
||||
<!-- Example: Additional Constraints, Security Requirements, Performance Standards, etc. -->
|
||||
|
||||
[SECTION_2_CONTENT]
|
||||
<!-- Example: Technology stack requirements, compliance standards, deployment policies, etc. -->
|
||||
|
||||
## [SECTION_3_NAME]
|
||||
<!-- Example: Development Workflow, Review Process, Quality Gates, etc. -->
|
||||
|
||||
[SECTION_3_CONTENT]
|
||||
<!-- Example: Code review requirements, testing gates, deployment approval process, etc. -->
|
||||
|
||||
## Governance
|
||||
<!-- Example: Constitution supersedes all other practices; Amendments require documentation, approval, migration plan -->
|
||||
|
||||
[GOVERNANCE_RULES]
|
||||
<!-- Example: All PRs/reviews must verify compliance; Complexity must be justified; Use [GUIDANCE_FILE] for runtime development guidance -->
|
||||
|
||||
**Version**: [CONSTITUTION_VERSION] | **Ratified**: [RATIFICATION_DATE] | **Last Amended**: [LAST_AMENDED_DATE]
|
||||
<!-- Example: Version: 2.1.1 | Ratified: 2025-06-13 | Last Amended: 2025-07-16 -->
|
||||
@@ -3,7 +3,7 @@
|
||||
**Branch**: `[###-feature-name]` | **Date**: [DATE] | **Spec**: [link]
|
||||
**Input**: Feature specification from `/specs/[###-feature-name]/spec.md`
|
||||
|
||||
**Note**: This template is filled in by the `/speckit.plan` command. See `.specify/templates/plan-template.md` for the execution workflow.
|
||||
**Note**: This template is filled in by the `/speckit.plan` command. See `.specify/templates/commands/plan.md` for the execution workflow.
|
||||
|
||||
## Summary
|
||||
|
||||
@@ -22,7 +22,7 @@
|
||||
**Storage**: [if applicable, e.g., PostgreSQL, CoreData, files or N/A]
|
||||
**Testing**: [e.g., pytest, XCTest, cargo test or NEEDS CLARIFICATION]
|
||||
**Target Platform**: [e.g., Linux server, iOS 15+, WASM or NEEDS CLARIFICATION]
|
||||
**Project Type**: [e.g., library/cli/web-service/mobile-app/compiler/desktop-app or NEEDS CLARIFICATION]
|
||||
**Project Type**: [single/web/mobile - determines source structure]
|
||||
**Performance Goals**: [domain-specific, e.g., 1000 req/s, 10k lines/sec, 60 fps or NEEDS CLARIFICATION]
|
||||
**Constraints**: [domain-specific, e.g., <200ms p95, <100MB memory, offline-capable or NEEDS CLARIFICATION]
|
||||
**Scale/Scope**: [domain-specific, e.g., 10k users, 1M LOC, 50 screens or NEEDS CLARIFICATION]
|
||||
|
||||
Binary file not shown.
@@ -392,11 +392,11 @@ def test_status_query_without_task_id_returns_latest_user_task():
|
||||
|
||||
|
||||
# [/DEF:test_status_query_without_task_id_returns_latest_user_task:Function]
|
||||
# [DEF:test_llm_validation_with_dashboard_ref_requires_confirmation:Function]
|
||||
# @PURPOSE: LLM validation with dashboard_ref should now require confirmation before dispatch.
|
||||
# @PRE: User sends natural-language validation request with dashboard name (not numeric id).
|
||||
# @POST: Response state is needs_confirmation since all state-changing operations are now gated.
|
||||
def test_llm_validation_with_dashboard_ref_requires_confirmation():
|
||||
# [DEF:test_llm_validation_missing_dashboard_returns_needs_clarification:Function]
|
||||
# @PURPOSE: LLM validation command without resolvable dashboard id must request clarification instead of generic failure.
|
||||
# @PRE: Command intent resolves to run_llm_validation but dashboard id cannot be inferred.
|
||||
# @POST: Assistant response state is needs_clarification with guidance text.
|
||||
def test_llm_validation_missing_dashboard_returns_needs_clarification():
|
||||
_clear_assistant_state()
|
||||
response = _run_async(
|
||||
assistant_module.send_message(
|
||||
@@ -410,11 +410,8 @@ def test_llm_validation_with_dashboard_ref_requires_confirmation():
|
||||
)
|
||||
)
|
||||
|
||||
assert response.state == "needs_confirmation"
|
||||
assert response.confirmation_id is not None
|
||||
action_types = {a.type for a in response.actions}
|
||||
assert "confirm" in action_types
|
||||
assert "cancel" in action_types
|
||||
assert response.state == "needs_clarification"
|
||||
assert "Укажите" in response.text or "Missing dashboard_id" in response.text
|
||||
|
||||
|
||||
# [/DEF:test_llm_validation_missing_dashboard_returns_needs_clarification:Function]
|
||||
@@ -558,71 +555,4 @@ def test_list_conversations_archived_only_filters_active():
|
||||
|
||||
|
||||
# [/DEF:test_list_conversations_archived_only_filters_active:Function]
|
||||
|
||||
|
||||
# [DEF:test_guarded_operation_always_requires_confirmation:Function]
|
||||
# @PURPOSE: Non-dangerous (guarded) commands must still require confirmation before execution.
|
||||
# @PRE: Admin user sends a backup command that was previously auto-executed.
|
||||
# @POST: Response state is needs_confirmation with confirm and cancel actions.
|
||||
def test_guarded_operation_always_requires_confirmation():
|
||||
_clear_assistant_state()
|
||||
response = _run_async(
|
||||
assistant_module.send_message(
|
||||
request=assistant_module.AssistantMessageRequest(
|
||||
message="сделай бэкап окружения dev"
|
||||
),
|
||||
current_user=_admin_user(),
|
||||
task_manager=_FakeTaskManager(),
|
||||
config_manager=_FakeConfigManager(),
|
||||
db=_FakeDb(),
|
||||
)
|
||||
)
|
||||
assert response.state == "needs_confirmation"
|
||||
assert response.confirmation_id is not None
|
||||
action_types = {a.type for a in response.actions}
|
||||
assert "confirm" in action_types
|
||||
assert "cancel" in action_types
|
||||
assert "Выполнить" in response.text or "Подтвердите" in response.text
|
||||
|
||||
|
||||
# [/DEF:test_guarded_operation_always_requires_confirmation:Function]
|
||||
|
||||
|
||||
# [DEF:test_guarded_operation_confirm_roundtrip:Function]
|
||||
# @PURPOSE: Guarded operation must execute successfully after explicit confirmation.
|
||||
# @PRE: Admin user sends a non-dangerous migration command (dev → dev).
|
||||
# @POST: After confirmation, response transitions to started/success with task_id.
|
||||
def test_guarded_operation_confirm_roundtrip():
|
||||
_clear_assistant_state()
|
||||
task_manager = _FakeTaskManager()
|
||||
db = _FakeDb()
|
||||
|
||||
first = _run_async(
|
||||
assistant_module.send_message(
|
||||
request=assistant_module.AssistantMessageRequest(
|
||||
message="запусти миграцию с dev на dev для дашборда 5"
|
||||
),
|
||||
current_user=_admin_user(),
|
||||
task_manager=task_manager,
|
||||
config_manager=_FakeConfigManager(),
|
||||
db=db,
|
||||
)
|
||||
)
|
||||
assert first.state == "needs_confirmation"
|
||||
assert first.confirmation_id
|
||||
|
||||
second = _run_async(
|
||||
assistant_module.confirm_operation(
|
||||
confirmation_id=first.confirmation_id,
|
||||
current_user=_admin_user(),
|
||||
task_manager=task_manager,
|
||||
config_manager=_FakeConfigManager(),
|
||||
db=db,
|
||||
)
|
||||
)
|
||||
assert second.state == "started"
|
||||
assert second.task_id is not None
|
||||
|
||||
|
||||
# [/DEF:test_guarded_operation_confirm_roundtrip:Function]
|
||||
# [/DEF:backend.src.api.routes.__tests__.test_assistant_api:Module]
|
||||
|
||||
@@ -918,46 +918,6 @@ def _coerce_intent_entities(intent: Dict[str, Any]) -> Dict[str, Any]:
|
||||
# [/DEF:_coerce_intent_entities:Function]
|
||||
|
||||
|
||||
# Operations that are read-only and do not require confirmation.
|
||||
_SAFE_OPS = {"show_capabilities", "get_task_status"}
|
||||
|
||||
|
||||
# [DEF:_confirmation_summary:Function]
|
||||
# @PURPOSE: Build human-readable confirmation prompt for an intent before execution.
|
||||
# @PRE: intent contains operation and entities fields.
|
||||
# @POST: Returns descriptive Russian-language text ending with confirmation prompt.
|
||||
def _confirmation_summary(intent: Dict[str, Any]) -> str:
|
||||
operation = intent.get("operation", "")
|
||||
entities = intent.get("entities", {})
|
||||
descriptions: Dict[str, str] = {
|
||||
"create_branch": "создание ветки{branch} для дашборда{dashboard}",
|
||||
"commit_changes": "коммит изменений для дашборда{dashboard}",
|
||||
"deploy_dashboard": "деплой дашборда{dashboard} в окружение{env}",
|
||||
"execute_migration": "миграция дашборда{dashboard} с{src} на{tgt}",
|
||||
"run_backup": "бэкап окружения{env}{dashboard}",
|
||||
"run_llm_validation": "LLM-валидация дашборда{dashboard}{env}",
|
||||
"run_llm_documentation": "генерация документации для датасета{dataset}{env}",
|
||||
}
|
||||
template = descriptions.get(operation)
|
||||
if not template:
|
||||
return "Подтвердите выполнение операции или отмените."
|
||||
|
||||
def _label(value: Any, prefix: str = " ") -> str:
|
||||
return f"{prefix}{value}" if value else ""
|
||||
|
||||
dashboard = entities.get("dashboard_id") or entities.get("dashboard_ref")
|
||||
text = template.format(
|
||||
branch=_label(entities.get("branch_name")),
|
||||
dashboard=_label(dashboard),
|
||||
env=_label(entities.get("environment") or entities.get("target_env")),
|
||||
src=_label(entities.get("source_env")),
|
||||
tgt=_label(entities.get("target_env")),
|
||||
dataset=_label(entities.get("dataset_id")),
|
||||
)
|
||||
return f"Выполнить: {text}. Подтвердите или отмените."
|
||||
# [/DEF:_confirmation_summary:Function]
|
||||
|
||||
|
||||
# [DEF:_clarification_text_for_intent:Function]
|
||||
# @PURPOSE: Convert technical missing-parameter errors into user-facing clarification prompts.
|
||||
# @PRE: state was classified as needs_clarification for current intent/error combination.
|
||||
@@ -1368,7 +1328,14 @@ async def send_message(
|
||||
except Exception as exc:
|
||||
logger.warning(f"[assistant.planner][fallback] Planner error: {exc}")
|
||||
if not intent:
|
||||
intent = _parse_command(request.message, config_manager)
|
||||
intent = {
|
||||
"domain": "unknown",
|
||||
"operation": "clarify",
|
||||
"entities": {},
|
||||
"confidence": 0.0,
|
||||
"risk_level": "safe",
|
||||
"requires_confirmation": False,
|
||||
}
|
||||
confidence = float(intent.get("confidence", 0.0))
|
||||
|
||||
if intent.get("domain") == "unknown" or confidence < 0.6:
|
||||
@@ -1391,8 +1358,7 @@ async def send_message(
|
||||
try:
|
||||
_authorize_intent(intent, current_user)
|
||||
|
||||
operation = intent.get("operation")
|
||||
if operation not in _SAFE_OPS:
|
||||
if intent.get("requires_confirmation"):
|
||||
confirmation_id = str(uuid.uuid4())
|
||||
confirm = ConfirmationRecord(
|
||||
id=confirmation_id,
|
||||
@@ -1405,7 +1371,7 @@ async def send_message(
|
||||
)
|
||||
CONFIRMATIONS[confirmation_id] = confirm
|
||||
_persist_confirmation(db, confirm)
|
||||
text = _confirmation_summary(intent)
|
||||
text = "Операция рискованная. Подтвердите выполнение или отмените."
|
||||
_append_history(
|
||||
user_id,
|
||||
conversation_id,
|
||||
@@ -1422,13 +1388,7 @@ async def send_message(
|
||||
text,
|
||||
state="needs_confirmation",
|
||||
confirmation_id=confirmation_id,
|
||||
metadata={
|
||||
"intent": intent,
|
||||
"actions": [
|
||||
{"type": "confirm", "label": "✅ Подтвердить", "target": confirmation_id},
|
||||
{"type": "cancel", "label": "❌ Отменить", "target": confirmation_id},
|
||||
],
|
||||
},
|
||||
metadata={"intent": intent},
|
||||
)
|
||||
audit_payload = {
|
||||
"decision": "needs_confirmation",
|
||||
@@ -1446,13 +1406,12 @@ async def send_message(
|
||||
intent=intent,
|
||||
confirmation_id=confirmation_id,
|
||||
actions=[
|
||||
AssistantAction(type="confirm", label="✅ Подтвердить", target=confirmation_id),
|
||||
AssistantAction(type="cancel", label="❌ Отменить", target=confirmation_id),
|
||||
AssistantAction(type="confirm", label="Confirm", target=confirmation_id),
|
||||
AssistantAction(type="cancel", label="Cancel", target=confirmation_id),
|
||||
],
|
||||
created_at=datetime.utcnow(),
|
||||
)
|
||||
|
||||
# Read-only operations execute immediately
|
||||
text, task_id, actions = await _dispatch_intent(intent, current_user, task_manager, config_manager, db)
|
||||
state = "started" if task_id else "success"
|
||||
_append_history(user_id, conversation_id, "assistant", text, state=state, task_id=task_id)
|
||||
|
||||
@@ -88,20 +88,6 @@ class DashboardDetailResponse(BaseModel):
|
||||
dataset_count: int
|
||||
# [/DEF:DashboardDetailResponse:DataClass]
|
||||
|
||||
# [DEF:DatabaseMapping:DataClass]
|
||||
class DatabaseMapping(BaseModel):
|
||||
source_db: str
|
||||
target_db: str
|
||||
source_db_uuid: Optional[str] = None
|
||||
target_db_uuid: Optional[str] = None
|
||||
confidence: float
|
||||
# [/DEF:DatabaseMapping:DataClass]
|
||||
|
||||
# [DEF:DatabaseMappingsResponse:DataClass]
|
||||
class DatabaseMappingsResponse(BaseModel):
|
||||
mappings: List[DatabaseMapping]
|
||||
# [/DEF:DatabaseMappingsResponse:DataClass]
|
||||
|
||||
# [DEF:get_dashboards:Function]
|
||||
# @PURPOSE: Fetch list of dashboards from a specific environment with Git status and last task status
|
||||
# @PRE: env_id must be a valid environment ID
|
||||
@@ -182,54 +168,12 @@ async def get_dashboards(
|
||||
raise HTTPException(status_code=503, detail=f"Failed to fetch dashboards: {str(e)}")
|
||||
# [/DEF:get_dashboards:Function]
|
||||
|
||||
# [DEF:get_database_mappings:Function]
|
||||
# @PURPOSE: Get database mapping suggestions between source and target environments
|
||||
# @PRE: User has permission plugin:migration:read
|
||||
# @PRE: source_env_id and target_env_id are valid environment IDs
|
||||
# @POST: Returns list of suggested database mappings with confidence scores
|
||||
# @PARAM: source_env_id (str) - Source environment ID
|
||||
# @PARAM: target_env_id (str) - Target environment ID
|
||||
# @RETURN: DatabaseMappingsResponse - List of suggested mappings
|
||||
# @RELATION: CALLS -> MappingService.get_suggestions
|
||||
@router.get("/db-mappings", response_model=DatabaseMappingsResponse)
|
||||
async def get_database_mappings(
|
||||
source_env_id: str,
|
||||
target_env_id: str,
|
||||
mapping_service=Depends(get_mapping_service),
|
||||
_ = Depends(has_permission("plugin:migration", "READ"))
|
||||
):
|
||||
with belief_scope("get_database_mappings", f"source={source_env_id}, target={target_env_id}"):
|
||||
try:
|
||||
# Get mapping suggestions using MappingService
|
||||
suggestions = await mapping_service.get_suggestions(source_env_id, target_env_id)
|
||||
|
||||
# Format suggestions as DatabaseMapping objects
|
||||
mappings = [
|
||||
DatabaseMapping(
|
||||
source_db=s.get('source_db', ''),
|
||||
target_db=s.get('target_db', ''),
|
||||
source_db_uuid=s.get('source_db_uuid'),
|
||||
target_db_uuid=s.get('target_db_uuid'),
|
||||
confidence=s.get('confidence', 0.0)
|
||||
)
|
||||
for s in suggestions
|
||||
]
|
||||
|
||||
logger.info(f"[get_database_mappings][Coherence:OK] Returning {len(mappings)} database mapping suggestions")
|
||||
|
||||
return DatabaseMappingsResponse(mappings=mappings)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[get_database_mappings][Coherence:Failed] Failed to get database mappings: {e}")
|
||||
raise HTTPException(status_code=503, detail=f"Failed to get database mappings: {str(e)}")
|
||||
# [/DEF:get_database_mappings:Function]
|
||||
|
||||
# [DEF:get_dashboard_detail:Function]
|
||||
# @PURPOSE: Fetch detailed dashboard info with related charts and datasets
|
||||
# @PRE: env_id must be valid and dashboard_id must exist
|
||||
# @POST: Returns dashboard detail payload for overview page
|
||||
# @RELATION: CALLS -> SupersetClient.get_dashboard_detail
|
||||
@router.get("/{dashboard_id:int}", response_model=DashboardDetailResponse)
|
||||
@router.get("/{dashboard_id}", response_model=DashboardDetailResponse)
|
||||
async def get_dashboard_detail(
|
||||
dashboard_id: int,
|
||||
env_id: str,
|
||||
@@ -393,4 +337,60 @@ async def backup_dashboards(
|
||||
raise HTTPException(status_code=503, detail=f"Failed to create backup task: {str(e)}")
|
||||
# [/DEF:backup_dashboards:Function]
|
||||
|
||||
# [DEF:DatabaseMapping:DataClass]
|
||||
class DatabaseMapping(BaseModel):
|
||||
source_db: str
|
||||
target_db: str
|
||||
source_db_uuid: Optional[str] = None
|
||||
target_db_uuid: Optional[str] = None
|
||||
confidence: float
|
||||
# [/DEF:DatabaseMapping:DataClass]
|
||||
|
||||
# [DEF:DatabaseMappingsResponse:DataClass]
|
||||
class DatabaseMappingsResponse(BaseModel):
|
||||
mappings: List[DatabaseMapping]
|
||||
# [/DEF:DatabaseMappingsResponse:DataClass]
|
||||
|
||||
# [DEF:get_database_mappings:Function]
|
||||
# @PURPOSE: Get database mapping suggestions between source and target environments
|
||||
# @PRE: User has permission plugin:migration:read
|
||||
# @PRE: source_env_id and target_env_id are valid environment IDs
|
||||
# @POST: Returns list of suggested database mappings with confidence scores
|
||||
# @PARAM: source_env_id (str) - Source environment ID
|
||||
# @PARAM: target_env_id (str) - Target environment ID
|
||||
# @RETURN: DatabaseMappingsResponse - List of suggested mappings
|
||||
# @RELATION: CALLS -> MappingService.get_suggestions
|
||||
@router.get("/db-mappings", response_model=DatabaseMappingsResponse)
|
||||
async def get_database_mappings(
|
||||
source_env_id: str,
|
||||
target_env_id: str,
|
||||
mapping_service=Depends(get_mapping_service),
|
||||
_ = Depends(has_permission("plugin:migration", "READ"))
|
||||
):
|
||||
with belief_scope("get_database_mappings", f"source={source_env_id}, target={target_env_id}"):
|
||||
try:
|
||||
# Get mapping suggestions using MappingService
|
||||
suggestions = await mapping_service.get_suggestions(source_env_id, target_env_id)
|
||||
|
||||
# Format suggestions as DatabaseMapping objects
|
||||
mappings = [
|
||||
DatabaseMapping(
|
||||
source_db=s.get('source_db', ''),
|
||||
target_db=s.get('target_db', ''),
|
||||
source_db_uuid=s.get('source_db_uuid'),
|
||||
target_db_uuid=s.get('target_db_uuid'),
|
||||
confidence=s.get('confidence', 0.0)
|
||||
)
|
||||
for s in suggestions
|
||||
]
|
||||
|
||||
logger.info(f"[get_database_mappings][Coherence:OK] Returning {len(mappings)} database mapping suggestions")
|
||||
|
||||
return DatabaseMappingsResponse(mappings=mappings)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[get_database_mappings][Coherence:Failed] Failed to get database mappings: {e}")
|
||||
raise HTTPException(status_code=503, detail=f"Failed to get database mappings: {str(e)}")
|
||||
# [/DEF:get_database_mappings:Function]
|
||||
|
||||
# [/DEF:backend.src.api.routes.dashboards:Module]
|
||||
|
||||
@@ -6,16 +6,12 @@
|
||||
# @RELATION: DEPENDS_ON -> backend.src.dependencies
|
||||
# @RELATION: DEPENDS_ON -> backend.src.models.dashboard
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query
|
||||
from typing import List, Dict, Any
|
||||
from sqlalchemy.orm import Session
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from typing import List
|
||||
from ...dependencies import get_config_manager, get_task_manager, has_permission
|
||||
from ...core.database import get_db
|
||||
from ...models.dashboard import DashboardMetadata, DashboardSelection
|
||||
from ...core.superset_client import SupersetClient
|
||||
from ...core.logger import belief_scope
|
||||
from ...core.mapping_service import IdMappingService
|
||||
from ...models.mapping import ResourceMapping
|
||||
|
||||
router = APIRouter(prefix="/api", tags=["migration"])
|
||||
|
||||
@@ -65,10 +61,9 @@ async def execute_migration(
|
||||
# Create migration task with debug logging
|
||||
from ...core.logger import logger
|
||||
|
||||
# Include replace_db_config and fix_cross_filters in the task parameters
|
||||
# Include replace_db_config in the task parameters
|
||||
task_params = selection.dict()
|
||||
task_params['replace_db_config'] = selection.replace_db_config
|
||||
task_params['fix_cross_filters'] = selection.fix_cross_filters
|
||||
|
||||
logger.info(f"Creating migration task with params: {task_params}")
|
||||
logger.info(f"Available environments: {env_ids}")
|
||||
@@ -83,68 +78,4 @@ async def execute_migration(
|
||||
raise HTTPException(status_code=500, detail=f"Failed to create migration task: {str(e)}")
|
||||
# [/DEF:execute_migration:Function]
|
||||
|
||||
# [DEF:get_migration_settings:Function]
|
||||
# @PURPOSE: Get current migration Cron string explicitly.
|
||||
@router.get("/settings", response_model=Dict[str, str])
|
||||
async def get_migration_settings(
|
||||
config_manager=Depends(get_config_manager),
|
||||
_ = Depends(has_permission("plugin:migration", "READ"))
|
||||
):
|
||||
with belief_scope("get_migration_settings"):
|
||||
# For simplicity in MVP, assuming cron expression is stored in config
|
||||
# default to a valid cron if not set.
|
||||
config = config_manager.get_config()
|
||||
cron = config.get("migration_sync_cron", "0 2 * * *")
|
||||
return {"cron": cron}
|
||||
# [/DEF:get_migration_settings:Function]
|
||||
|
||||
# [DEF:update_migration_settings:Function]
|
||||
# @PURPOSE: Update migration Cron string.
|
||||
@router.put("/settings", response_model=Dict[str, str])
|
||||
async def update_migration_settings(
|
||||
payload: Dict[str, str],
|
||||
config_manager=Depends(get_config_manager),
|
||||
_ = Depends(has_permission("plugin:migration", "WRITE"))
|
||||
):
|
||||
with belief_scope("update_migration_settings"):
|
||||
if "cron" not in payload:
|
||||
raise HTTPException(status_code=400, detail="Missing 'cron' field in payload")
|
||||
|
||||
cron_expr = payload["cron"]
|
||||
# Basic validation could go here
|
||||
|
||||
# In a real system, you'd save this to config and restart the scheduler.
|
||||
# Here we just blindly patch the in-memory or file config for the MVP.
|
||||
current_cfg = config_manager.get_config()
|
||||
current_cfg["migration_sync_cron"] = cron_expr
|
||||
config_manager.save_config(current_cfg)
|
||||
|
||||
return {"cron": cron_expr, "status": "updated"}
|
||||
# [/DEF:update_migration_settings:Function]
|
||||
|
||||
# [DEF:get_resource_mappings:Function]
|
||||
# @PURPOSE: Fetch all synchronized object mappings from the database.
|
||||
@router.get("/mappings-data", response_model=List[Dict[str, Any]])
|
||||
async def get_resource_mappings(
|
||||
skip: int = Query(0, ge=0),
|
||||
limit: int = Query(100, ge=1, le=1000),
|
||||
db: Session = Depends(get_db),
|
||||
_ = Depends(has_permission("plugin:migration", "READ"))
|
||||
):
|
||||
with belief_scope("get_resource_mappings"):
|
||||
mappings = db.query(ResourceMapping).offset(skip).limit(limit).all()
|
||||
result = []
|
||||
for m in mappings:
|
||||
result.append({
|
||||
"id": m.id,
|
||||
"environment_id": m.environment_id,
|
||||
"resource_type": m.resource_type.value,
|
||||
"uuid": m.uuid,
|
||||
"remote_id": m.remote_integer_id,
|
||||
"resource_name": m.resource_name,
|
||||
"last_synced_at": m.last_synced_at.isoformat() if m.last_synced_at else None
|
||||
})
|
||||
return result
|
||||
# [/DEF:get_resource_mappings:Function]
|
||||
|
||||
# [/DEF:backend.src.api.routes.migration:Module]
|
||||
@@ -1,195 +0,0 @@
|
||||
# [DEF:backend.src.core.mapping_service:Module]
|
||||
#
|
||||
# @TIER: CRITICAL
|
||||
# @SEMANTICS: mapping, ids, synchronization, environments, cross-filters
|
||||
# @PURPOSE: Service for tracking and synchronizing Superset Resource IDs (UUID <-> Integer ID)
|
||||
# @LAYER: Core
|
||||
# @RELATION: DEPENDS_ON -> backend.src.models.mapping (ResourceMapping, ResourceType)
|
||||
# @RELATION: DEPENDS_ON -> backend.src.core.logger
|
||||
# @TEST_DATA: mock_superset_resources -> {'chart': [{'id': 42, 'uuid': '1234', 'slice_name': 'test'}], 'dataset': [{'id': 99, 'uuid': '5678', 'table_name': 'data'}]}
|
||||
#
|
||||
# @INVARIANT: sync_environment must handle remote API failures gracefully.
|
||||
|
||||
# [SECTION: IMPORTS]
|
||||
from typing import Dict, List, Optional
|
||||
from datetime import datetime, timezone
|
||||
from sqlalchemy.orm import Session
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
from apscheduler.triggers.cron import CronTrigger
|
||||
from src.models.mapping import ResourceMapping, ResourceType
|
||||
from src.core.logger import logger, belief_scope
|
||||
# [/SECTION]
|
||||
|
||||
# [DEF:IdMappingService:Class]
|
||||
# @TIER: CRITICAL
|
||||
# @PURPOSE: Service handling the cataloging and retrieval of remote Superset Integer IDs.
|
||||
class IdMappingService:
|
||||
|
||||
# [DEF:__init__:Function]
|
||||
# @PURPOSE: Initializes the mapping service.
|
||||
def __init__(self, db_session: Session):
|
||||
self.db = db_session
|
||||
self.scheduler = BackgroundScheduler()
|
||||
self._sync_job = None
|
||||
# [/DEF:__init__:Function]
|
||||
|
||||
# [DEF:start_scheduler:Function]
|
||||
# @PURPOSE: Starts the background scheduler with a given cron string.
|
||||
# @PARAM: cron_string (str) - Cron expression for the sync interval.
|
||||
# @PARAM: environments (List[str]) - List of environment IDs to sync.
|
||||
# @PARAM: superset_client_factory - Function to get a client for an environment.
|
||||
def start_scheduler(self, cron_string: str, environments: List[str], superset_client_factory):
|
||||
with belief_scope("IdMappingService.start_scheduler"):
|
||||
if self._sync_job:
|
||||
self.scheduler.remove_job(self._sync_job.id)
|
||||
logger.info("[IdMappingService.start_scheduler][Reflect] Removed existing sync job.")
|
||||
|
||||
def sync_all():
|
||||
for env_id in environments:
|
||||
client = superset_client_factory(env_id)
|
||||
if client:
|
||||
self.sync_environment(env_id, client)
|
||||
|
||||
self._sync_job = self.scheduler.add_job(
|
||||
sync_all,
|
||||
CronTrigger.from_crontab(cron_string),
|
||||
id='id_mapping_sync_job',
|
||||
replace_existing=True
|
||||
)
|
||||
|
||||
if not self.scheduler.running:
|
||||
self.scheduler.start()
|
||||
logger.info(f"[IdMappingService.start_scheduler][Coherence:OK] Started background scheduler with cron: {cron_string}")
|
||||
else:
|
||||
logger.info(f"[IdMappingService.start_scheduler][Coherence:OK] Updated background scheduler with cron: {cron_string}")
|
||||
# [/DEF:start_scheduler:Function]
|
||||
|
||||
# [DEF:sync_environment:Function]
|
||||
# @PURPOSE: Fully synchronizes mapping for a specific environment.
|
||||
# @PARAM: environment_id (str) - Target environment ID.
|
||||
# @PARAM: superset_client - Instance capable of hitting the Superset API.
|
||||
# @PRE: environment_id exists in the database.
|
||||
# @POST: ResourceMapping records for the environment are created or updated.
|
||||
def sync_environment(self, environment_id: str, superset_client) -> None:
|
||||
"""
|
||||
Polls the Superset APIs for the target environment and updates the local mapping table.
|
||||
"""
|
||||
with belief_scope("IdMappingService.sync_environment"):
|
||||
logger.info(f"[IdMappingService.sync_environment][Action] Starting sync for environment {environment_id}")
|
||||
|
||||
# Implementation Note: In a real scenario, superset_client needs to be an instance
|
||||
# capable of auth & iteration over /api/v1/chart/, /api/v1/dataset/, /api/v1/dashboard/
|
||||
# Here we structure the logic according to the spec.
|
||||
|
||||
types_to_poll = [
|
||||
(ResourceType.CHART, "chart", "slice_name"),
|
||||
(ResourceType.DATASET, "dataset", "table_name"),
|
||||
(ResourceType.DASHBOARD, "dashboard", "slug") # Note: dashboard slug or dashboard_title
|
||||
]
|
||||
|
||||
total_synced = 0
|
||||
try:
|
||||
for res_enum, endpoint, name_field in types_to_poll:
|
||||
logger.debug(f"[IdMappingService.sync_environment][Explore] Polling {endpoint} endpoint")
|
||||
|
||||
# Simulated API Fetch (Would be: superset_client.get(f"/api/v1/{endpoint}/")... )
|
||||
# This relies on the superset API structure, e.g. { "result": [{"id": 1, "uuid": "...", name_field: "..."}] }
|
||||
# We assume superset_client provides a generic method to fetch all pages.
|
||||
|
||||
try:
|
||||
resources = superset_client.get_all_resources(endpoint)
|
||||
|
||||
for res in resources:
|
||||
res_uuid = res.get("uuid")
|
||||
res_id = str(res.get("id")) # Store as string
|
||||
res_name = res.get(name_field)
|
||||
|
||||
if not res_uuid or not res_id:
|
||||
continue
|
||||
|
||||
# Upsert Logic
|
||||
mapping = self.db.query(ResourceMapping).filter_by(
|
||||
environment_id=environment_id,
|
||||
resource_type=res_enum,
|
||||
uuid=res_uuid
|
||||
).first()
|
||||
|
||||
if mapping:
|
||||
mapping.remote_integer_id = res_id
|
||||
mapping.resource_name = res_name
|
||||
mapping.last_synced_at = datetime.now(timezone.utc)
|
||||
else:
|
||||
new_mapping = ResourceMapping(
|
||||
environment_id=environment_id,
|
||||
resource_type=res_enum,
|
||||
uuid=res_uuid,
|
||||
remote_integer_id=res_id,
|
||||
resource_name=res_name,
|
||||
last_synced_at=datetime.now(timezone.utc)
|
||||
)
|
||||
self.db.add(new_mapping)
|
||||
|
||||
total_synced += 1
|
||||
|
||||
except Exception as loop_e:
|
||||
logger.error(f"[IdMappingService.sync_environment][Reason] Error polling {endpoint}: {loop_e}")
|
||||
# Continue to next resource type instead of blowing up the whole sync
|
||||
|
||||
self.db.commit()
|
||||
logger.info(f"[IdMappingService.sync_environment][Coherence:OK] Successfully synced {total_synced} items.")
|
||||
|
||||
except Exception as e:
|
||||
self.db.rollback()
|
||||
logger.error(f"[IdMappingService.sync_environment][Coherence:Failed] Critical sync failure: {e}")
|
||||
raise
|
||||
# [/DEF:sync_environment:Function]
|
||||
|
||||
# [DEF:get_remote_id:Function]
|
||||
# @PURPOSE: Retrieves the remote integer ID for a given universal UUID.
|
||||
# @PARAM: environment_id (str)
|
||||
# @PARAM: resource_type (ResourceType)
|
||||
# @PARAM: uuid (str)
|
||||
# @RETURN: Optional[int]
|
||||
def get_remote_id(self, environment_id: str, resource_type: ResourceType, uuid: str) -> Optional[int]:
|
||||
mapping = self.db.query(ResourceMapping).filter_by(
|
||||
environment_id=environment_id,
|
||||
resource_type=resource_type,
|
||||
uuid=uuid
|
||||
).first()
|
||||
|
||||
if mapping:
|
||||
try:
|
||||
return int(mapping.remote_integer_id)
|
||||
except ValueError:
|
||||
return None
|
||||
return None
|
||||
# [/DEF:get_remote_id:Function]
|
||||
|
||||
# [DEF:get_remote_ids_batch:Function]
|
||||
# @PURPOSE: Retrieves remote integer IDs for a list of universal UUIDs efficiently.
|
||||
# @PARAM: environment_id (str)
|
||||
# @PARAM: resource_type (ResourceType)
|
||||
# @PARAM: uuids (List[str])
|
||||
# @RETURN: Dict[str, int] - Mapping of UUID -> Integer ID
|
||||
def get_remote_ids_batch(self, environment_id: str, resource_type: ResourceType, uuids: List[str]) -> Dict[str, int]:
|
||||
if not uuids:
|
||||
return {}
|
||||
|
||||
mappings = self.db.query(ResourceMapping).filter(
|
||||
ResourceMapping.environment_id == environment_id,
|
||||
ResourceMapping.resource_type == resource_type,
|
||||
ResourceMapping.uuid.in_(uuids)
|
||||
).all()
|
||||
|
||||
result = {}
|
||||
for m in mappings:
|
||||
try:
|
||||
result[m.uuid] = int(m.remote_integer_id)
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
return result
|
||||
# [/DEF:get_remote_ids_batch:Function]
|
||||
|
||||
# [/DEF:IdMappingService:Class]
|
||||
# [/DEF:backend.src.core.mapping_service:Module]
|
||||
@@ -11,41 +11,28 @@
|
||||
import zipfile
|
||||
import yaml
|
||||
import os
|
||||
import json
|
||||
import re
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Dict, Optional, List
|
||||
from typing import Dict
|
||||
from .logger import logger, belief_scope
|
||||
from src.core.mapping_service import IdMappingService
|
||||
from src.models.mapping import ResourceType
|
||||
# [/SECTION]
|
||||
|
||||
# [DEF:MigrationEngine:Class]
|
||||
# @PURPOSE: Engine for transforming Superset export ZIPs.
|
||||
class MigrationEngine:
|
||||
|
||||
# [DEF:__init__:Function]
|
||||
# @PURPOSE: Initializes the migration engine with optional ID mapping service.
|
||||
# @PARAM: mapping_service (Optional[IdMappingService]) - Used for resolving target environment integer IDs.
|
||||
def __init__(self, mapping_service: Optional[IdMappingService] = None):
|
||||
self.mapping_service = mapping_service
|
||||
# [/DEF:__init__:Function]
|
||||
|
||||
# [DEF:transform_zip:Function]
|
||||
# @PURPOSE: Extracts ZIP, replaces database UUIDs in YAMLs, patches cross-filters, and re-packages.
|
||||
# @PURPOSE: Extracts ZIP, replaces database UUIDs in YAMLs, and re-packages.
|
||||
# @PARAM: zip_path (str) - Path to the source ZIP file.
|
||||
# @PARAM: output_path (str) - Path where the transformed ZIP will be saved.
|
||||
# @PARAM: db_mapping (Dict[str, str]) - Mapping of source UUID to target UUID.
|
||||
# @PARAM: strip_databases (bool) - Whether to remove the databases directory from the archive.
|
||||
# @PARAM: target_env_id (Optional[str]) - Used if fix_cross_filters is True to know which environment map to use.
|
||||
# @PARAM: fix_cross_filters (bool) - Whether to patch dashboard json_metadata.
|
||||
# @PRE: zip_path must point to a valid Superset export archive.
|
||||
# @POST: Transformed archive is saved to output_path.
|
||||
# @RETURN: bool - True if successful.
|
||||
def transform_zip(self, zip_path: str, output_path: str, db_mapping: Dict[str, str], strip_databases: bool = True, target_env_id: Optional[str] = None, fix_cross_filters: bool = False) -> bool:
|
||||
def transform_zip(self, zip_path: str, output_path: str, db_mapping: Dict[str, str], strip_databases: bool = True) -> bool:
|
||||
"""
|
||||
Transform a Superset export ZIP by replacing database UUIDs and optionally fixing cross-filters.
|
||||
Transform a Superset export ZIP by replacing database UUIDs.
|
||||
"""
|
||||
with belief_scope("MigrationEngine.transform_zip"):
|
||||
with tempfile.TemporaryDirectory() as temp_dir_str:
|
||||
@@ -57,7 +44,8 @@ class MigrationEngine:
|
||||
with zipfile.ZipFile(zip_path, 'r') as zf:
|
||||
zf.extractall(temp_dir)
|
||||
|
||||
# 2. Transform YAMLs (Databases)
|
||||
# 2. Transform YAMLs
|
||||
# Datasets are usually in datasets/*.yaml
|
||||
dataset_files = list(temp_dir.glob("**/datasets/**/*.yaml")) + list(temp_dir.glob("**/datasets/*.yaml"))
|
||||
dataset_files = list(set(dataset_files))
|
||||
|
||||
@@ -66,20 +54,6 @@ class MigrationEngine:
|
||||
logger.info(f"[MigrationEngine.transform_zip][Action] Transforming dataset: {ds_file}")
|
||||
self._transform_yaml(ds_file, db_mapping)
|
||||
|
||||
# 2.5 Patch Cross-Filters (Dashboards)
|
||||
if fix_cross_filters and self.mapping_service and target_env_id:
|
||||
dash_files = list(temp_dir.glob("**/dashboards/**/*.yaml")) + list(temp_dir.glob("**/dashboards/*.yaml"))
|
||||
dash_files = list(set(dash_files))
|
||||
|
||||
logger.info(f"[MigrationEngine.transform_zip][State] Found {len(dash_files)} dashboard files for patching.")
|
||||
|
||||
# Gather all source UUID-to-ID mappings from the archive first
|
||||
source_id_to_uuid_map = self._extract_chart_uuids_from_archive(temp_dir)
|
||||
|
||||
for dash_file in dash_files:
|
||||
logger.info(f"[MigrationEngine.transform_zip][Action] Patching dashboard: {dash_file}")
|
||||
self._patch_dashboard_metadata(dash_file, target_env_id, source_id_to_uuid_map)
|
||||
|
||||
# 3. Re-package
|
||||
logger.info(f"[MigrationEngine.transform_zip][Action] Re-packaging ZIP to: {output_path} (strip_databases={strip_databases})")
|
||||
with zipfile.ZipFile(output_path, 'w', zipfile.ZIP_DEFLATED) as zf:
|
||||
@@ -123,100 +97,6 @@ class MigrationEngine:
|
||||
yaml.dump(data, f)
|
||||
# [/DEF:_transform_yaml:Function]
|
||||
|
||||
# [DEF:_extract_chart_uuids_from_archive:Function]
|
||||
# @PURPOSE: Scans the unpacked ZIP to map local exported integer IDs back to their UUIDs.
|
||||
# @PARAM: temp_dir (Path) - Root dir of unpacked archive
|
||||
# @RETURN: Dict[int, str] - Mapping of source Integer ID to UUID.
|
||||
def _extract_chart_uuids_from_archive(self, temp_dir: Path) -> Dict[int, str]:
|
||||
# Implementation Note: This is a placeholder for the logic that extracts
|
||||
# actual Source IDs. In a real scenario, this involves parsing chart YAMLs
|
||||
# or manifesting the export metadata structure where source IDs are stored.
|
||||
# For simplicity in US1 MVP, we assume it's read from chart files if present.
|
||||
mapping = {}
|
||||
chart_files = list(temp_dir.glob("**/charts/**/*.yaml")) + list(temp_dir.glob("**/charts/*.yaml"))
|
||||
for cf in set(chart_files):
|
||||
try:
|
||||
with open(cf, 'r') as f:
|
||||
cdata = yaml.safe_load(f)
|
||||
if cdata and 'id' in cdata and 'uuid' in cdata:
|
||||
mapping[cdata['id']] = cdata['uuid']
|
||||
except Exception:
|
||||
pass
|
||||
return mapping
|
||||
# [/DEF:_extract_chart_uuids_from_archive:Function]
|
||||
|
||||
# [DEF:_patch_dashboard_metadata:Function]
|
||||
# @PURPOSE: Replaces integer IDs in json_metadata.
|
||||
# @PARAM: file_path (Path)
|
||||
# @PARAM: target_env_id (str)
|
||||
# @PARAM: source_map (Dict[int, str])
|
||||
def _patch_dashboard_metadata(self, file_path: Path, target_env_id: str, source_map: Dict[int, str]):
|
||||
with belief_scope("MigrationEngine._patch_dashboard_metadata"):
|
||||
try:
|
||||
with open(file_path, 'r') as f:
|
||||
data = yaml.safe_load(f)
|
||||
|
||||
if not data or 'json_metadata' not in data:
|
||||
return
|
||||
|
||||
metadata_str = data['json_metadata']
|
||||
if not metadata_str:
|
||||
return
|
||||
|
||||
metadata = json.loads(metadata_str)
|
||||
modified = False
|
||||
|
||||
# We need to deeply traverse and replace. For MVP, string replacement over the raw JSON is an option,
|
||||
# but careful dict traversal is safer.
|
||||
|
||||
# Fetch target UUIDs for everything we know:
|
||||
uuids_needed = list(source_map.values())
|
||||
target_ids = self.mapping_service.get_remote_ids_batch(target_env_id, ResourceType.CHART, uuids_needed)
|
||||
|
||||
if not target_ids:
|
||||
logger.info("[MigrationEngine._patch_dashboard_metadata][Reflect] No remote target IDs found in mapping database.")
|
||||
return
|
||||
|
||||
# Map Source Int -> Target Int
|
||||
source_to_target = {}
|
||||
missing_targets = []
|
||||
for s_id, s_uuid in source_map.items():
|
||||
if s_uuid in target_ids:
|
||||
source_to_target[s_id] = target_ids[s_uuid]
|
||||
else:
|
||||
missing_targets.append(s_id)
|
||||
|
||||
if missing_targets:
|
||||
logger.warning(f"[MigrationEngine._patch_dashboard_metadata][Coherence:Recoverable] Missing target IDs for source IDs: {missing_targets}. Cross-filters for these IDs might break.")
|
||||
|
||||
if not source_to_target:
|
||||
logger.info("[MigrationEngine._patch_dashboard_metadata][Reflect] No source IDs matched remotely. Skipping patch.")
|
||||
return
|
||||
|
||||
# Complex metadata traversal would go here (e.g. for native_filter_configuration)
|
||||
# We use regex replacement over the string for safety over unknown nested dicts.
|
||||
|
||||
new_metadata_str = metadata_str
|
||||
|
||||
# Replace chartId and datasetId assignments explicitly.
|
||||
# Pattern: "datasetId": 42 or "chartId": 42
|
||||
for s_id, t_id in source_to_target.items():
|
||||
# Replace in native_filter_configuration targets
|
||||
new_metadata_str = re.sub(r'("datasetId"\s*:\s*)' + str(s_id) + r'(\b)', r'\g<1>' + str(t_id) + r'\g<2>', new_metadata_str)
|
||||
new_metadata_str = re.sub(r'("chartId"\s*:\s*)' + str(s_id) + r'(\b)', r'\g<1>' + str(t_id) + r'\g<2>', new_metadata_str)
|
||||
|
||||
# Re-parse to validate valid JSON
|
||||
data['json_metadata'] = json.dumps(json.loads(new_metadata_str))
|
||||
|
||||
with open(file_path, 'w') as f:
|
||||
yaml.dump(data, f)
|
||||
logger.info(f"[MigrationEngine._patch_dashboard_metadata][Reason] Re-serialized modified JSON metadata for dashboard.")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[MigrationEngine._patch_dashboard_metadata][Coherence:Failed] Metadata patch failed: {e}")
|
||||
|
||||
# [/DEF:_patch_dashboard_metadata:Function]
|
||||
|
||||
# [/DEF:MigrationEngine:Class]
|
||||
|
||||
# [/DEF:backend.src.core.migration_engine:Module]
|
||||
|
||||
@@ -26,7 +26,6 @@ class DashboardSelection(BaseModel):
|
||||
source_env_id: str
|
||||
target_env_id: str
|
||||
replace_db_config: bool = False
|
||||
fix_cross_filters: bool = True
|
||||
# [/DEF:DashboardSelection:Class]
|
||||
|
||||
# [/DEF:backend.src.models.dashboard:Module]
|
||||
@@ -19,16 +19,6 @@ import enum
|
||||
|
||||
Base = declarative_base()
|
||||
|
||||
# [DEF:ResourceType:Class]
|
||||
# @TIER: TRIVIAL
|
||||
# @PURPOSE: Enumeration of possible Superset resource types for ID mapping.
|
||||
class ResourceType(str, enum.Enum):
|
||||
CHART = "chart"
|
||||
DATASET = "dataset"
|
||||
DASHBOARD = "dashboard"
|
||||
# [/DEF:ResourceType:Class]
|
||||
|
||||
|
||||
# [DEF:MigrationStatus:Class]
|
||||
# @TIER: TRIVIAL
|
||||
# @PURPOSE: Enumeration of possible migration job statuses.
|
||||
@@ -80,21 +70,6 @@ class MigrationJob(Base):
|
||||
status = Column(SQLEnum(MigrationStatus), default=MigrationStatus.PENDING)
|
||||
replace_db = Column(Boolean, default=False)
|
||||
created_at = Column(DateTime(timezone=True), server_default=func.now())
|
||||
# [DEF:ResourceMapping:Class]
|
||||
# @TIER: STANDARD
|
||||
# @PURPOSE: Maps a universal UUID for a resource to its actual ID on a specific environment.
|
||||
# @TEST_DATA: resource_mapping_record -> {'environment_id': 'prod-env-1', 'resource_type': 'chart', 'uuid': '123e4567-e89b-12d3-a456-426614174000', 'remote_integer_id': '42'}
|
||||
class ResourceMapping(Base):
|
||||
__tablename__ = "resource_mappings"
|
||||
|
||||
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
|
||||
environment_id = Column(String, ForeignKey("environments.id"), nullable=False)
|
||||
resource_type = Column(SQLEnum(ResourceType), nullable=False)
|
||||
uuid = Column(String, nullable=False)
|
||||
remote_integer_id = Column(String, nullable=False) # Stored as string to handle potentially large or composite IDs safely, though Superset usually uses integers.
|
||||
resource_name = Column(String, nullable=True) # Used for UI display
|
||||
last_synced_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
|
||||
# [/DEF:ResourceMapping:Class]
|
||||
# [/DEF:MigrationJob:Class]
|
||||
|
||||
# [/DEF:backend.src.models.mapping:Module]
|
||||
|
||||
|
||||
@@ -27,7 +27,7 @@ class TestEncryptionManager:
|
||||
# Re-implement the same logic as EncryptionManager to avoid import issues
|
||||
# with the llm_provider module's relative imports
|
||||
import os
|
||||
key = os.getenv("ENCRYPTION_KEY", "ZcytYzi0iHIl4Ttr-GdAEk117aGRogkGvN3wiTxrPpE=").encode()
|
||||
key = os.getenv("ENCRYPTION_KEY", "REMOVED_HISTORICAL_SECRET_DO_NOT_USE").encode()
|
||||
fernet = Fernet(key)
|
||||
|
||||
class EncryptionManager:
|
||||
|
||||
@@ -24,7 +24,7 @@ class EncryptionManager:
|
||||
# @PRE: ENCRYPTION_KEY env var must be set or use default dev key.
|
||||
# @POST: Fernet instance ready for encryption/decryption.
|
||||
def __init__(self):
|
||||
self.key = os.getenv("ENCRYPTION_KEY", "ZcytYzi0iHIl4Ttr-GdAEk117aGRogkGvN3wiTxrPpE=").encode()
|
||||
self.key = os.getenv("ENCRYPTION_KEY", "REMOVED_HISTORICAL_SECRET_DO_NOT_USE").encode()
|
||||
self.fernet = Fernet(self.key)
|
||||
# [/DEF:EncryptionManager.__init__:Function]
|
||||
|
||||
|
||||
BIN
backend/tasks.db
BIN
backend/tasks.db
Binary file not shown.
@@ -1,76 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Debug script to test Superset API authentication"""
|
||||
|
||||
from pprint import pprint
|
||||
from src.core.superset_client import SupersetClient
|
||||
from src.core.config_manager import ConfigManager
|
||||
|
||||
|
||||
def main():
|
||||
print("Debugging Superset API authentication...")
|
||||
|
||||
config = ConfigManager()
|
||||
|
||||
# Select first available environment
|
||||
environments = config.get_environments()
|
||||
|
||||
if not environments:
|
||||
print("No environments configured")
|
||||
return
|
||||
|
||||
env = environments[0]
|
||||
print(f"\nTesting environment: {env.name}")
|
||||
print(f"URL: {env.url}")
|
||||
|
||||
try:
|
||||
# Test API client authentication
|
||||
print("\n--- Testing API Authentication ---")
|
||||
client = SupersetClient(env)
|
||||
tokens = client.authenticate()
|
||||
|
||||
print("\nAPI Auth Success!")
|
||||
print(f"Access Token: {tokens.get('access_token', 'N/A')}")
|
||||
print(f"CSRF Token: {tokens.get('csrf_token', 'N/A')}")
|
||||
|
||||
# Debug cookies from session
|
||||
print("\n--- Session Cookies ---")
|
||||
for cookie in client.network.session.cookies:
|
||||
print(f"{cookie.name}={cookie.value}")
|
||||
|
||||
# Test accessing UI via requests
|
||||
print("\n--- Testing UI Access ---")
|
||||
ui_url = env.url.rstrip('/').replace('/api/v1', '')
|
||||
print(f"UI URL: {ui_url}")
|
||||
|
||||
# Try to access UI home page
|
||||
ui_response = client.network.session.get(ui_url, timeout=30, allow_redirects=True)
|
||||
print(f"Status Code: {ui_response.status_code}")
|
||||
print(f"URL: {ui_response.url}")
|
||||
|
||||
# Check response headers
|
||||
print("\n--- Response Headers ---")
|
||||
pprint(dict(ui_response.headers))
|
||||
|
||||
print("\n--- Response Content Preview (200 chars) ---")
|
||||
print(repr(ui_response.text[:200]))
|
||||
|
||||
if ui_response.status_code == 200:
|
||||
print("\nUI Access: Success")
|
||||
|
||||
# Try to access a dashboard
|
||||
# For testing, just use the home page
|
||||
print("\n--- Checking if login is required ---")
|
||||
if "login" in ui_response.url.lower() or "login" in ui_response.text.lower():
|
||||
print("❌ Not logged in to UI")
|
||||
else:
|
||||
print("✅ Logged in to UI")
|
||||
|
||||
except Exception as e:
|
||||
print(f"\n❌ Error: {type(e).__name__}: {e}")
|
||||
import traceback
|
||||
print("\nStack Trace:")
|
||||
print(traceback.format_exc())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,44 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Test script to debug API key decryption issue."""
|
||||
|
||||
from src.core.database import SessionLocal
|
||||
from src.models.llm import LLMProvider
|
||||
from cryptography.fernet import Fernet
|
||||
import os
|
||||
|
||||
# Get the encryption key
|
||||
key = os.getenv("ENCRYPTION_KEY", "ZcytYzi0iHIl4Ttr-GdAEk117aGRogkGvN3wiTxrPpE=").encode()
|
||||
print(f"Encryption key (first 20 chars): {key[:20]}")
|
||||
print(f"Encryption key length: {len(key)}")
|
||||
|
||||
# Create Fernet instance
|
||||
fernet = Fernet(key)
|
||||
|
||||
# Get provider from database
|
||||
db = SessionLocal()
|
||||
provider = db.query(LLMProvider).filter(LLMProvider.id == '6c899741-4108-4196-aea4-f38ad2f0150e').first()
|
||||
|
||||
if provider:
|
||||
print("\nProvider found:")
|
||||
print(f" ID: {provider.id}")
|
||||
print(f" Name: {provider.name}")
|
||||
print(f" Encrypted API Key (first 50 chars): {provider.api_key[:50]}")
|
||||
print(f" Encrypted API Key Length: {len(provider.api_key)}")
|
||||
|
||||
# Test decryption
|
||||
print("\nAttempting decryption...")
|
||||
try:
|
||||
decrypted = fernet.decrypt(provider.api_key.encode()).decode()
|
||||
print("Decryption successful!")
|
||||
print(f" Decrypted key length: {len(decrypted)}")
|
||||
print(f" Decrypted key (first 8 chars): {decrypted[:8]}")
|
||||
print(f" Decrypted key is empty: {len(decrypted) == 0}")
|
||||
except Exception as e:
|
||||
print(f"Decryption failed with error: {e}")
|
||||
print(f"Error type: {type(e).__name__}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
else:
|
||||
print("Provider not found")
|
||||
|
||||
db.close()
|
||||
@@ -1 +0,0 @@
|
||||
[{"key[": 20, ")\n\n# Create Fernet instance\nfernet = Fernet(key)\n\n# Test encrypting an empty string\nempty_encrypted = fernet.encrypt(b\"": ".", "print(f": "nEncrypted empty string: {empty_encrypted"}, {"test-api-key-12345\"\ntest_encrypted = fernet.encrypt(test_key.encode()).decode()\nprint(f": "nEncrypted test key: {test_encrypted"}, {"gAAAAABphhwSZie0OwXjJ78Fk-c4Uo6doNJXipX49AX7Bypzp4ohiRX3hXPXKb45R1vhNUOqbm6Ke3-eRwu_KdWMZ9chFBKmqw==\"\nprint(f": "nStored encrypted key: {stored_key"}, {"len(stored_key)}": "Check if stored key matches empty string encryption\nif stored_key == empty_encrypted:\n print(", "string!": "else:\n print(", "print(f": "mpty string encryption: {empty_encrypted"}, {"stored_key}": "Try to decrypt the stored key\ntry:\n decrypted = fernet.decrypt(stored_key.encode()).decode()\n print(f", "print(f": "ecrypted key length: {len(decrypted)"}, {")\nexcept Exception as e:\n print(f": "nDecryption failed with error: {e"}]
|
||||
@@ -1,99 +0,0 @@
|
||||
# [DEF:backend.tests.core.test_mapping_service:Module]
|
||||
#
|
||||
# @TIER: STANDARD
|
||||
# @PURPOSE: Unit tests for the IdMappingService matching UUIDs to integer IDs.
|
||||
# @LAYER: Domain
|
||||
# @RELATION: VERIFIES -> backend.src.core.mapping_service
|
||||
#
|
||||
import pytest
|
||||
from datetime import datetime, timezone
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
import sys
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
# Add backend directory to sys.path so 'src' can be resolved
|
||||
backend_dir = str(Path(__file__).parent.parent.parent.resolve())
|
||||
if backend_dir not in sys.path:
|
||||
sys.path.insert(0, backend_dir)
|
||||
|
||||
from src.models.mapping import Base, ResourceMapping, ResourceType
|
||||
from src.core.mapping_service import IdMappingService
|
||||
|
||||
@pytest.fixture
|
||||
def db_session():
|
||||
# In-memory SQLite for testing
|
||||
engine = create_engine('sqlite:///:memory:')
|
||||
Base.metadata.create_all(engine)
|
||||
Session = sessionmaker(bind=engine)
|
||||
session = Session()
|
||||
yield session
|
||||
session.close()
|
||||
|
||||
class MockSupersetClient:
|
||||
def __init__(self, resources):
|
||||
self.resources = resources
|
||||
|
||||
def get_all_resources(self, endpoint):
|
||||
return self.resources.get(endpoint, [])
|
||||
|
||||
def test_sync_environment_upserts_correctly(db_session):
|
||||
service = IdMappingService(db_session)
|
||||
mock_client = MockSupersetClient({
|
||||
"chart": [
|
||||
{"id": 42, "uuid": "123e4567-e89b-12d3-a456-426614174000", "slice_name": "Test Chart"}
|
||||
]
|
||||
})
|
||||
|
||||
service.sync_environment("test-env", mock_client)
|
||||
|
||||
mapping = db_session.query(ResourceMapping).first()
|
||||
assert mapping is not None
|
||||
assert mapping.environment_id == "test-env"
|
||||
assert mapping.resource_type == ResourceType.CHART
|
||||
assert mapping.uuid == "123e4567-e89b-12d3-a456-426614174000"
|
||||
assert mapping.remote_integer_id == "42"
|
||||
assert mapping.resource_name == "Test Chart"
|
||||
|
||||
def test_get_remote_id_returns_integer(db_session):
|
||||
service = IdMappingService(db_session)
|
||||
mapping = ResourceMapping(
|
||||
environment_id="test-env",
|
||||
resource_type=ResourceType.DATASET,
|
||||
uuid="uuid-1",
|
||||
remote_integer_id="99",
|
||||
resource_name="Test DS",
|
||||
last_synced_at=datetime.now(timezone.utc)
|
||||
)
|
||||
db_session.add(mapping)
|
||||
db_session.commit()
|
||||
|
||||
result = service.get_remote_id("test-env", ResourceType.DATASET, "uuid-1")
|
||||
assert result == 99
|
||||
|
||||
def test_get_remote_ids_batch_returns_dict(db_session):
|
||||
service = IdMappingService(db_session)
|
||||
m1 = ResourceMapping(
|
||||
environment_id="test-env",
|
||||
resource_type=ResourceType.DASHBOARD,
|
||||
uuid="uuid-1",
|
||||
remote_integer_id="11"
|
||||
)
|
||||
m2 = ResourceMapping(
|
||||
environment_id="test-env",
|
||||
resource_type=ResourceType.DASHBOARD,
|
||||
uuid="uuid-2",
|
||||
remote_integer_id="22"
|
||||
)
|
||||
db_session.add_all([m1, m2])
|
||||
db_session.commit()
|
||||
|
||||
result = service.get_remote_ids_batch("test-env", ResourceType.DASHBOARD, ["uuid-1", "uuid-2", "uuid-missing"])
|
||||
|
||||
assert len(result) == 2
|
||||
assert result["uuid-1"] == 11
|
||||
assert result["uuid-2"] == 22
|
||||
assert "uuid-missing" not in result
|
||||
|
||||
# [/DEF:backend.tests.core.test_mapping_service:Module]
|
||||
@@ -1,66 +0,0 @@
|
||||
# [DEF:backend.tests.core.test_migration_engine:Module]
|
||||
#
|
||||
# @TIER: STANDARD
|
||||
# @PURPOSE: Unit tests for MigrationEngine's cross-filter patching algorithms.
|
||||
# @LAYER: Domain
|
||||
# @RELATION: VERIFIES -> backend.src.core.migration_engine
|
||||
#
|
||||
import pytest
|
||||
import tempfile
|
||||
import json
|
||||
import yaml
|
||||
import sys
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
backend_dir = str(Path(__file__).parent.parent.parent.resolve())
|
||||
if backend_dir not in sys.path:
|
||||
sys.path.insert(0, backend_dir)
|
||||
|
||||
from src.core.migration_engine import MigrationEngine
|
||||
from src.core.mapping_service import IdMappingService
|
||||
from src.models.mapping import ResourceType
|
||||
|
||||
class MockMappingService:
|
||||
def __init__(self, mappings):
|
||||
self.mappings = mappings
|
||||
|
||||
def get_remote_ids_batch(self, env_id, resource_type, uuids):
|
||||
result = {}
|
||||
for uuid in uuids:
|
||||
if uuid in self.mappings:
|
||||
result[uuid] = self.mappings[uuid]
|
||||
return result
|
||||
|
||||
def test_patch_dashboard_metadata_replaces_ids():
|
||||
engine = MigrationEngine(MockMappingService({"uuid-target-1": 999}))
|
||||
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
file_path = Path(td) / "dash.yaml"
|
||||
|
||||
# Setup mock dashboard file
|
||||
original_metadata = {
|
||||
"native_filter_configuration": [
|
||||
{
|
||||
"targets": [{"datasetId": 10}, {"datasetId": 42}] # 42 is our source ID
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
with open(file_path, 'w') as f:
|
||||
yaml.dump({"json_metadata": json.dumps(original_metadata)}, f)
|
||||
|
||||
source_map = {42: "uuid-target-1"} # Source ID 42 translates to Target ID 999
|
||||
|
||||
engine._patch_dashboard_metadata(file_path, "test-env", source_map)
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
data = yaml.safe_load(f)
|
||||
new_metadata = json.loads(data["json_metadata"])
|
||||
|
||||
# Since simple string replacement isn't implemented strictly in the engine yet
|
||||
# (we left a placeholder `pass` for dataset replacement), this test sets up the
|
||||
# infrastructure to verify the patch once fully mapped.
|
||||
pass
|
||||
|
||||
# [/DEF:backend.tests.core.test_migration_engine:Module]
|
||||
@@ -644,12 +644,7 @@
|
||||
<div class="mt-3 flex flex-wrap gap-2">
|
||||
{#each message.actions as action}
|
||||
<button
|
||||
class="rounded-md border px-3 py-1.5 text-xs font-semibold transition
|
||||
{action.type === 'confirm'
|
||||
? 'border-emerald-300 bg-emerald-50 text-emerald-700 hover:bg-emerald-100'
|
||||
: action.type === 'cancel'
|
||||
? 'border-rose-300 bg-rose-50 text-rose-700 hover:bg-rose-100'
|
||||
: 'border-slate-300 bg-white text-slate-700 hover:bg-slate-100'}"
|
||||
class="rounded-md border border-slate-300 px-2.5 py-1 text-xs font-medium text-slate-700 transition hover:bg-slate-100"
|
||||
on:click={() => handleAction(action, message)}
|
||||
>
|
||||
{action.label}
|
||||
|
||||
@@ -10,23 +10,20 @@
|
||||
|
||||
<script lang="ts">
|
||||
// [SECTION: IMPORTS]
|
||||
import { onMount } from "svelte";
|
||||
import EnvSelector from "../../components/EnvSelector.svelte";
|
||||
import DashboardGrid from "../../components/DashboardGrid.svelte";
|
||||
import MappingTable from "../../components/MappingTable.svelte";
|
||||
import TaskRunner from "../../components/TaskRunner.svelte";
|
||||
import TaskHistory from "../../components/TaskHistory.svelte";
|
||||
import TaskLogViewer from "../../components/TaskLogViewer.svelte";
|
||||
import PasswordPrompt from "../../components/PasswordPrompt.svelte";
|
||||
import { api } from "../../lib/api.js";
|
||||
import { selectedTask } from "../../lib/stores.js";
|
||||
import { resumeTask } from "../../services/taskService.js";
|
||||
import type {
|
||||
DashboardMetadata,
|
||||
DashboardSelection,
|
||||
} from "../../types/dashboard";
|
||||
import { t } from "$lib/i18n";
|
||||
import { Button, Card, PageHeader } from "$lib/ui";
|
||||
import { onMount } from 'svelte';
|
||||
import EnvSelector from '../../components/EnvSelector.svelte';
|
||||
import DashboardGrid from '../../components/DashboardGrid.svelte';
|
||||
import MappingTable from '../../components/MappingTable.svelte';
|
||||
import TaskRunner from '../../components/TaskRunner.svelte';
|
||||
import TaskHistory from '../../components/TaskHistory.svelte';
|
||||
import TaskLogViewer from '../../components/TaskLogViewer.svelte';
|
||||
import PasswordPrompt from '../../components/PasswordPrompt.svelte';
|
||||
import { api } from '../../lib/api.js';
|
||||
import { selectedTask } from '../../lib/stores.js';
|
||||
import { resumeTask } from '../../services/taskService.js';
|
||||
import type { DashboardMetadata, DashboardSelection } from '../../types/dashboard';
|
||||
import { t } from '$lib/i18n';
|
||||
import { Button, Card, PageHeader } from '$lib/ui';
|
||||
// [/SECTION]
|
||||
|
||||
// [SECTION: STATE]
|
||||
@@ -34,7 +31,6 @@
|
||||
let sourceEnvId = "";
|
||||
let targetEnvId = "";
|
||||
let replaceDb = false;
|
||||
let fixCrossFilters = true;
|
||||
let loading = true;
|
||||
let error = "";
|
||||
let dashboards: DashboardMetadata[] = [];
|
||||
@@ -44,12 +40,12 @@
|
||||
let mappings: any[] = [];
|
||||
let suggestions: any[] = [];
|
||||
let fetchingDbs = false;
|
||||
|
||||
|
||||
// UI State for Modals
|
||||
let showLogViewer = false;
|
||||
let logViewerTaskId: string | null = null;
|
||||
let logViewerTaskStatus: string | null = null;
|
||||
|
||||
|
||||
let showPasswordPrompt = false;
|
||||
let passwordPromptDatabases: string[] = [];
|
||||
let passwordPromptErrorMessage = "";
|
||||
@@ -105,18 +101,13 @@
|
||||
if (!sourceEnvId || !targetEnvId) return;
|
||||
fetchingDbs = true;
|
||||
error = "";
|
||||
|
||||
|
||||
try {
|
||||
const [src, tgt, maps, sugs] = await Promise.all([
|
||||
api.requestApi(`/environments/${sourceEnvId}/databases`),
|
||||
api.requestApi(`/environments/${targetEnvId}/databases`),
|
||||
api.requestApi(
|
||||
`/mappings?source_env_id=${sourceEnvId}&target_env_id=${targetEnvId}`,
|
||||
),
|
||||
api.postApi(`/mappings/suggest`, {
|
||||
source_env_id: sourceEnvId,
|
||||
target_env_id: targetEnvId,
|
||||
}),
|
||||
api.requestApi(`/mappings?source_env_id=${sourceEnvId}&target_env_id=${targetEnvId}`),
|
||||
api.postApi(`/mappings/suggest`, { source_env_id: sourceEnvId, target_env_id: targetEnvId })
|
||||
]);
|
||||
|
||||
sourceDatabases = src;
|
||||
@@ -139,25 +130,22 @@
|
||||
*/
|
||||
async function handleMappingUpdate(event: CustomEvent) {
|
||||
const { sourceUuid, targetUuid } = event.detail;
|
||||
const sDb = sourceDatabases.find((d) => d.uuid === sourceUuid);
|
||||
const tDb = targetDatabases.find((d) => d.uuid === targetUuid);
|
||||
|
||||
const sDb = sourceDatabases.find(d => d.uuid === sourceUuid);
|
||||
const tDb = targetDatabases.find(d => d.uuid === targetUuid);
|
||||
|
||||
if (!sDb || !tDb) return;
|
||||
|
||||
try {
|
||||
const savedMapping = await api.postApi("/mappings", {
|
||||
const savedMapping = await api.postApi('/mappings', {
|
||||
source_env_id: sourceEnvId,
|
||||
target_env_id: targetEnvId,
|
||||
source_db_uuid: sourceUuid,
|
||||
target_db_uuid: targetUuid,
|
||||
source_db_name: sDb.database_name,
|
||||
target_db_name: tDb.database_name,
|
||||
target_db_name: tDb.database_name
|
||||
});
|
||||
|
||||
mappings = [
|
||||
...mappings.filter((m) => m.source_db_uuid !== sourceUuid),
|
||||
savedMapping,
|
||||
];
|
||||
mappings = [...mappings.filter(m => m.source_db_uuid !== sourceUuid), savedMapping];
|
||||
} catch (e) {
|
||||
error = e.message;
|
||||
}
|
||||
@@ -169,10 +157,10 @@
|
||||
// @PRE: event.detail contains task object.
|
||||
// @POST: logViewer state updated and showLogViewer set to true.
|
||||
function handleViewLogs(event: CustomEvent) {
|
||||
const task = event.detail;
|
||||
logViewerTaskId = task.id;
|
||||
logViewerTaskStatus = task.status;
|
||||
showLogViewer = true;
|
||||
const task = event.detail;
|
||||
logViewerTaskId = task.id;
|
||||
logViewerTaskStatus = task.status;
|
||||
showLogViewer = true;
|
||||
}
|
||||
// [/DEF:handleViewLogs:Function]
|
||||
|
||||
@@ -184,23 +172,19 @@
|
||||
// For now, we rely on the WebSocket or manual check.
|
||||
// Ideally, TaskHistory or TaskRunner emits an event when input is needed.
|
||||
// Or we watch selectedTask.
|
||||
|
||||
$: if (
|
||||
$selectedTask &&
|
||||
$selectedTask.status === "AWAITING_INPUT" &&
|
||||
$selectedTask.input_request
|
||||
) {
|
||||
const req = $selectedTask.input_request;
|
||||
if (req.type === "database_password") {
|
||||
passwordPromptDatabases = req.databases || [];
|
||||
passwordPromptErrorMessage = req.error_message || "";
|
||||
showPasswordPrompt = true;
|
||||
}
|
||||
} else if (!$selectedTask || $selectedTask.status !== "AWAITING_INPUT") {
|
||||
// Close prompt if task is no longer waiting (e.g. resumed)
|
||||
// But only if we are viewing this task.
|
||||
// showPasswordPrompt = false;
|
||||
// Actually, don't auto-close, let the user or success handler close it.
|
||||
|
||||
$: if ($selectedTask && $selectedTask.status === 'AWAITING_INPUT' && $selectedTask.input_request) {
|
||||
const req = $selectedTask.input_request;
|
||||
if (req.type === 'database_password') {
|
||||
passwordPromptDatabases = req.databases || [];
|
||||
passwordPromptErrorMessage = req.error_message || "";
|
||||
showPasswordPrompt = true;
|
||||
}
|
||||
} else if (!$selectedTask || $selectedTask.status !== 'AWAITING_INPUT') {
|
||||
// Close prompt if task is no longer waiting (e.g. resumed)
|
||||
// But only if we are viewing this task.
|
||||
// showPasswordPrompt = false;
|
||||
// Actually, don't auto-close, let the user or success handler close it.
|
||||
}
|
||||
// [/DEF:handlePasswordPrompt:Function]
|
||||
|
||||
@@ -209,19 +193,18 @@
|
||||
// @PRE: event.detail contains passwords.
|
||||
// @POST: resumeTask is called and showPasswordPrompt is hidden on success.
|
||||
async function handleResumeMigration(event: CustomEvent) {
|
||||
if (!$selectedTask) return;
|
||||
|
||||
const { passwords } = event.detail;
|
||||
try {
|
||||
await resumeTask($selectedTask.id, passwords);
|
||||
showPasswordPrompt = false;
|
||||
// Task status update will be handled by store/websocket
|
||||
} catch (e) {
|
||||
console.error("Failed to resume task:", e);
|
||||
passwordPromptErrorMessage =
|
||||
e.message || $t.migration?.resume_failed || "Failed to resume task";
|
||||
// Keep prompt open
|
||||
}
|
||||
if (!$selectedTask) return;
|
||||
|
||||
const { passwords } = event.detail;
|
||||
try {
|
||||
await resumeTask($selectedTask.id, passwords);
|
||||
showPasswordPrompt = false;
|
||||
// Task status update will be handled by store/websocket
|
||||
} catch (e) {
|
||||
console.error("Failed to resume task:", e);
|
||||
passwordPromptErrorMessage = e.message || ($t.migration?.resume_failed || "Failed to resume task");
|
||||
// Keep prompt open
|
||||
}
|
||||
}
|
||||
// [/DEF:handleResumeMigration:Function]
|
||||
|
||||
@@ -233,21 +216,15 @@
|
||||
*/
|
||||
async function startMigration() {
|
||||
if (!sourceEnvId || !targetEnvId) {
|
||||
error =
|
||||
$t.migration?.select_both_envs ||
|
||||
"Please select both source and target environments.";
|
||||
error = $t.migration?.select_both_envs || "Please select both source and target environments.";
|
||||
return;
|
||||
}
|
||||
if (sourceEnvId === targetEnvId) {
|
||||
error =
|
||||
$t.migration?.different_envs ||
|
||||
"Source and target environments must be different.";
|
||||
error = $t.migration?.different_envs || "Source and target environments must be different.";
|
||||
return;
|
||||
}
|
||||
if (selectedDashboardIds.length === 0) {
|
||||
error =
|
||||
$t.migration?.select_dashboards ||
|
||||
"Please select at least one dashboard to migrate.";
|
||||
error = $t.migration?.select_dashboards || "Please select at least one dashboard to migrate.";
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -257,37 +234,28 @@
|
||||
selected_ids: selectedDashboardIds,
|
||||
source_env_id: sourceEnvId,
|
||||
target_env_id: targetEnvId,
|
||||
replace_db_config: replaceDb,
|
||||
fix_cross_filters: fixCrossFilters,
|
||||
replace_db_config: replaceDb
|
||||
};
|
||||
console.log(
|
||||
`[MigrationDashboard][Action] Starting migration with selection:`,
|
||||
selection,
|
||||
);
|
||||
const result = await api.postApi("/migration/execute", selection);
|
||||
console.log(
|
||||
`[MigrationDashboard][Action] Migration started: ${result.task_id} - ${result.message}`,
|
||||
);
|
||||
|
||||
console.log(`[MigrationDashboard][Action] Starting migration with selection:`, selection);
|
||||
const result = await api.postApi('/migration/execute', selection);
|
||||
console.log(`[MigrationDashboard][Action] Migration started: ${result.task_id} - ${result.message}`);
|
||||
|
||||
// Wait a brief moment for the backend to ensure the task is retrievable
|
||||
await new Promise((r) => setTimeout(r, 500));
|
||||
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
|
||||
// Fetch full task details and switch to TaskRunner view
|
||||
try {
|
||||
const task = await api.getTask(result.task_id);
|
||||
selectedTask.set(task);
|
||||
} catch (fetchErr) {
|
||||
// Fallback: create a temporary task object to switch view immediately
|
||||
console.warn(
|
||||
$t.migration?.task_placeholder_warn ||
|
||||
"Could not fetch task details immediately, using placeholder.",
|
||||
);
|
||||
console.warn($t.migration?.task_placeholder_warn || "Could not fetch task details immediately, using placeholder.");
|
||||
selectedTask.set({
|
||||
id: result.task_id,
|
||||
plugin_id: "superset-migration",
|
||||
status: "RUNNING",
|
||||
logs: [],
|
||||
params: {},
|
||||
id: result.task_id,
|
||||
plugin_id: 'superset-migration',
|
||||
status: 'RUNNING',
|
||||
logs: [],
|
||||
params: {}
|
||||
});
|
||||
}
|
||||
} catch (e) {
|
||||
@@ -301,7 +269,7 @@
|
||||
<!-- [SECTION: TEMPLATE] -->
|
||||
<div class="max-w-4xl mx-auto p-6">
|
||||
<PageHeader title={$t.nav.migration} />
|
||||
|
||||
|
||||
<TaskHistory on:viewLogs={handleViewLogs} />
|
||||
|
||||
{#if $selectedTask}
|
||||
@@ -317,9 +285,7 @@
|
||||
{#if loading}
|
||||
<p>{$t.migration?.loading_envs || "Loading environments..."}</p>
|
||||
{:else if error}
|
||||
<div
|
||||
class="bg-red-100 border border-red-400 text-red-700 px-4 py-3 rounded mb-4"
|
||||
>
|
||||
<div class="bg-red-100 border border-red-400 text-red-700 px-4 py-3 rounded mb-4">
|
||||
{error}
|
||||
</div>
|
||||
{/if}
|
||||
@@ -339,10 +305,8 @@
|
||||
|
||||
<!-- [DEF:DashboardSelectionSection:Component] -->
|
||||
<div class="mb-8">
|
||||
<h2 class="text-lg font-medium mb-4">
|
||||
{$t.migration?.select_dashboards_title || "Select Dashboards"}
|
||||
</h2>
|
||||
|
||||
<h2 class="text-lg font-medium mb-4">{$t.migration?.select_dashboards_title || "Select Dashboards"}</h2>
|
||||
|
||||
{#if sourceEnvId}
|
||||
<DashboardGrid
|
||||
{dashboards}
|
||||
@@ -350,54 +314,30 @@
|
||||
environmentId={sourceEnvId}
|
||||
/>
|
||||
{:else}
|
||||
<p class="text-gray-500 italic">
|
||||
{$t.dashboard?.select_source ||
|
||||
"Select a source environment to view dashboards."}
|
||||
</p>
|
||||
<p class="text-gray-500 italic">{$t.dashboard?.select_source || "Select a source environment to view dashboards."}</p>
|
||||
{/if}
|
||||
</div>
|
||||
<!-- [/DEF:DashboardSelectionSection:Component] -->
|
||||
|
||||
<div class="mb-4">
|
||||
<div class="flex items-center mb-2">
|
||||
<input
|
||||
id="fix-cross-filters"
|
||||
type="checkbox"
|
||||
bind:checked={fixCrossFilters}
|
||||
class="h-4 w-4 text-indigo-600 focus:ring-indigo-500 border-gray-300 rounded"
|
||||
/>
|
||||
<label for="fix-cross-filters" class="ml-2 block text-sm text-gray-900">
|
||||
{$t.migration?.fix_cross_filters ||
|
||||
"Fix Cross-Filters (Auto-repair broken links during migration)"}
|
||||
</label>
|
||||
</div>
|
||||
|
||||
<div class="flex items-center">
|
||||
<input
|
||||
id="replace-db"
|
||||
type="checkbox"
|
||||
bind:checked={replaceDb}
|
||||
on:change={() => {
|
||||
if (replaceDb && sourceDatabases.length === 0) fetchDatabases();
|
||||
}}
|
||||
class="h-4 w-4 text-indigo-600 focus:ring-indigo-500 border-gray-300 rounded"
|
||||
/>
|
||||
<label for="replace-db" class="ml-2 block text-sm text-gray-900">
|
||||
{$t.migration?.replace_db || "Replace Database (Apply Mappings)"}
|
||||
</label>
|
||||
</div>
|
||||
<div class="flex items-center mb-4">
|
||||
<input
|
||||
id="replace-db"
|
||||
type="checkbox"
|
||||
bind:checked={replaceDb}
|
||||
on:change={() => { if (replaceDb && sourceDatabases.length === 0) fetchDatabases(); }}
|
||||
class="h-4 w-4 text-indigo-600 focus:ring-indigo-500 border-gray-300 rounded"
|
||||
/>
|
||||
<label for="replace-db" class="ml-2 block text-sm text-gray-900">
|
||||
{$t.migration?.replace_db || "Replace Database (Apply Mappings)"}
|
||||
</label>
|
||||
</div>
|
||||
|
||||
{#if replaceDb}
|
||||
<div class="mb-8 p-4 border rounded-md bg-gray-50">
|
||||
<h3 class="text-md font-medium mb-4">
|
||||
{$t.migration?.database_mappings || "Database Mappings"}
|
||||
</h3>
|
||||
<h3 class="text-md font-medium mb-4">{$t.migration?.database_mappings || "Database Mappings"}</h3>
|
||||
{#if fetchingDbs}
|
||||
<p>
|
||||
{$t.migration?.loading_dbs ||
|
||||
"Loading databases and suggestions..."}
|
||||
</p>
|
||||
<p>{$t.migration?.loading_dbs || "Loading databases and suggestions..."}</p>
|
||||
{:else if sourceDatabases.length > 0}
|
||||
<MappingTable
|
||||
{sourceDatabases}
|
||||
@@ -419,10 +359,7 @@
|
||||
|
||||
<Button
|
||||
on:click={startMigration}
|
||||
disabled={!sourceEnvId ||
|
||||
!targetEnvId ||
|
||||
sourceEnvId === targetEnvId ||
|
||||
selectedDashboardIds.length === 0}
|
||||
disabled={!sourceEnvId || !targetEnvId || sourceEnvId === targetEnvId || selectedDashboardIds.length === 0}
|
||||
>
|
||||
{$t.migration?.start || "Start Migration"}
|
||||
</Button>
|
||||
@@ -431,20 +368,21 @@
|
||||
|
||||
<!-- Modals -->
|
||||
<TaskLogViewer
|
||||
bind:show={showLogViewer}
|
||||
taskId={logViewerTaskId}
|
||||
taskStatus={logViewerTaskStatus}
|
||||
on:close={() => (showLogViewer = false)}
|
||||
bind:show={showLogViewer}
|
||||
taskId={logViewerTaskId}
|
||||
taskStatus={logViewerTaskStatus}
|
||||
on:close={() => showLogViewer = false}
|
||||
/>
|
||||
|
||||
<PasswordPrompt
|
||||
bind:show={showPasswordPrompt}
|
||||
databases={passwordPromptDatabases}
|
||||
errorMessage={passwordPromptErrorMessage}
|
||||
on:resume={handleResumeMigration}
|
||||
on:cancel={() => (showPasswordPrompt = false)}
|
||||
bind:show={showPasswordPrompt}
|
||||
databases={passwordPromptDatabases}
|
||||
errorMessage={passwordPromptErrorMessage}
|
||||
on:resume={handleResumeMigration}
|
||||
on:cancel={() => showPasswordPrompt = false}
|
||||
/>
|
||||
|
||||
<!-- [/SECTION] -->
|
||||
|
||||
|
||||
<!-- [/DEF:MigrationDashboard:Component] -->
|
||||
|
||||
@@ -22,9 +22,9 @@
|
||||
|
||||
const DEFAULT_LLM_PROMPTS = {
|
||||
dashboard_validation_prompt:
|
||||
'Analyze the attached dashboard screenshot and the following execution logs for health and visual issues.\\n\\nLogs:\\n{logs}\\n\\nProvide the analysis in JSON format with the following structure:\\n{\\n \\"status\\": \\"PASS\\" | \\"WARN\\" | \\"FAIL\\",\\n \\"summary\\": \\"Short summary of findings\\",\\n \\"issues\\": [\\n {\\n \\"severity\\": \\"WARN\\" | \\"FAIL\\",\\n \\"message\\": \\"Description of the issue\\",\\n \\"location\\": \\"Optional location info (e.g. chart name)\\"\\n }\\n ]\\n}',
|
||||
"Analyze the attached dashboard screenshot and the following execution logs for health and visual issues.\\n\\nLogs:\\n{logs}\\n\\nProvide the analysis in JSON format with the following structure:\\n{\\n \\\"status\\\": \\\"PASS\\\" | \\\"WARN\\\" | \\\"FAIL\\\",\\n \\\"summary\\\": \\\"Short summary of findings\\\",\\n \\\"issues\\\": [\\n {\\n \\\"severity\\\": \\\"WARN\\\" | \\\"FAIL\\\",\\n \\\"message\\\": \\\"Description of the issue\\\",\\n \\\"location\\\": \\\"Optional location info (e.g. chart name)\\\"\\n }\\n ]\\n}",
|
||||
documentation_prompt:
|
||||
'Generate professional documentation for the following dataset and its columns.\\nDataset: {dataset_name}\\nColumns: {columns_json}\\n\\nProvide the documentation in JSON format:\\n{\\n \\"dataset_description\\": \\"General description of the dataset\\",\\n \\"column_descriptions\\": [\\n {\\n \\"name\\": \\"column_name\\",\\n \\"description\\": \\"Generated description\\"\\n }\\n ]\\n}',
|
||||
"Generate professional documentation for the following dataset and its columns.\\nDataset: {dataset_name}\\nColumns: {columns_json}\\n\\nProvide the documentation in JSON format:\\n{\\n \\\"dataset_description\\\": \\\"General description of the dataset\\\",\\n \\\"column_descriptions\\\": [\\n {\\n \\\"name\\\": \\\"column_name\\\",\\n \\\"description\\\": \\\"Generated description\\\"\\n }\\n ]\\n}",
|
||||
git_commit_prompt:
|
||||
"Generate a concise and professional git commit message based on the following diff and recent history.\\nUse Conventional Commits format (e.g., feat: ..., fix: ..., docs: ...).\\n\\nRecent History:\\n{history}\\n\\nDiff:\\n{diff}\\n\\nCommit Message:",
|
||||
};
|
||||
@@ -59,7 +59,6 @@
|
||||
// Load settings on mount
|
||||
onMount(async () => {
|
||||
await loadSettings();
|
||||
await loadMigrationSettings();
|
||||
});
|
||||
|
||||
// Load consolidated settings from API
|
||||
@@ -96,8 +95,7 @@
|
||||
...DEFAULT_LLM_PROVIDER_BINDINGS,
|
||||
...(llm?.provider_bindings || {}),
|
||||
};
|
||||
normalized.assistant_planner_provider =
|
||||
llm?.assistant_planner_provider || "";
|
||||
normalized.assistant_planner_provider = llm?.assistant_planner_provider || "";
|
||||
normalized.assistant_planner_model = llm?.assistant_planner_model || "";
|
||||
return normalized;
|
||||
}
|
||||
@@ -118,9 +116,7 @@
|
||||
|
||||
function getProviderById(providerId) {
|
||||
if (!providerId) return null;
|
||||
return (
|
||||
(settings?.llm_providers || []).find((p) => p.id === providerId) || null
|
||||
);
|
||||
return (settings?.llm_providers || []).find((p) => p.id === providerId) || null;
|
||||
}
|
||||
|
||||
function isDashboardValidationBindingValid() {
|
||||
@@ -133,9 +129,6 @@
|
||||
// Handle tab change
|
||||
function handleTabChange(tab) {
|
||||
activeTab = tab;
|
||||
if (tab === "migration") {
|
||||
loadMigrationSettings();
|
||||
}
|
||||
}
|
||||
|
||||
// Get tab class
|
||||
@@ -145,44 +138,6 @@
|
||||
: "text-gray-600 hover:text-gray-800 border-transparent hover:border-gray-300";
|
||||
}
|
||||
|
||||
// Migration Settings State
|
||||
let migrationCron = "0 2 * * *";
|
||||
let displayMappings = [];
|
||||
let isSavingMigration = false;
|
||||
let isLoadingMigration = false;
|
||||
|
||||
async function loadMigrationSettings() {
|
||||
isLoadingMigration = true;
|
||||
try {
|
||||
const settingsRes = await api.requestApi("/migration/settings");
|
||||
migrationCron = settingsRes.cron;
|
||||
const mappingsRes = await api.requestApi("/migration/mappings-data");
|
||||
displayMappings = mappingsRes;
|
||||
} catch (err) {
|
||||
console.error("[SettingsPage][Migration] Failed to load:", err);
|
||||
} finally {
|
||||
isLoadingMigration = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function saveMigrationSettings() {
|
||||
isSavingMigration = true;
|
||||
try {
|
||||
await api.putApi("/migration/settings", { cron: migrationCron });
|
||||
addToast(
|
||||
$t.settings?.save_success || "Migration settings saved",
|
||||
"success",
|
||||
);
|
||||
} catch (err) {
|
||||
addToast(
|
||||
$t.settings?.save_failed || "Failed to save migration settings",
|
||||
"error",
|
||||
);
|
||||
} finally {
|
||||
isSavingMigration = false;
|
||||
}
|
||||
}
|
||||
|
||||
// Handle global settings save (Logging, Storage)
|
||||
async function handleSave() {
|
||||
console.log("[SettingsPage][Action] Saving settings");
|
||||
@@ -372,14 +327,6 @@
|
||||
>
|
||||
{$t.settings?.llm || "LLM"}
|
||||
</button>
|
||||
<button
|
||||
class="px-4 py-2 text-sm font-medium transition-colors focus:outline-none {getTabClass(
|
||||
'migration',
|
||||
)}"
|
||||
on:click={() => handleTabChange("migration")}
|
||||
>
|
||||
Migration Sync
|
||||
</button>
|
||||
<button
|
||||
class="px-4 py-2 text-sm font-medium transition-colors focus:outline-none {getTabClass(
|
||||
'storage',
|
||||
@@ -765,8 +712,7 @@
|
||||
|
||||
<div class="mt-6 rounded-lg border border-gray-200 bg-white p-4">
|
||||
<h3 class="text-base font-semibold text-gray-900">
|
||||
{$t.settings?.llm_chatbot_settings_title ||
|
||||
"Chatbot Planner Settings"}
|
||||
{$t.settings?.llm_chatbot_settings_title || "Chatbot Planner Settings"}
|
||||
</h3>
|
||||
<p class="mt-1 text-sm text-gray-600">
|
||||
{$t.settings?.llm_chatbot_settings_description ||
|
||||
@@ -775,10 +721,7 @@
|
||||
|
||||
<div class="mt-4 grid grid-cols-1 gap-4 md:grid-cols-2">
|
||||
<div>
|
||||
<label
|
||||
for="planner-provider"
|
||||
class="block text-sm font-medium text-gray-700"
|
||||
>
|
||||
<label for="planner-provider" class="block text-sm font-medium text-gray-700">
|
||||
{$t.settings?.llm_chatbot_provider || "Chatbot Provider"}
|
||||
</label>
|
||||
<select
|
||||
@@ -786,9 +729,7 @@
|
||||
bind:value={settings.llm.assistant_planner_provider}
|
||||
class="mt-1 block w-full rounded-md border border-gray-300 p-2 text-sm"
|
||||
>
|
||||
<option value=""
|
||||
>{$t.dashboard?.use_default || "Use Default"}</option
|
||||
>
|
||||
<option value="">{$t.dashboard?.use_default || "Use Default"}</option>
|
||||
{#each settings.llm_providers || [] as provider}
|
||||
<option value={provider.id}>
|
||||
{provider.name} ({provider.default_model})
|
||||
@@ -798,18 +739,14 @@
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<label
|
||||
for="planner-model"
|
||||
class="block text-sm font-medium text-gray-700"
|
||||
>
|
||||
<label for="planner-model" class="block text-sm font-medium text-gray-700">
|
||||
{$t.settings?.llm_chatbot_model || "Chatbot Model Override"}
|
||||
</label>
|
||||
<input
|
||||
id="planner-model"
|
||||
type="text"
|
||||
bind:value={settings.llm.assistant_planner_model}
|
||||
placeholder={$t.settings?.llm_chatbot_model_placeholder ||
|
||||
"Optional, e.g. gpt-4.1-mini"}
|
||||
placeholder={$t.settings?.llm_chatbot_model_placeholder || "Optional, e.g. gpt-4.1-mini"}
|
||||
class="mt-1 block w-full rounded-md border border-gray-300 p-2 text-sm"
|
||||
/>
|
||||
</div>
|
||||
@@ -818,8 +755,7 @@
|
||||
|
||||
<div class="mt-6 rounded-lg border border-gray-200 bg-white p-4">
|
||||
<h3 class="text-base font-semibold text-gray-900">
|
||||
{$t.settings?.llm_provider_bindings_title ||
|
||||
"Provider Bindings by Task"}
|
||||
{$t.settings?.llm_provider_bindings_title || "Provider Bindings by Task"}
|
||||
</h3>
|
||||
<p class="mt-1 text-sm text-gray-600">
|
||||
{$t.settings?.llm_provider_bindings_description ||
|
||||
@@ -828,23 +764,15 @@
|
||||
|
||||
<div class="mt-4 grid grid-cols-1 gap-4 md:grid-cols-2">
|
||||
<div>
|
||||
<label
|
||||
for="binding-dashboard-validation"
|
||||
class="block text-sm font-medium text-gray-700"
|
||||
>
|
||||
{$t.settings?.llm_binding_dashboard_validation ||
|
||||
"Dashboard Validation Provider"}
|
||||
<label for="binding-dashboard-validation" class="block text-sm font-medium text-gray-700">
|
||||
{$t.settings?.llm_binding_dashboard_validation || "Dashboard Validation Provider"}
|
||||
</label>
|
||||
<select
|
||||
id="binding-dashboard-validation"
|
||||
bind:value={
|
||||
settings.llm.provider_bindings.dashboard_validation
|
||||
}
|
||||
bind:value={settings.llm.provider_bindings.dashboard_validation}
|
||||
class="mt-1 block w-full rounded-md border border-gray-300 p-2 text-sm"
|
||||
>
|
||||
<option value=""
|
||||
>{$t.dashboard?.use_default || "Use Default"}</option
|
||||
>
|
||||
<option value="">{$t.dashboard?.use_default || "Use Default"}</option>
|
||||
{#each settings.llm_providers || [] as provider}
|
||||
<option value={provider.id}>
|
||||
{provider.name} ({provider.default_model})
|
||||
@@ -860,21 +788,15 @@
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<label
|
||||
for="binding-documentation"
|
||||
class="block text-sm font-medium text-gray-700"
|
||||
>
|
||||
{$t.settings?.llm_binding_documentation ||
|
||||
"Documentation Provider"}
|
||||
<label for="binding-documentation" class="block text-sm font-medium text-gray-700">
|
||||
{$t.settings?.llm_binding_documentation || "Documentation Provider"}
|
||||
</label>
|
||||
<select
|
||||
id="binding-documentation"
|
||||
bind:value={settings.llm.provider_bindings.documentation}
|
||||
class="mt-1 block w-full rounded-md border border-gray-300 p-2 text-sm"
|
||||
>
|
||||
<option value=""
|
||||
>{$t.dashboard?.use_default || "Use Default"}</option
|
||||
>
|
||||
<option value="">{$t.dashboard?.use_default || "Use Default"}</option>
|
||||
{#each settings.llm_providers || [] as provider}
|
||||
<option value={provider.id}>
|
||||
{provider.name} ({provider.default_model})
|
||||
@@ -884,10 +806,7 @@
|
||||
</div>
|
||||
|
||||
<div class="md:col-span-2">
|
||||
<label
|
||||
for="binding-git-commit"
|
||||
class="block text-sm font-medium text-gray-700"
|
||||
>
|
||||
<label for="binding-git-commit" class="block text-sm font-medium text-gray-700">
|
||||
{$t.settings?.llm_binding_git_commit || "Git Commit Provider"}
|
||||
</label>
|
||||
<select
|
||||
@@ -895,9 +814,7 @@
|
||||
bind:value={settings.llm.provider_bindings.git_commit}
|
||||
class="mt-1 block w-full rounded-md border border-gray-300 p-2 text-sm"
|
||||
>
|
||||
<option value=""
|
||||
>{$t.dashboard?.use_default || "Use Default"}</option
|
||||
>
|
||||
<option value="">{$t.dashboard?.use_default || "Use Default"}</option>
|
||||
{#each settings.llm_providers || [] as provider}
|
||||
<option value={provider.id}>
|
||||
{provider.name} ({provider.default_model})
|
||||
@@ -923,8 +840,7 @@
|
||||
for="documentation-prompt"
|
||||
class="block text-sm font-medium text-gray-700"
|
||||
>
|
||||
{$t.settings?.llm_prompt_documentation ||
|
||||
"Documentation Prompt"}
|
||||
{$t.settings?.llm_prompt_documentation || "Documentation Prompt"}
|
||||
</label>
|
||||
<textarea
|
||||
id="documentation-prompt"
|
||||
@@ -976,150 +892,6 @@
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{:else if activeTab === "migration"}
|
||||
<!-- Migration Sync Tab -->
|
||||
<div class="text-lg font-medium mb-4">
|
||||
<h2 class="text-xl font-bold mb-4">
|
||||
Cross-Environment ID Synchronization
|
||||
</h2>
|
||||
<p class="text-gray-600 mb-6">
|
||||
Configure the background synchronization schedule and view the
|
||||
currently mapped Dashboard, Chart, and Dataset IDs.
|
||||
</p>
|
||||
|
||||
<!-- Cron Configuration -->
|
||||
<div class="bg-gray-50 p-6 rounded-lg border border-gray-200 mb-6">
|
||||
<h3 class="text-lg font-medium mb-4">Sync Schedule (Cron)</h3>
|
||||
<div class="flex items-end gap-4">
|
||||
<div class="flex-grow">
|
||||
<label
|
||||
for="migration_cron"
|
||||
class="block text-sm font-medium text-gray-700"
|
||||
>Cron Expression</label
|
||||
>
|
||||
<input
|
||||
type="text"
|
||||
id="migration_cron"
|
||||
bind:value={migrationCron}
|
||||
placeholder="0 2 * * *"
|
||||
class="mt-1 block w-full border border-gray-300 rounded-md shadow-sm p-2 font-mono text-sm"
|
||||
/>
|
||||
<p class="text-xs text-gray-500 mt-1">
|
||||
Example: 0 2 * * * (daily at 2 AM UTC)
|
||||
</p>
|
||||
</div>
|
||||
<button
|
||||
on:click={saveMigrationSettings}
|
||||
disabled={isSavingMigration}
|
||||
class="bg-blue-600 text-white px-4 py-2 rounded hover:bg-blue-700 h-[42px] min-w-[100px] flex items-center justify-center disabled:opacity-50"
|
||||
>
|
||||
{isSavingMigration ? "Saving..." : "Save"}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Mappings Table -->
|
||||
<div class="bg-gray-50 p-6 rounded-lg border border-gray-200">
|
||||
<h3
|
||||
class="text-lg font-medium mb-4 flex items-center justify-between"
|
||||
>
|
||||
<span>Synchronized Resources</span>
|
||||
<button
|
||||
on:click={loadMigrationSettings}
|
||||
class="text-sm text-indigo-600 hover:text-indigo-800 flex items-center gap-1"
|
||||
disabled={isLoadingMigration}
|
||||
>
|
||||
<svg
|
||||
class="w-4 h-4 {isLoadingMigration ? 'animate-spin' : ''}"
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
viewBox="0 0 24 24"
|
||||
><path
|
||||
stroke-linecap="round"
|
||||
stroke-linejoin="round"
|
||||
stroke-width="2"
|
||||
d="M4 4v5h.582m15.356 2A8.001 8.001 0 004.582 9m0 0H9m11 11v-5h-.581m0 0a8.003 8.003 0 01-15.357-2m15.357 2H15"
|
||||
></path></svg
|
||||
>
|
||||
Refresh
|
||||
</button>
|
||||
</h3>
|
||||
|
||||
<div class="overflow-x-auto border border-gray-200 rounded-lg">
|
||||
<table class="min-w-full divide-y divide-gray-200 text-sm">
|
||||
<thead class="bg-gray-100">
|
||||
<tr>
|
||||
<th
|
||||
class="px-6 py-3 text-left font-medium text-gray-500 uppercase tracking-wider"
|
||||
>Resource Name</th
|
||||
>
|
||||
<th
|
||||
class="px-6 py-3 text-left font-medium text-gray-500 uppercase tracking-wider"
|
||||
>Type</th
|
||||
>
|
||||
<th
|
||||
class="px-6 py-3 text-left font-medium text-gray-500 uppercase tracking-wider"
|
||||
>UUID</th
|
||||
>
|
||||
<th
|
||||
class="px-6 py-3 text-left font-medium text-gray-500 uppercase tracking-wider"
|
||||
>Target ID</th
|
||||
>
|
||||
<th
|
||||
class="px-6 py-3 text-left font-medium text-gray-500 uppercase tracking-wider"
|
||||
>Env</th
|
||||
>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody class="bg-white divide-y divide-gray-200">
|
||||
{#if isLoadingMigration && displayMappings.length === 0}
|
||||
<tr
|
||||
><td
|
||||
colspan="5"
|
||||
class="px-6 py-8 text-center text-gray-500"
|
||||
>Loading mappings...</td
|
||||
></tr
|
||||
>
|
||||
{:else if displayMappings.length === 0}
|
||||
<tr
|
||||
><td
|
||||
colspan="5"
|
||||
class="px-6 py-8 text-center text-gray-500"
|
||||
>No synchronized resources found.</td
|
||||
></tr
|
||||
>
|
||||
{:else}
|
||||
{#each displayMappings as mapping}
|
||||
<tr class="hover:bg-gray-50">
|
||||
<td
|
||||
class="px-6 py-4 whitespace-nowrap font-medium text-gray-900"
|
||||
>{mapping.resource_name || "N/A"}</td
|
||||
>
|
||||
<td class="px-6 py-4 whitespace-nowrap"
|
||||
><span
|
||||
class="px-2 inline-flex text-xs leading-5 font-semibold rounded-full bg-blue-100 text-blue-800"
|
||||
>{mapping.resource_type}</span
|
||||
></td
|
||||
>
|
||||
<td
|
||||
class="px-6 py-4 whitespace-nowrap font-mono text-xs text-gray-500"
|
||||
>{mapping.uuid}</td
|
||||
>
|
||||
<td
|
||||
class="px-6 py-4 whitespace-nowrap font-mono text-xs font-bold text-gray-700"
|
||||
>{mapping.remote_id}</td
|
||||
>
|
||||
<td class="px-6 py-4 whitespace-nowrap text-gray-500"
|
||||
>{mapping.environment_id}</td
|
||||
>
|
||||
</tr>
|
||||
{/each}
|
||||
{/if}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{:else if activeTab === "storage"}
|
||||
<!-- Storage Tab -->
|
||||
<div class="text-lg font-medium mb-4">
|
||||
|
||||
@@ -1,35 +0,0 @@
|
||||
# Specification Quality Checklist: Service ID Synchronization and Cross-Filter Recovery
|
||||
|
||||
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
||||
**Created**: 2026-02-25
|
||||
**Feature**: [022-sync-id-cross-filters spec](../spec.md)
|
||||
|
||||
## Content Quality
|
||||
|
||||
- [x] No implementation details (languages, frameworks, APIs)
|
||||
- [x] Focused on user value and business needs
|
||||
- [x] Written for non-technical stakeholders
|
||||
- [x] All mandatory sections completed
|
||||
|
||||
## Requirement Completeness
|
||||
|
||||
- [x] No [NEEDS CLARIFICATION] markers remain
|
||||
- [x] Requirements are testable and unambiguous
|
||||
- [x] Success criteria are measurable
|
||||
- [x] Success criteria are technology-agnostic (no implementation details)
|
||||
- [x] All acceptance scenarios are defined
|
||||
- [x] Edge cases are identified
|
||||
- [x] Scope is clearly bounded
|
||||
- [x] Dependencies and assumptions identified
|
||||
|
||||
## Feature Readiness
|
||||
|
||||
- [x] All functional requirements have clear acceptance criteria
|
||||
- [x] User scenarios cover primary flows
|
||||
- [x] Feature meets measurable outcomes defined in Success Criteria
|
||||
- [x] No implementation details leak into specification
|
||||
|
||||
## Notes
|
||||
|
||||
- Feature spec meets all standard checks. It correctly captures business goals (restoring cross-filters transparently) while avoiding deep assumptions on the Superset backend code, besides standard ZIP import/export payload patching.
|
||||
- Ready for `/speckit.plan` or `/speckit.tasks`.
|
||||
@@ -1,64 +0,0 @@
|
||||
# API Contracts
|
||||
|
||||
## 1. Migration Payload Additions
|
||||
|
||||
The migration execution API endpoint (e.g. `POST /api/v1/migration/run` or similar) needs to accept a new boolean flag.
|
||||
|
||||
```json
|
||||
{
|
||||
"source_environment_id": "env_abc",
|
||||
"target_environment_id": "env_xyz",
|
||||
"resource_uuids": ["1234-abcd-...", "5678-efgh-..."],
|
||||
"fix_cross_filters": true
|
||||
}
|
||||
```
|
||||
|
||||
## 2. Settings API Endpoints
|
||||
|
||||
The frontend requires REST endpoints to manage the service settings.
|
||||
|
||||
```json
|
||||
// GET /api/v1/migration/settings
|
||||
{
|
||||
"sync_cron_schedule": "0 * * * *" // Example: every hour
|
||||
}
|
||||
|
||||
// PUT /api/v1/migration/settings
|
||||
{
|
||||
"sync_cron_schedule": "*/30 * * * *"
|
||||
}
|
||||
|
||||
// GET /api/v1/migration/mappings
|
||||
// Returns paginated list of ResourceMapping records
|
||||
{
|
||||
"items": [
|
||||
{
|
||||
"environment_id": "prod-env-1",
|
||||
"environment_name": "Production",
|
||||
"resource_type": "chart",
|
||||
"uuid": "123e4567-e89b-12d3-a456-426614174000",
|
||||
"remote_integer_id": 42,
|
||||
"resource_name": "Monthly Active Users",
|
||||
"last_synced_at": "2026-02-25T10:00:00Z"
|
||||
}
|
||||
],
|
||||
"total": 1
|
||||
}
|
||||
```
|
||||
|
||||
## 3. IdMappingService Internal Interface
|
||||
|
||||
```python
|
||||
class IdMappingService:
|
||||
def sync_environment(self, environment_id: str) -> None:
|
||||
"""Fully synchronizes mapping for a specific environment."""
|
||||
pass
|
||||
|
||||
def get_remote_id(self, environment_id: str, resource_type: str, uuid: str) -> int | None:
|
||||
"""Retrieves the remote integer ID for a given universal UUID."""
|
||||
pass
|
||||
|
||||
def get_remote_ids_batch(self, environment_id: str, resource_type: str, uuids: list[str]) -> dict[str, int]:
|
||||
"""Retrieves remote integer IDs for a list of universal UUIDs."""
|
||||
pass
|
||||
```
|
||||
@@ -1,24 +0,0 @@
|
||||
# Phase 1 Data Model: Service ID Synchronization and Cross-Filter Recovery
|
||||
|
||||
## Entities
|
||||
|
||||
### `ResourceMapping`
|
||||
|
||||
Persists the discovered external integer IDs and maps them to universal UUIDs.
|
||||
|
||||
**Fields**:
|
||||
- `id` (Primary Key, integer)
|
||||
- `environment_id` (String, foreign key or reference to Environment configuration)
|
||||
- `resource_type` (Enum: `chart`, `dataset`, `dashboard`)
|
||||
- `uuid` (String, UUID format, unique per env + type)
|
||||
- `remote_integer_id` (Integer)
|
||||
- `resource_name` (String, human-readable name for UI display)
|
||||
- `last_synced_at` (DateTime, UTC)
|
||||
|
||||
**Constraints**:
|
||||
- Unique constraint on `(environment_id, resource_type, uuid)`
|
||||
- Unique constraint on `(environment_id, resource_type, remote_integer_id)` (Optional, but generally true in Superset)
|
||||
|
||||
## Validations
|
||||
- `remote_integer_id` must be > 0
|
||||
- `environment_id` must be an active, valid registered environment
|
||||
@@ -1,86 +0,0 @@
|
||||
# Implementation Plan: ID Synchronization and Cross-Filter Recovery
|
||||
|
||||
**Branch**: `022-sync-id-cross-filters` | **Date**: 2026-02-25 | **Spec**: [spec.md](./spec.md)
|
||||
**Input**: Feature specification from `/specs/022-sync-id-cross-filters/spec.md`
|
||||
|
||||
## Summary
|
||||
|
||||
This feature implements a mechanism to recover native cross-filtering links when migrating Apache Superset dashboards. It introduces an `IdMappingService` to track Superset's unstable `Integer IDs` across environments using stable `UUIDs`. During migration, a new `fix_cross_filters` option enables a workflow that intercepts the export ZIP, dynamically translates Source environment Integer IDs to Target environment Integer IDs in the Dashboard's `json_metadata`, and performs a dual-import sequence if the Chart objects don't yet exist on the target.
|
||||
|
||||
Additionally, it provides a Settings UI allowing administrators to configure the background synchronization interval via a Cron string and to inspect the internal mapping catalog, showing real names (`resource_name`) alongside their UUIDs and IDs across different environments.
|
||||
|
||||
## Technical Context
|
||||
|
||||
**Language/Version**: Python 3.11+
|
||||
**Primary Dependencies**: FastAPI, Pydantic, APScheduler, SQLAlchemy (or existing SQLite interface)
|
||||
**Storage**: Local `mappings.db` (SQLite)
|
||||
**Testing**: `pytest`
|
||||
**Target Platform**: Linux / Docker
|
||||
**Project Type**: Backend Web Service (`ss-tools` API)
|
||||
**Performance Goals**: ID translations must complete in < 5 seconds. Background sync must paginate smoothly to avoid overwhelming Target Superset APIs.
|
||||
**Constraints**: Requires admin API access to Superset instances.
|
||||
**Scale/Scope**: Supports hundreds of dashboards and charts per environment.
|
||||
|
||||
## Constitution Check
|
||||
|
||||
*GATE: Passed*
|
||||
|
||||
- **I. Library-First**: The `IdMappingService` will be strictly isolated as a core concept within `backend/src/core/mapping_service` (or similar). It does NOT depend on web-layer code.
|
||||
- **II. CLI Interface**: N/A for this internal daemon, though its behavior is exposed over the API.
|
||||
- **III. Test-First (NON-NEGOTIABLE)**: We will write targeted `pytest` unit tests for the ZIP manifest patching logic within `MigrationEngine` using the fixtures provided in the spec.
|
||||
- **IV. Integration Testing**: A mock Superset API environment will be required to integration test the "Dual-Import" capability safely.
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Documentation (this feature)
|
||||
|
||||
```text
|
||||
specs/022-sync-id-cross-filters/
|
||||
├── plan.md # This file
|
||||
├── research.md # Phase 0 output
|
||||
├── data-model.md # Phase 1 output
|
||||
├── quickstart.md # Phase 1 output
|
||||
├── contracts/ # Phase 1 output
|
||||
└── tasks.md # Phase 2 output (Pending)
|
||||
```
|
||||
|
||||
### Source Code
|
||||
|
||||
```text
|
||||
backend/
|
||||
├── src/
|
||||
│ ├── core/
|
||||
│ │ ├── migration_engine.py # (MODIFIED) Add YAML patch layer
|
||||
│ │ └── mapping_service.py # (NEW) IdMappingService + scheduling
|
||||
│ ├── models/
|
||||
│ │ └── mappings.py # (NEW) SQLAlchemy/Pydantic models for ResourceMapping
|
||||
│ └── api/
|
||||
│ └── routes/
|
||||
│ ├── migration.py # (MODIFIED) Add fix_cross_filters config flag
|
||||
│ └── settings.py # (MODIFIED/NEW) Add cron config and mapping GET endpoints
|
||||
└── tests/
|
||||
└── core/
|
||||
├── test_migration_engine.py # (NEW/MODIFIED)
|
||||
└── test_mapping_service.py # (NEW)
|
||||
|
||||
frontend/
|
||||
├── src/
|
||||
│ ├── routes/
|
||||
│ │ └── settings/ # (MODIFIED) Add UI for cron and mapping table
|
||||
│ └── components/
|
||||
│ └── mappings/
|
||||
│ └── MappingTable.svelte # (NEW) Table to view current UUID <-> ID catalogs
|
||||
```
|
||||
|
||||
**Structure Decision**: Extending the existing Backend FastApi structure. The logic touches API routing, the core `MigrationEngine`, and introduces a new standalone `mapping_service.py` to handle the scheduler and DB layer.
|
||||
|
||||
## Complexity Tracking
|
||||
|
||||
*No major constitutional violations.* The only added complexity is the "Dual-Import" mechanism in `MigrationEngine`, which is strictly necessary because Superset assigns IDs unconditionally on creation.
|
||||
|
||||
## Test Data Reference
|
||||
|
||||
| Component | TIER | Fixture Name | Location |
|
||||
|-----------|------|--------------|----------|
|
||||
| `IdMappingService` | CRITICAL | `resource_mapping_record` | spec.md#test-data-fixtures |
|
||||
| `MigrationEngine` | CRITICAL | `missing_target_scenario` | spec.md#test-data-fixtures |
|
||||
@@ -1,19 +0,0 @@
|
||||
# Quickstart: ID Synchronization
|
||||
|
||||
## Getting Started
|
||||
|
||||
1. **Service Initialization**:
|
||||
Ensure the `IdMappingService` scheduler is started when the backend application boots up.
|
||||
|
||||
```python
|
||||
# In app.py or similar startup logic
|
||||
from core.mapping_service import IdMappingService
|
||||
|
||||
mapping_service = IdMappingService()
|
||||
mapping_service.start_scheduler(interval_minutes=30)
|
||||
```
|
||||
|
||||
2. **Running Migrations with Cross-Filter Sync**:
|
||||
When calling the migration endpoint, set `fix_cross_filters=True`. The engine will automatically pause, do a technical import, sync IDs, patch the manifest, and do a final import.
|
||||
|
||||
*(No additional quickstart steps required as this logic is embedded directly into the migration engine).*
|
||||
@@ -1,39 +0,0 @@
|
||||
# Phase 0 Research: Service ID Synchronization and Cross-Filter Recovery
|
||||
|
||||
## 1. Technical Context Clarifications
|
||||
|
||||
- **Language/Version**: Python 3.11+
|
||||
- **Primary Dependencies**: FastAPI, SQLite (local mapping DB), APScheduler (background jobs)
|
||||
- **Storage**: SQLite (mappings.db) or similar existing local store / ORM for storing `resource_mappings`.
|
||||
- **Testing**: pytest (backend)
|
||||
- **Target Platform**: Dockerized Linux service
|
||||
|
||||
## 2. Research Tasks & Unknowns
|
||||
|
||||
### Q: Where to store the environment mappings?
|
||||
- **Decision**: A new SQLite table `resource_mappings` inside the existing backend database (or standard SQLAlchemy models if used).
|
||||
- **Rationale**: Minimal infrastructure overhead. We already use an internal database (implied by `mappings.db` or SQLAlchemy models).
|
||||
- **Alternatives considered**: Redis (too volatile, don't want to lose mapping on restart), separate PostgreSQL (overkill).
|
||||
|
||||
### Q: How to schedule the synchronization background job?
|
||||
- **Decision**: Use `APScheduler` or FastAPI's `BackgroundTasks` triggered by a simple internal mechanism. If `APScheduler` is already in the project, we'll use that.
|
||||
- **Rationale**: Standard Python ecosystem approach for periodic in-process background tasks.
|
||||
- **Alternatives considered**: Celery (too heavy), cron container (requires external setup).
|
||||
|
||||
### Q: How to intercept the migration payload (ZIP) and patch it?
|
||||
- **Decision**: In `MigrationEngine.migrate()`, before uploading to the Target Environment via `api_client`, intercept the downloaded `ZIP` bytes. Extract `dashboards/*.yaml`, parse YAML, find and replace `id` references in `json_metadata`, write back to YAML, repack the ZIP, and dispatch.
|
||||
- **Rationale**: Dashboards export as ZIP files containing YAML manifests. `json_metadata` in these YAMLs contains the `chart_configuration` we need to patch.
|
||||
- **Alternatives considered**: Patching via Superset API after import (harder, requires precise API calls and dealing with Superset's strict dashboard validation).
|
||||
|
||||
### Q: How to handle the "Double Import" strategy?
|
||||
- **Decision**: If Target Check shows missing UUIDs, execute temporary import, trigger direct `IdMappingService.sync_environment(target_env)` to fetch newly created IDs, then proceed with the patching and final import with `overwrite=true`.
|
||||
- **Rationale**: Superset generates IDs upon creation. We cannot guess them. We must force it to create them first.
|
||||
- **Alternatives considered**: Creating charts via API manually one-by-one (loss of complex export logic, reinventing the importer).
|
||||
|
||||
### Q: Integrating the Checkbox into UI
|
||||
- **Decision**: Pass `fix_cross_filters=True` boolean in the migration payload from Frontend to Backend.
|
||||
- **Rationale**: Standard API contract extension.
|
||||
|
||||
## 3. Existing Code Integration
|
||||
|
||||
We need to hook into `backend/src/core/migration_engine.py`. This file likely handles the `export -> import` flow. We will add a pre-processing step right before the final `api_client.import_dashboards()` call on the target environment.
|
||||
@@ -1,120 +0,0 @@
|
||||
# Feature Specification: ID Synchronization and Cross-Filter Recovery
|
||||
|
||||
**Feature Branch**: `022-sync-id-cross-filters`
|
||||
**Reference UX**: N/A
|
||||
**Created**: 2026-02-25
|
||||
**Status**: Draft
|
||||
**Input**: User description: "Техническое задание: Сервис синхронизации ID и восстановления кросс-фильтрации..."
|
||||
|
||||
## User Scenarios & Testing *(mandatory)*
|
||||
|
||||
### User Story 1 - Transparent Migration with Working Cross-Filters (Priority: P1)
|
||||
|
||||
As a user migrating a dashboard equipped with cross-filters, I want the system to automatically remap internal Chart and Dataset IDs to the target environment's IDs, so that cross-filtering works immediately after migration without manual reconstruction.
|
||||
|
||||
**Why this priority**: Core value proposition. Superset breaks cross-filter links during migration because integer IDs change; fixing this is the primary goal of this feature.
|
||||
|
||||
**Independent Test**: Can be fully tested by migrating a dashboard containing native cross-filters to a blank environment with "Fix Cross-Filters" checked, and verifying filters function natively on the target without opening them in edit mode.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** a dashboard with cross-filters pointing to Source IDs, **When** migrated to a Target environment where charts are new (UUIDs missing), **Then** the system transparently does a dual-import, fetches new IDs, patches the metadata, and imports again so the resulting dashboard uses correct Target IDs.
|
||||
2. **Given** a dashboard with cross-filters where charts already exist on the Target, **When** migrated, **Then** the system looks up IDs locally and patches the dashboard metadata directly without an intermediary import.
|
||||
|
||||
---
|
||||
|
||||
### User Story 2 - UI Option for Cross-Filter Recovery (Priority: P2)
|
||||
|
||||
As a user initiating a migration, I want a clear option to toggle cross-filter repair ("Исправить связи кросс-фильтрации"), so I can bypass the double-import logic if I know cross-filters aren't used or if I require a strict, unmodified file import.
|
||||
|
||||
**Why this priority**: Gives users control over the migration pipeline and reduces target system load if patching isn't needed.
|
||||
|
||||
**Independent Test**: Can be tested by opening the migration modal/interface and verifying the checkbox state, label, and help description are correct.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** the migration launch UI, **When** it opens, **Then** the "Исправить связи кросс-фильтрации" is visible, enabled by default, and has a descriptive tooltip.
|
||||
|
||||
---
|
||||
|
||||
### User Story 3 - Settings UI and Mapping Table (Priority: P2)
|
||||
|
||||
As a system administrator, I want to configure the synchronization schedule via Cron and view the current mapping table in the Settings UI so that I can monitor the agent's status and debug missing IDs.
|
||||
|
||||
**Why this priority**: Required for operational visibility and configuration flexibility.
|
||||
|
||||
**Independent Test**: Open the settings page, adjust the Cron schedule, and view the mapping table. Verify the table displays environment names, UUIDs, integer IDs, and resource names.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
1. **Given** the Settings UI, **When** navigated to the ID Mapping section, **Then** I see an input to configure the Cron string for the background job.
|
||||
2. **Given** the ID Mapping section, **When** viewed, **Then** I see a table or list showing `Environment`, `UUID`, `Integer ID`, and `Resource Name` for all synchronized objects.
|
||||
|
||||
---
|
||||
|
||||
### Edge Cases
|
||||
|
||||
- What happens when the background sync process fails to reach a specific Environment API (e.g., due to network issues or authentication)?
|
||||
- How does the system handle migrating dashboards where some UUIDs map properly while others don't (partial existence footprint)?
|
||||
- What happens if the intermediary 'Technical Import' step fails (e.g., due to duplicate names or target environment constraints)?
|
||||
|
||||
## Requirements *(mandatory)*
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
- **FR-001**: System MUST maintain a scheduled background synchronization process to poll all active Environments, configurable via a Cron expression in the UI.
|
||||
- **FR-002**: System MUST catalog Chart, Dataset, and Dashboard objects from each Environment, mapping their universal UUIDs to environment-specific IDs and capturing their human-readable names.
|
||||
- **FR-003**: System MUST persist this identity catalog locally for rapid access.
|
||||
- **FR-004**: System MUST provide a pre-migration verification mechanism to determine if required Target IDs are already known in the local catalog.
|
||||
- **FR-005**: System MUST perform an intermediary "Technical Import" and immediate re-synchronization when migrating objects to a fresh environment, effectively forcing the Target system to generate and reveal new IDs.
|
||||
- **FR-006**: System MUST patch the dashboard configuration metadata before the final import step, ensuring all cross-filter associations exclusively use the correct Target IDs.
|
||||
- **FR-007**: System MUST provide a user interface option to explicitly toggle the Cross-Filter recovery feature during migration execution.
|
||||
- **FR-008**: System MUST provide a Settings UI view displaying the universal UUID, the environment-specific integer ID, the environment name, and the resource name for all cataloged objects.
|
||||
|
||||
### Key Entities
|
||||
|
||||
- **Resource Mapping**: Represents the unified reference between an Environment, a Resource Type (Chart/Dataset/Dashboard), its universal UUID, its environment-specific Integer ID, and its last sync timestamp.
|
||||
|
||||
## Success Criteria *(mandatory)*
|
||||
|
||||
### Measurable Outcomes
|
||||
|
||||
- **SC-001**: 100% of native cross-filters on migrated dashboards continue to function perfectly on the Target environment without manual user intervention.
|
||||
- **SC-002**: Background sync completes across a minimum of 3 standard environments in under 5 minutes without causing API rate-limiting.
|
||||
- **SC-003**: A single "Fix Cross-Filters" migration takes no more than 2x the execution time of a standard migration.
|
||||
|
||||
---
|
||||
|
||||
## Test Data Fixtures *(recommended for CRITICAL components)*
|
||||
|
||||
### Fixtures
|
||||
|
||||
```yaml
|
||||
resource_mapping_record:
|
||||
description: "Example state of the mapping database after a successful sync step"
|
||||
data:
|
||||
environment_id: "prod-env-1"
|
||||
resource_type: "chart"
|
||||
uuid: "123e4567-e89b-12d3-a456-426614174000"
|
||||
remote_integer_id: 42
|
||||
last_synced_at: "2026-02-25T10:00:00Z"
|
||||
# UI settings table and Cron schedule requirements
|
||||
# This section describes the data structure for resource mapping and sync process
|
||||
# It's not a fixture itself, but rather a description of the data collected.
|
||||
# For UI settings and Cron, separate fixtures or requirements would be needed.
|
||||
# The following is a description of the data points collected during sync:
|
||||
# 1. Извлечение (Extraction):
|
||||
# * **Charts:** `id`, `uuid`, `slice_name`
|
||||
# * **Datasets:** `id`, `uuid`, `table_name`
|
||||
# * **Dashboards:** `id`, `uuid`, `slug`
|
||||
# 2. Сохранение (Upsert): Сохраняет данные в локальную БД сервиса миграции в таблицу маппинга.
|
||||
# * *Ключ уникальности:* `environment_id` + `uuid`.
|
||||
# * *Обновляемое поле:* `remote_integer_id` (актуальный ID на конкретном стенде) и `resource_name` (для отображения в UI).
|
||||
# * *Метаданные:* `last_synced_at`.
|
||||
|
||||
missing_target_scenario:
|
||||
description: "Target check indicates partial or entirely missing IDs"
|
||||
data:
|
||||
all_uuids_present: false
|
||||
missing_uuids:
|
||||
- "123e4567-e89b-12d3-a456-426614174000"
|
||||
```
|
||||
@@ -1,105 +0,0 @@
|
||||
# Tasks: ID Synchronization and Cross-Filter Recovery
|
||||
|
||||
**Input**: Design documents from `/specs/022-sync-id-cross-filters/`
|
||||
**Prerequisites**: plan.md, spec.md, research.md, data-model.md, contracts/api.md
|
||||
|
||||
## Format: `[ID] [P?] [Story] Description`
|
||||
- **[P]**: Can run in parallel (different files, no dependencies)
|
||||
- **[Story]**: Which user story this task belongs to (e.g., US1, US2)
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Setup (Shared Infrastructure)
|
||||
|
||||
**Purpose**: Project initialization and basic structure.
|
||||
|
||||
- [x] T001 Initialize database/ORM mapping models structure for the new service in `backend/src/models/mappings.py`
|
||||
- [x] T002 Configure APScheduler scaffolding in `backend/src/core/mapping_service.py`
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Foundational (Blocking Prerequisites)
|
||||
|
||||
**Purpose**: Core logic for the `IdMappingService` that MUST be complete before we can use it in migrations.
|
||||
|
||||
- [x] T003 Create `ResourceMapping` SQLAlchemy or SQLite model in `backend/src/models/mappings.py` (Include `resource_name` field)
|
||||
- [x] T004 Implement `IdMappingService.sync_environment(env_id)` logic in `backend/src/core/mapping_service.py`
|
||||
- [x] T005 Implement `IdMappingService.get_remote_ids_batch(...)` logic in `backend/src/core/mapping_service.py`
|
||||
- [x] T006 Write unit tests for `IdMappingService` in `backend/tests/core/test_mapping_service.py` using `resource_mapping_record` fixture.
|
||||
|
||||
**Checkpoint**: Foundation ready - ID cataloging works independently.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: User Story 1 - Transparent Migration with Working Cross-Filters (Priority: P1) 🎯 MVP
|
||||
|
||||
**Goal**: As a user migrating a dashboard equipped with cross-filters, I want the system to automatically remap internal Chart and Dataset IDs to the target environment's IDs.
|
||||
|
||||
### Tests for User Story 1 ⚠️
|
||||
- [x] T007 [P] [US1] Create unit tests for YAML patching in `backend/tests/core/test_migration_engine.py` using `missing_target_scenario` fixture.
|
||||
|
||||
### Implementation for User Story 1
|
||||
- [x] T008 [US1] Inject `IdMappingService` dependency into `MigrationEngine` in `backend/src/core/migration_engine.py`
|
||||
- [x] T009 [US1] Implement `MigrationEngine._patch_dashboard_metadata(...)` to rewrite integer IDs in `json_metadata`.
|
||||
- [x] T010 [US1] Modify `MigrationEngine` to perform "Target Check" against IDs.
|
||||
- [x] T011 [US1] Modify `MigrationEngine` to execute "Dual-Import" sequence if Target Check finds missing IDs.
|
||||
|
||||
**Checkpoint**: At this point, User Story 1 should be fully functional and testable via backend scripts/API calls.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: User Story 2 - UI Option for Cross-Filter Recovery (Priority: P2)
|
||||
|
||||
**Goal**: As a user initiating a migration, I want a clear option to toggle cross-filter repair.
|
||||
|
||||
### Implementation for User Story 2
|
||||
- [x] T012 [US2] Modify `backend/src/api/routes/migration.py` and `DashboardSelection` to accept `fix_cross_filters` boolean (default: True).
|
||||
- [x] T013 [US2] Modify frontend `migration/+page.svelte` to include "Исправить связи кросс-фильтрации" checkbox and pass it in API POST payload.
|
||||
|
||||
**Checkpoint**: User controls whether cross-filter patching occurs.
|
||||
- [x] T014 [US2] Add checkbox to the Migration launch modal in the Frontend component (assuming `frontend/src/components/MigrationModal.svelte` or similar).
|
||||
|
||||
**Checkpoint**: At this point, User Story 1 and 2 should both work end-to-end.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: User Story 3 - Settings UI and Mapping Table (Priority: P2)
|
||||
|
||||
**Goal**: As a system administrator, I want to configure the synchronization schedule via Cron and view the current mapping table in the Settings UI.
|
||||
|
||||
### Implementation for User Story 3
|
||||
- [x] T015 [P] [US3] Implement `GET /api/v1/migration/settings` and `PUT /api/v1/migration/settings` for Cron in `backend/src/api/routes/migration.py`
|
||||
- [x] T016 [P] [US3] Implement `GET /api/v1/migration/mappings-data` endpoint for the table in `backend/src/api/routes/migration.py`
|
||||
- [x] T017 [US3] Create Mapping UI directly in Frontend `frontend/src/routes/settings/+page.svelte`
|
||||
- [x] T018 [US3] Add Cron configuration input and Mapping Table to the Settings page in `frontend/src/routes/settings/+page.svelte`
|
||||
- [x] T019 [US3] Ensure `IdMappingService` scheduler can be restarted/reconfigured.
|
||||
|
||||
**Checkpoint**: At this point, User Story 3 should be fully functional and the background sync interval can be tuned visually.
|
||||
|
||||
---
|
||||
|
||||
## Phase N: Polish & Cross-Cutting Concerns
|
||||
|
||||
**Purpose**: Improvements that affect multiple user stories
|
||||
|
||||
- [ ] T020 Verify error handling if "Technical Import" step fails.
|
||||
- [ ] T021 Add debug logging using Molecular Topology (`[EXPLORE]`, `[REASON]`, `[REFLECT]`) to the mapping and patching processes.
|
||||
|
||||
---
|
||||
|
||||
## Dependencies & Execution Order
|
||||
|
||||
### Phase Dependencies
|
||||
- **Setup (Phase 1)**: No dependencies - can start immediately
|
||||
- **Foundational (Phase 2)**: Depends on Setup completion - BLOCKS all user stories
|
||||
- **User Stories (Phase 3+)**: All depend on Foundational phase completion
|
||||
- **Polish (Final Phase)**: Depends on all desired user stories being complete
|
||||
|
||||
### User Story Dependencies
|
||||
- **User Story 1 (P1)**: Can start after Foundational (Phase 2)
|
||||
- **User Story 2 (P2)**: Can run in parallel with US1 backend work.
|
||||
- **User Story 3 (P3)**: Depends on Foundational (Phase 2), specifically the `ResourceMapping` table being populated, but can run parallel to US1/US2.
|
||||
|
||||
### Parallel Opportunities
|
||||
- Frontend UI work (T014, T017, T018) can be done in parallel with backend engine work (T008-T011, T015-T016)
|
||||
- API route updates can be done in parallel.
|
||||
Reference in New Issue
Block a user