38 lines
2.6 KiB
Markdown
38 lines
2.6 KiB
Markdown
# Quickstart: Dashboard Health Windows
|
|
|
|
## Overview
|
|
This feature introduces "Execution Windows" for automated LLM dashboard validations, allowing administrators to schedule checks over a period of time (e.g., 1am-5am) rather than at a single exact minute. It also adds a centralized "Health Center" UI to easily monitor which dashboards are currently failing their checks.
|
|
|
|
## Implementation Steps
|
|
|
|
1. **Backend Database Updates**:
|
|
- Create `ValidationPolicy` SQLAlchemy model (in `backend/src/models/llm.py` or new file).
|
|
- Generate Alembic migration for the new table.
|
|
|
|
2. **Backend API Routes**:
|
|
- Create CRUD routes for `ValidationPolicy` (e.g., `backend/src/api/routes/settings.py` or new `automation.py`).
|
|
- Enhance `backend/src/api/routes/dashboards.py` (or create `health.py`) to provide the aggregated health projection (joining Superset dashboard list with the latest `ValidationRecord`).
|
|
|
|
3. **Scheduler Integration**:
|
|
- Implement `ThrottledSchedulerConfigurator` logic inside or alongside `backend/src/core/scheduler.py`.
|
|
- Ensure it can parse active `ValidationPolicy` rows and translate them into distinct `apscheduler` triggers spaced evenly across the `[window_start, window_end]` interval.
|
|
|
|
4. **Frontend Settings (Automation)**:
|
|
- Add a new tab/page under Settings for "Automation Policies".
|
|
- Implement list view and Create/Edit modal for policies.
|
|
- Include intuitive time pickers for the Execution Window.
|
|
|
|
5. **Frontend Health Center**:
|
|
- Create `/dashboards/health` route in SvelteKit.
|
|
- Implement the `HealthMatrix` traffic light summary component.
|
|
- Build the data table to display the dashboard names, status badges, and issue summaries.
|
|
|
|
6. **UI Integration**:
|
|
- Add a derived store to `frontend/src/lib/stores/sidebar.js` (or similar) to fetch/calculate the `[🔴 N]` badge for the Dashboards navigation item.
|
|
- Update `backend/src/api/routes/assistant.py` so the LLM assistant can answer questions about "failing dashboards" and provide deep links to the new Health Center.
|
|
|
|
## Testing Strategy
|
|
- **Backend Unit Tests**: Verify the math of `ThrottledSchedulerConfigurator` (e.g., 60 tasks in 60 minutes = 1 minute intervals). Test boundary conditions (100 tasks in 1 minute).
|
|
- **Backend Route Tests**: Verify the API correctly aggregates the *latest* validation record for the Health Center, ignoring older passing/failing records.
|
|
- **Frontend Unit Tests**: Test `HealthMatrix` rendering with different combinations of Pass/Warn/Fail counts.
|
|
- **E2E/Integration**: Create a policy, wait for it to schedule, run a mock validation, and verify it appears in the Health Center. |