Compare commits
53 Commits
013-unify-
...
cc244c2d86
| Author | SHA1 | Date | |
|---|---|---|---|
| cc244c2d86 | |||
| d10c23e658 | |||
| 1042b35d1b | |||
| 16ffeb1ed6 | |||
| da34deac02 | |||
| 51e9ee3fcc | |||
| edf9286071 | |||
| a542e7d2df | |||
| a863807cf2 | |||
| e2bc68683f | |||
| 43cb82697b | |||
| 4ba28cf93e | |||
| 343f2e29f5 | |||
| c9a53578fd | |||
| 07ec2d9797 | |||
| e9d3f3c827 | |||
| 26ba015b75 | |||
| 49129d3e86 | |||
| d99a13d91f | |||
| 203ce446f4 | |||
| c96d50a3f4 | |||
| 3bbe320949 | |||
| 2d2435642d | |||
| ec8d67c956 | |||
| 76baeb1038 | |||
| 11c59fb420 | |||
| b2529973eb | |||
| ae1d630ad6 | |||
| 9a9c5879e6 | |||
| 696aac32e7 | |||
| 7a9b1a190a | |||
| a3dc1fb2b9 | |||
| 297b29986d | |||
| 4c6fc8256d | |||
| a747a163c8 | |||
| fce0941e98 | |||
| 45c077b928 | |||
| 9ed3a5992d | |||
| a032fe8457 | |||
| 4c9d554432 | |||
| 6962a78112 | |||
| 3d75a21127 | |||
| 07914c8728 | |||
| cddc259b76 | |||
| dcbf0a7d7f | |||
| 65f61c1f80 | |||
| cb7386f274 | |||
| 83e34e1799 | |||
| d197303b9f | |||
| a43f8fb021 | |||
| 4aa01b6470 | |||
| 35b423979d | |||
| 2ffc3cc68f |
5
.gitignore
vendored
5
.gitignore
vendored
@@ -66,7 +66,4 @@ backend/mappings.db
|
||||
|
||||
|
||||
backend/tasks.db
|
||||
|
||||
# Git Integration repositories
|
||||
backend/git_repos/
|
||||
backend/backend/git_repos
|
||||
backend/logs
|
||||
|
||||
@@ -27,6 +27,11 @@ Auto-generated from all feature plans. Last updated: 2025-12-19
|
||||
- SQLite (for config/history), Filesystem (local Git repositories) (011-git-integration-dashboard)
|
||||
- Node.js 18+ (Frontend Build), Svelte 5.x + SvelteKit, Tailwind CSS, `date-fns` (existing) (013-unify-frontend-css)
|
||||
- LocalStorage (for language preference) (013-unify-frontend-css)
|
||||
- Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI (Backend), SvelteKit (Frontend) (014-file-storage-ui)
|
||||
- Local Filesystem (for artifacts), Config (for storage path) (014-file-storage-ui)
|
||||
- Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI (Backend), SvelteKit + Tailwind CSS (Frontend) (015-frontend-nav-redesign)
|
||||
- N/A (UI reorganization and API integration) (015-frontend-nav-redesign)
|
||||
- SQLite (`auth.db`) for Users, Roles, Permissions, and Mappings. (016-multi-user-auth)
|
||||
|
||||
- Python 3.9+ (Backend), Node.js 18+ (Frontend Build) (001-plugin-arch-svelte-ui)
|
||||
|
||||
@@ -47,9 +52,9 @@ cd src; pytest; ruff check .
|
||||
Python 3.9+ (Backend), Node.js 18+ (Frontend Build): Follow standard conventions
|
||||
|
||||
## Recent Changes
|
||||
- 013-unify-frontend-css: Added Node.js 18+ (Frontend Build), Svelte 5.x + SvelteKit, Tailwind CSS, `date-fns` (existing)
|
||||
- 011-git-integration-dashboard: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI, SvelteKit, GitPython (or CLI git), Pydantic, SQLAlchemy, Superset API
|
||||
- 011-git-integration-dashboard: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI, SvelteKit, GitPython (or CLI git), Pydantic, SQLAlchemy, Superset API
|
||||
- 016-multi-user-auth: Added Python 3.9+ (Backend), Node.js 18+ (Frontend)
|
||||
- 015-frontend-nav-redesign: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI (Backend), SvelteKit + Tailwind CSS (Frontend)
|
||||
- 014-file-storage-ui: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI (Backend), SvelteKit (Frontend)
|
||||
|
||||
|
||||
<!-- MANUAL ADDITIONS START -->
|
||||
|
||||
@@ -1,11 +1,10 @@
|
||||
<!--
|
||||
SYNC IMPACT REPORT
|
||||
Version: 1.7.1 (Simplified Workflow)
|
||||
Version: 1.8.0 (Frontend Unification)
|
||||
Changes:
|
||||
- Simplified Generation Workflow to a single phase: Code Generation from `tasks.md`.
|
||||
- Removed multi-phase Architecture/Implementation split to streamline development.
|
||||
- Added Principle VIII: Unified Frontend Experience (Mandating Design System & i18n).
|
||||
Templates Status:
|
||||
- .specify/templates/plan-template.md: ✅ Aligned (Dynamic check).
|
||||
- .specify/templates/plan-template.md: ✅ Aligned.
|
||||
- .specify/templates/spec-template.md: ✅ Aligned.
|
||||
- .specify/templates/tasks-template.md: ✅ Aligned.
|
||||
-->
|
||||
@@ -37,6 +36,11 @@ To maintain semantic coherence, code must adhere to the complexity limits (Modul
|
||||
### VII. Everything is a Plugin
|
||||
All functional extensions, tools, or major features must be implemented as modular Plugins inheriting from `PluginBase`. Logic should not reside in standalone services or scripts unless strictly necessary for core infrastructure. This ensures a unified execution model via the `TaskManager`, consistent logging, and modularity.
|
||||
|
||||
### VIII. Unified Frontend Experience
|
||||
To ensure a consistent and accessible user experience, all frontend implementations must strictly adhere to the unified design and localization standards.
|
||||
- **Component Reusability**: All UI elements MUST utilize the standardized Svelte component library (`src/lib/ui`) and centralized design tokens. Ad-hoc styling and hardcoded values are prohibited.
|
||||
- **Internationalization (i18n)**: All user-facing text MUST be extracted to the translation system (`src/lib/i18n`). Hardcoded strings in the UI are prohibited.
|
||||
|
||||
## File Structure Standards
|
||||
Refer to **Section III (File Structure Standard)** in `semantic_protocol.md` for the authoritative definitions of:
|
||||
- Python Module Headers (`.py`)
|
||||
@@ -64,4 +68,4 @@ This Constitution establishes the "Semantic Code Generation Protocol" as the sup
|
||||
- **Amendments**: Changes to core principles require a Constitution amendment. Changes to technical syntax require a Protocol update.
|
||||
- **Compliance**: Failure to adhere to the Protocol constitutes a build failure.
|
||||
|
||||
**Version**: 1.7.1 | **Ratified**: 2025-12-19 | **Last Amended**: 2026-01-13
|
||||
**Version**: 1.8.0 | **Ratified**: 2025-12-19 | **Last Amended**: 2026-01-26
|
||||
|
||||
Submodule backend/backend/git_repos/12 updated: d592fa7ed5...f46772443a
@@ -1,269 +0,0 @@
|
||||
2025-12-20 19:55:11,325 - INFO - [BackupPlugin][Entry] Starting backup for superset.
|
||||
2025-12-20 19:55:11,325 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
|
||||
2025-12-20 19:55:11,327 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
Traceback (most recent call last):
|
||||
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 43, in setup_clients
|
||||
config = SupersetConfig(
|
||||
^^^^^^^^^^^^^^^
|
||||
File "/home/user/ss-tools/backend/venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
|
||||
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
2025-12-20 21:01:49,905 - INFO - [BackupPlugin][Entry] Starting backup for superset.
|
||||
2025-12-20 21:01:49,906 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
|
||||
2025-12-20 21:01:49,988 - INFO - [setup_clients][Action] Loading environments from ConfigManager
|
||||
2025-12-20 21:01:49,990 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
Traceback (most recent call last):
|
||||
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
|
||||
config = SupersetConfig(
|
||||
^^^^^^^^^^^^^^^
|
||||
File "/home/user/ss-tools/venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
|
||||
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
2025-12-20 22:42:32,538 - INFO - [BackupPlugin][Entry] Starting backup for superset.
|
||||
2025-12-20 22:42:32,538 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
|
||||
2025-12-20 22:42:32,583 - INFO - [setup_clients][Action] Loading environments from ConfigManager
|
||||
2025-12-20 22:42:32,587 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
Traceback (most recent call last):
|
||||
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
|
||||
config = SupersetConfig(
|
||||
^^^^^^^^^^^^^^^
|
||||
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
|
||||
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
2025-12-20 22:54:29,770 - INFO - [BackupPlugin][Entry] Starting backup for .
|
||||
2025-12-20 22:54:29,771 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
|
||||
2025-12-20 22:54:29,831 - INFO - [setup_clients][Action] Loading environments from ConfigManager
|
||||
2025-12-20 22:54:29,833 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
Traceback (most recent call last):
|
||||
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
|
||||
config = SupersetConfig(
|
||||
^^^^^^^^^^^^^^^
|
||||
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
|
||||
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
2025-12-20 22:54:34,078 - INFO - [BackupPlugin][Entry] Starting backup for superset.
|
||||
2025-12-20 22:54:34,078 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
|
||||
2025-12-20 22:54:34,079 - INFO - [setup_clients][Action] Loading environments from ConfigManager
|
||||
2025-12-20 22:54:34,079 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
Traceback (most recent call last):
|
||||
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
|
||||
config = SupersetConfig(
|
||||
^^^^^^^^^^^^^^^
|
||||
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
|
||||
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
2025-12-20 22:59:25,060 - INFO - [BackupPlugin][Entry] Starting backup for superset.
|
||||
2025-12-20 22:59:25,060 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
|
||||
2025-12-20 22:59:25,114 - INFO - [setup_clients][Action] Loading environments from ConfigManager
|
||||
2025-12-20 22:59:25,117 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
Traceback (most recent call last):
|
||||
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
|
||||
config = SupersetConfig(
|
||||
^^^^^^^^^^^^^^^
|
||||
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
|
||||
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
2025-12-20 23:00:31,156 - INFO - [BackupPlugin][Entry] Starting backup for superset.
|
||||
2025-12-20 23:00:31,156 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
|
||||
2025-12-20 23:00:31,157 - INFO - [setup_clients][Action] Loading environments from ConfigManager
|
||||
2025-12-20 23:00:31,162 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
Traceback (most recent call last):
|
||||
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
|
||||
config = SupersetConfig(
|
||||
^^^^^^^^^^^^^^^
|
||||
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
|
||||
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
2025-12-20 23:00:34,710 - INFO - [BackupPlugin][Entry] Starting backup for superset.
|
||||
2025-12-20 23:00:34,710 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
|
||||
2025-12-20 23:00:34,710 - INFO - [setup_clients][Action] Loading environments from ConfigManager
|
||||
2025-12-20 23:00:34,711 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
Traceback (most recent call last):
|
||||
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
|
||||
config = SupersetConfig(
|
||||
^^^^^^^^^^^^^^^
|
||||
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
|
||||
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
2025-12-20 23:01:43,894 - INFO - [BackupPlugin][Entry] Starting backup for superset.
|
||||
2025-12-20 23:01:43,894 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
|
||||
2025-12-20 23:01:43,895 - INFO - [setup_clients][Action] Loading environments from ConfigManager
|
||||
2025-12-20 23:01:43,895 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
Traceback (most recent call last):
|
||||
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
|
||||
config = SupersetConfig(
|
||||
^^^^^^^^^^^^^^^
|
||||
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
|
||||
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
2025-12-20 23:04:07,731 - INFO - [BackupPlugin][Entry] Starting backup for superset.
|
||||
2025-12-20 23:04:07,731 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
|
||||
2025-12-20 23:04:07,732 - INFO - [setup_clients][Action] Loading environments from ConfigManager
|
||||
2025-12-20 23:04:07,732 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
Traceback (most recent call last):
|
||||
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
|
||||
config = SupersetConfig(
|
||||
^^^^^^^^^^^^^^^
|
||||
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
|
||||
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
2025-12-20 23:06:39,641 - INFO - [BackupPlugin][Entry] Starting backup for superset.
|
||||
2025-12-20 23:06:39,642 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
|
||||
2025-12-20 23:06:39,687 - INFO - [setup_clients][Action] Loading environments from ConfigManager
|
||||
2025-12-20 23:06:39,689 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
Traceback (most recent call last):
|
||||
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
|
||||
config = SupersetConfig(
|
||||
^^^^^^^^^^^^^^^
|
||||
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
|
||||
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
|
||||
base_url
|
||||
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
|
||||
For further information visit https://errors.pydantic.dev/2.12/v/value_error
|
||||
2025-12-20 23:30:36,090 - INFO - [BackupPlugin][Entry] Starting backup for superset.
|
||||
2025-12-20 23:30:36,093 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
|
||||
2025-12-20 23:30:36,128 - INFO - [setup_clients][Action] Loading environments from ConfigManager
|
||||
2025-12-20 23:30:36,129 - INFO - [SupersetClient.__init__][Enter] Initializing SupersetClient.
|
||||
2025-12-20 23:30:36,129 - INFO - [APIClient.__init__][Entry] Initializing APIClient.
|
||||
2025-12-20 23:30:36,130 - WARNING - [_init_session][State] SSL verification disabled.
|
||||
2025-12-20 23:30:36,130 - INFO - [APIClient.__init__][Exit] APIClient initialized.
|
||||
2025-12-20 23:30:36,130 - INFO - [SupersetClient.__init__][Exit] SupersetClient initialized.
|
||||
2025-12-20 23:30:36,130 - INFO - [get_dashboards][Enter] Fetching dashboards.
|
||||
2025-12-20 23:30:36,131 - INFO - [authenticate][Enter] Authenticating to https://superset.bebesh.ru/api/v1
|
||||
2025-12-20 23:30:36,897 - INFO - [authenticate][Exit] Authenticated successfully.
|
||||
2025-12-20 23:30:37,527 - INFO - [get_dashboards][Exit] Found 11 dashboards.
|
||||
2025-12-20 23:30:37,527 - INFO - [BackupPlugin][Progress] Found 11 dashboards to export in superset.
|
||||
2025-12-20 23:30:37,529 - INFO - [export_dashboard][Enter] Exporting dashboard 11.
|
||||
2025-12-20 23:30:38,224 - INFO - [export_dashboard][Exit] Exported dashboard 11 to dashboard_export_20251220T203037.zip.
|
||||
2025-12-20 23:30:38,225 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
|
||||
2025-12-20 23:30:38,226 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/FCC New Coder Survey 2018/dashboard_export_20251220T203037.zip
|
||||
2025-12-20 23:30:38,227 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/FCC New Coder Survey 2018
|
||||
2025-12-20 23:30:38,230 - INFO - [export_dashboard][Enter] Exporting dashboard 10.
|
||||
2025-12-20 23:30:38,438 - INFO - [export_dashboard][Exit] Exported dashboard 10 to dashboard_export_20251220T203038.zip.
|
||||
2025-12-20 23:30:38,438 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
|
||||
2025-12-20 23:30:38,439 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/COVID Vaccine Dashboard/dashboard_export_20251220T203038.zip
|
||||
2025-12-20 23:30:38,439 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/COVID Vaccine Dashboard
|
||||
2025-12-20 23:30:38,440 - INFO - [export_dashboard][Enter] Exporting dashboard 9.
|
||||
2025-12-20 23:30:38,853 - INFO - [export_dashboard][Exit] Exported dashboard 9 to dashboard_export_20251220T203038.zip.
|
||||
2025-12-20 23:30:38,853 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
|
||||
2025-12-20 23:30:38,856 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Sales Dashboard/dashboard_export_20251220T203038.zip
|
||||
2025-12-20 23:30:38,856 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Sales Dashboard
|
||||
2025-12-20 23:30:38,858 - INFO - [export_dashboard][Enter] Exporting dashboard 8.
|
||||
2025-12-20 23:30:38,939 - INFO - [export_dashboard][Exit] Exported dashboard 8 to dashboard_export_20251220T203038.zip.
|
||||
2025-12-20 23:30:38,940 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
|
||||
2025-12-20 23:30:38,941 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Unicode Test/dashboard_export_20251220T203038.zip
|
||||
2025-12-20 23:30:38,941 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Unicode Test
|
||||
2025-12-20 23:30:38,942 - INFO - [export_dashboard][Enter] Exporting dashboard 7.
|
||||
2025-12-20 23:30:39,148 - INFO - [export_dashboard][Exit] Exported dashboard 7 to dashboard_export_20251220T203038.zip.
|
||||
2025-12-20 23:30:39,148 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
|
||||
2025-12-20 23:30:39,149 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Video Game Sales/dashboard_export_20251220T203038.zip
|
||||
2025-12-20 23:30:39,149 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Video Game Sales
|
||||
2025-12-20 23:30:39,150 - INFO - [export_dashboard][Enter] Exporting dashboard 6.
|
||||
2025-12-20 23:30:39,689 - INFO - [export_dashboard][Exit] Exported dashboard 6 to dashboard_export_20251220T203039.zip.
|
||||
2025-12-20 23:30:39,689 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
|
||||
2025-12-20 23:30:39,690 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Featured Charts/dashboard_export_20251220T203039.zip
|
||||
2025-12-20 23:30:39,691 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Featured Charts
|
||||
2025-12-20 23:30:39,692 - INFO - [export_dashboard][Enter] Exporting dashboard 5.
|
||||
2025-12-20 23:30:39,960 - INFO - [export_dashboard][Exit] Exported dashboard 5 to dashboard_export_20251220T203039.zip.
|
||||
2025-12-20 23:30:39,960 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
|
||||
2025-12-20 23:30:39,961 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Slack Dashboard/dashboard_export_20251220T203039.zip
|
||||
2025-12-20 23:30:39,961 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Slack Dashboard
|
||||
2025-12-20 23:30:39,962 - INFO - [export_dashboard][Enter] Exporting dashboard 4.
|
||||
2025-12-20 23:30:40,196 - INFO - [export_dashboard][Exit] Exported dashboard 4 to dashboard_export_20251220T203039.zip.
|
||||
2025-12-20 23:30:40,196 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
|
||||
2025-12-20 23:30:40,197 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/deck.gl Demo/dashboard_export_20251220T203039.zip
|
||||
2025-12-20 23:30:40,197 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/deck.gl Demo
|
||||
2025-12-20 23:30:40,198 - INFO - [export_dashboard][Enter] Exporting dashboard 3.
|
||||
2025-12-20 23:30:40,745 - INFO - [export_dashboard][Exit] Exported dashboard 3 to dashboard_export_20251220T203040.zip.
|
||||
2025-12-20 23:30:40,746 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
|
||||
2025-12-20 23:30:40,760 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Misc Charts/dashboard_export_20251220T203040.zip
|
||||
2025-12-20 23:30:40,761 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Misc Charts
|
||||
2025-12-20 23:30:40,762 - INFO - [export_dashboard][Enter] Exporting dashboard 2.
|
||||
2025-12-20 23:30:40,928 - INFO - [export_dashboard][Exit] Exported dashboard 2 to dashboard_export_20251220T203040.zip.
|
||||
2025-12-20 23:30:40,929 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
|
||||
2025-12-20 23:30:40,930 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/USA Births Names/dashboard_export_20251220T203040.zip
|
||||
2025-12-20 23:30:40,931 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/USA Births Names
|
||||
2025-12-20 23:30:40,932 - INFO - [export_dashboard][Enter] Exporting dashboard 1.
|
||||
2025-12-20 23:30:41,582 - INFO - [export_dashboard][Exit] Exported dashboard 1 to dashboard_export_20251220T203040.zip.
|
||||
2025-12-20 23:30:41,582 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
|
||||
2025-12-20 23:30:41,749 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/World Bank's Data/dashboard_export_20251220T203040.zip
|
||||
2025-12-20 23:30:41,750 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/World Bank's Data
|
||||
2025-12-20 23:30:41,752 - INFO - [consolidate_archive_folders][Enter] Consolidating archives in backups/SUPERSET
|
||||
2025-12-20 23:30:41,753 - INFO - [remove_empty_directories][Enter] Starting cleanup of empty directories in backups/SUPERSET
|
||||
2025-12-20 23:30:41,758 - INFO - [remove_empty_directories][Exit] Removed 0 empty directories.
|
||||
2025-12-20 23:30:41,758 - INFO - [BackupPlugin][CoherenceCheck:Passed] Backup logic completed for superset.
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
backend/mappings.db
Normal file
BIN
backend/mappings.db
Normal file
Binary file not shown.
@@ -1 +1 @@
|
||||
from . import plugins, tasks, settings, connections, environments, mappings, migration, git
|
||||
from . import plugins, tasks, settings, connections, environments, mappings, migration, git, storage
|
||||
|
||||
@@ -23,7 +23,7 @@ router = APIRouter()
|
||||
# [DEF:ScheduleSchema:DataClass]
|
||||
class ScheduleSchema(BaseModel):
|
||||
enabled: bool = False
|
||||
cron_expression: str = Field(..., pattern=r'^(@(annually|yearly|monthly|weekly|daily|hourly|reboot))|((((\d+,)*\d+|(\d+(\/|-)\d+)|\d+|\*) ?){5,7})$')
|
||||
cron_expression: str = Field(..., pattern=r'^(@(annually|yearly|monthly|weekly|daily|hourly|reboot))|((((\d+,)*\d+|(\d+(\/|-)\d+)|\d+|\*) ?){4,6})$')
|
||||
# [/DEF:ScheduleSchema:DataClass]
|
||||
|
||||
# [DEF:EnvironmentResponse:DataClass]
|
||||
|
||||
@@ -13,6 +13,7 @@
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from typing import List
|
||||
from ...core.config_models import AppConfig, Environment, GlobalSettings
|
||||
from ...models.storage import StorageConfig
|
||||
from ...dependencies import get_config_manager
|
||||
from ...core.config_manager import ConfigManager
|
||||
from ...core.logger import logger, belief_scope
|
||||
@@ -52,10 +53,38 @@ async def update_global_settings(
|
||||
):
|
||||
with belief_scope("update_global_settings"):
|
||||
logger.info("[update_global_settings][Entry] Updating global settings")
|
||||
|
||||
config_manager.update_global_settings(settings)
|
||||
return settings
|
||||
# [/DEF:update_global_settings:Function]
|
||||
|
||||
# [DEF:get_storage_settings:Function]
|
||||
# @PURPOSE: Retrieves storage-specific settings.
|
||||
# @RETURN: StorageConfig - The storage configuration.
|
||||
@router.get("/storage", response_model=StorageConfig)
|
||||
async def get_storage_settings(config_manager: ConfigManager = Depends(get_config_manager)):
|
||||
with belief_scope("get_storage_settings"):
|
||||
return config_manager.get_config().settings.storage
|
||||
# [/DEF:get_storage_settings:Function]
|
||||
|
||||
# [DEF:update_storage_settings:Function]
|
||||
# @PURPOSE: Updates storage-specific settings.
|
||||
# @PARAM: storage (StorageConfig) - The new storage settings.
|
||||
# @POST: Storage settings are updated and saved.
|
||||
# @RETURN: StorageConfig - The updated storage settings.
|
||||
@router.put("/storage", response_model=StorageConfig)
|
||||
async def update_storage_settings(storage: StorageConfig, config_manager: ConfigManager = Depends(get_config_manager)):
|
||||
with belief_scope("update_storage_settings"):
|
||||
is_valid, message = config_manager.validate_path(storage.root_path)
|
||||
if not is_valid:
|
||||
raise HTTPException(status_code=400, detail=message)
|
||||
|
||||
settings = config_manager.get_config().settings
|
||||
settings.storage = storage
|
||||
config_manager.update_global_settings(settings)
|
||||
return config_manager.get_config().settings.storage
|
||||
# [/DEF:update_storage_settings:Function]
|
||||
|
||||
# [DEF:get_environments:Function]
|
||||
# @PURPOSE: Lists all configured Superset environments.
|
||||
# @PRE: Config manager is available.
|
||||
@@ -179,30 +208,5 @@ async def test_environment_connection(
|
||||
return {"status": "error", "message": str(e)}
|
||||
# [/DEF:test_environment_connection:Function]
|
||||
|
||||
# [DEF:validate_backup_path:Function]
|
||||
# @PURPOSE: Validates if a backup path exists and is writable.
|
||||
# @PRE: Path is provided in path_data.
|
||||
# @POST: Returns success or error status.
|
||||
# @PARAM: path (str) - The path to validate.
|
||||
# @RETURN: dict - Validation result.
|
||||
@router.post("/validate-path")
|
||||
async def validate_backup_path(
|
||||
path_data: dict,
|
||||
config_manager: ConfigManager = Depends(get_config_manager)
|
||||
):
|
||||
with belief_scope("validate_backup_path"):
|
||||
path = path_data.get("path")
|
||||
if not path:
|
||||
raise HTTPException(status_code=400, detail="Path is required")
|
||||
|
||||
logger.info(f"[validate_backup_path][Entry] Validating path: {path}")
|
||||
|
||||
valid, message = config_manager.validate_path(path)
|
||||
|
||||
if not valid:
|
||||
return {"status": "error", "message": message}
|
||||
|
||||
return {"status": "success", "message": message}
|
||||
# [/DEF:validate_backup_path:Function]
|
||||
|
||||
# [/DEF:SettingsRouter:Module]
|
||||
|
||||
132
backend/src/api/routes/storage.py
Normal file
132
backend/src/api/routes/storage.py
Normal file
@@ -0,0 +1,132 @@
|
||||
# [DEF:storage_routes:Module]
|
||||
#
|
||||
# @SEMANTICS: storage, files, upload, download, backup, repository
|
||||
# @PURPOSE: API endpoints for file storage management (backups and repositories).
|
||||
# @LAYER: API
|
||||
# @RELATION: DEPENDS_ON -> backend.src.models.storage
|
||||
#
|
||||
# @INVARIANT: All paths must be validated against path traversal.
|
||||
|
||||
# [SECTION: IMPORTS]
|
||||
from fastapi import APIRouter, Depends, UploadFile, File, Form, HTTPException
|
||||
from fastapi.responses import FileResponse
|
||||
from typing import List, Optional
|
||||
from ...models.storage import StoredFile, FileCategory
|
||||
from ...dependencies import get_plugin_loader
|
||||
from ...plugins.storage.plugin import StoragePlugin
|
||||
from ...core.logger import belief_scope
|
||||
# [/SECTION]
|
||||
|
||||
router = APIRouter(tags=["storage"])
|
||||
|
||||
# [DEF:list_files:Function]
|
||||
# @PURPOSE: List all files and directories in the storage system.
|
||||
#
|
||||
# @PRE: None.
|
||||
# @POST: Returns a list of StoredFile objects.
|
||||
#
|
||||
# @PARAM: category (Optional[FileCategory]) - Filter by category.
|
||||
# @PARAM: path (Optional[str]) - Subpath within the category.
|
||||
# @RETURN: List[StoredFile] - List of files/directories.
|
||||
#
|
||||
# @RELATION: CALLS -> StoragePlugin.list_files
|
||||
@router.get("/files", response_model=List[StoredFile])
|
||||
async def list_files(
|
||||
category: Optional[FileCategory] = None,
|
||||
path: Optional[str] = None,
|
||||
plugin_loader=Depends(get_plugin_loader)
|
||||
):
|
||||
with belief_scope("list_files"):
|
||||
storage_plugin: StoragePlugin = plugin_loader.get_plugin("storage-manager")
|
||||
if not storage_plugin:
|
||||
raise HTTPException(status_code=500, detail="Storage plugin not loaded")
|
||||
return storage_plugin.list_files(category, path)
|
||||
# [/DEF:list_files:Function]
|
||||
|
||||
# [DEF:upload_file:Function]
|
||||
# @PURPOSE: Upload a file to the storage system.
|
||||
#
|
||||
# @PRE: category must be a valid FileCategory.
|
||||
# @PRE: file must be a valid UploadFile.
|
||||
# @POST: Returns the StoredFile object of the uploaded file.
|
||||
#
|
||||
# @PARAM: category (FileCategory) - Target category.
|
||||
# @PARAM: path (Optional[str]) - Target subpath.
|
||||
# @PARAM: file (UploadFile) - The file content.
|
||||
# @RETURN: StoredFile - Metadata of the uploaded file.
|
||||
#
|
||||
# @SIDE_EFFECT: Writes file to the filesystem.
|
||||
#
|
||||
# @RELATION: CALLS -> StoragePlugin.save_file
|
||||
@router.post("/upload", response_model=StoredFile, status_code=201)
|
||||
async def upload_file(
|
||||
category: FileCategory = Form(...),
|
||||
path: Optional[str] = Form(None),
|
||||
file: UploadFile = File(...),
|
||||
plugin_loader=Depends(get_plugin_loader)
|
||||
):
|
||||
with belief_scope("upload_file"):
|
||||
storage_plugin: StoragePlugin = plugin_loader.get_plugin("storage-manager")
|
||||
if not storage_plugin:
|
||||
raise HTTPException(status_code=500, detail="Storage plugin not loaded")
|
||||
try:
|
||||
return await storage_plugin.save_file(file, category, path)
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
# [/DEF:upload_file:Function]
|
||||
|
||||
# [DEF:delete_file:Function]
|
||||
# @PURPOSE: Delete a specific file or directory.
|
||||
#
|
||||
# @PRE: category must be a valid FileCategory.
|
||||
# @POST: Item is removed from storage.
|
||||
#
|
||||
# @PARAM: category (FileCategory) - File category.
|
||||
# @PARAM: path (str) - Relative path of the item.
|
||||
# @RETURN: None
|
||||
#
|
||||
# @SIDE_EFFECT: Deletes item from the filesystem.
|
||||
#
|
||||
# @RELATION: CALLS -> StoragePlugin.delete_file
|
||||
@router.delete("/files/{category}/{path:path}", status_code=204)
|
||||
async def delete_file(category: FileCategory, path: str, plugin_loader=Depends(get_plugin_loader)):
|
||||
with belief_scope("delete_file"):
|
||||
storage_plugin: StoragePlugin = plugin_loader.get_plugin("storage-manager")
|
||||
if not storage_plugin:
|
||||
raise HTTPException(status_code=500, detail="Storage plugin not loaded")
|
||||
try:
|
||||
storage_plugin.delete_file(category, path)
|
||||
except FileNotFoundError:
|
||||
raise HTTPException(status_code=404, detail="File not found")
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
# [/DEF:delete_file:Function]
|
||||
|
||||
# [DEF:download_file:Function]
|
||||
# @PURPOSE: Retrieve a file for download.
|
||||
#
|
||||
# @PRE: category must be a valid FileCategory.
|
||||
# @POST: Returns a FileResponse.
|
||||
#
|
||||
# @PARAM: category (FileCategory) - File category.
|
||||
# @PARAM: path (str) - Relative path of the file.
|
||||
# @RETURN: FileResponse - The file content.
|
||||
#
|
||||
# @RELATION: CALLS -> StoragePlugin.get_file_path
|
||||
@router.get("/download/{category}/{path:path}")
|
||||
async def download_file(category: FileCategory, path: str, plugin_loader=Depends(get_plugin_loader)):
|
||||
with belief_scope("download_file"):
|
||||
storage_plugin: StoragePlugin = plugin_loader.get_plugin("storage-manager")
|
||||
if not storage_plugin:
|
||||
raise HTTPException(status_code=500, detail="Storage plugin not loaded")
|
||||
try:
|
||||
abs_path = storage_plugin.get_file_path(category, path)
|
||||
filename = Path(path).name
|
||||
return FileResponse(path=abs_path, filename=filename)
|
||||
except FileNotFoundError:
|
||||
raise HTTPException(status_code=404, detail="File not found")
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
# [/DEF:download_file:Function]
|
||||
|
||||
# [/DEF:storage_routes:Module]
|
||||
@@ -18,7 +18,7 @@ import os
|
||||
|
||||
from .dependencies import get_task_manager, get_scheduler_service
|
||||
from .core.logger import logger, belief_scope
|
||||
from .api.routes import plugins, tasks, settings, environments, mappings, migration, connections, git
|
||||
from .api.routes import plugins, tasks, settings, environments, mappings, migration, connections, git, storage
|
||||
from .core.database import init_db
|
||||
|
||||
# [DEF:App:Global]
|
||||
@@ -89,6 +89,7 @@ app.include_router(environments.router, prefix="/api/environments", tags=["Envir
|
||||
app.include_router(mappings.router)
|
||||
app.include_router(migration.router)
|
||||
app.include_router(git.router)
|
||||
app.include_router(storage.router, prefix="/api/storage", tags=["Storage"])
|
||||
|
||||
# [DEF:websocket_endpoint:Function]
|
||||
# @PURPOSE: Provides a WebSocket endpoint for real-time log streaming of a task.
|
||||
|
||||
@@ -62,14 +62,18 @@ class ConfigManager:
|
||||
logger.info(f"[_load_config][Action] Config file not found. Creating default.")
|
||||
default_config = AppConfig(
|
||||
environments=[],
|
||||
settings=GlobalSettings(backup_path="backups")
|
||||
settings=GlobalSettings()
|
||||
)
|
||||
self._save_config_to_disk(default_config)
|
||||
return default_config
|
||||
|
||||
try:
|
||||
with open(self.config_path, "r") as f:
|
||||
data = json.load(f)
|
||||
|
||||
# Check for deprecated field
|
||||
if "settings" in data and "backup_path" in data["settings"]:
|
||||
del data["settings"]["backup_path"]
|
||||
|
||||
config = AppConfig(**data)
|
||||
logger.info(f"[_load_config][Coherence:OK] Configuration loaded")
|
||||
return config
|
||||
@@ -79,7 +83,7 @@ class ConfigManager:
|
||||
# For now, return default to be safe, but log the error prominently.
|
||||
return AppConfig(
|
||||
environments=[],
|
||||
settings=GlobalSettings(backup_path="backups")
|
||||
settings=GlobalSettings(storage=StorageConfig())
|
||||
)
|
||||
# [/DEF:_load_config:Function]
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
from typing import List, Optional
|
||||
from ..models.storage import StorageConfig
|
||||
|
||||
# [DEF:Schedule:DataClass]
|
||||
# @PURPOSE: Represents a backup schedule configuration.
|
||||
@@ -42,7 +43,7 @@ class LoggingConfig(BaseModel):
|
||||
# [DEF:GlobalSettings:DataClass]
|
||||
# @PURPOSE: Represents global application settings.
|
||||
class GlobalSettings(BaseModel):
|
||||
backup_path: str
|
||||
storage: StorageConfig = Field(default_factory=StorageConfig)
|
||||
default_environment_id: Optional[str] = None
|
||||
logging: LoggingConfig = Field(default_factory=LoggingConfig)
|
||||
|
||||
|
||||
@@ -28,6 +28,7 @@ class BeliefFormatter(logging.Formatter):
|
||||
# @POST: Returns formatted string.
|
||||
# @PARAM: record (logging.LogRecord) - The log record to format.
|
||||
# @RETURN: str - The formatted log message.
|
||||
# @SEMANTICS: logging, formatter, context
|
||||
def format(self, record):
|
||||
anchor_id = getattr(_belief_state, 'anchor_id', None)
|
||||
if anchor_id:
|
||||
@@ -54,6 +55,7 @@ class LogEntry(BaseModel):
|
||||
# @PARAM: message (str) - Optional entry message.
|
||||
# @PRE: anchor_id must be provided.
|
||||
# @POST: Thread-local belief state is updated and entry/exit logs are generated.
|
||||
# @SEMANTICS: logging, context, belief_state
|
||||
@contextmanager
|
||||
def belief_scope(anchor_id: str, message: str = ""):
|
||||
# Log Entry if enabled
|
||||
@@ -88,6 +90,7 @@ def belief_scope(anchor_id: str, message: str = ""):
|
||||
# @PRE: config is a valid LoggingConfig instance.
|
||||
# @POST: Logger level, handlers, and belief state flag are updated.
|
||||
# @PARAM: config (LoggingConfig) - The logging configuration.
|
||||
# @SEMANTICS: logging, configuration, initialization
|
||||
def configure_logger(config):
|
||||
global _enable_belief_state
|
||||
_enable_belief_state = config.enable_belief_state
|
||||
@@ -140,6 +143,7 @@ class WebSocketLogHandler(logging.Handler):
|
||||
# @PRE: capacity is an integer.
|
||||
# @POST: Instance initialized with empty deque.
|
||||
# @PARAM: capacity (int) - Maximum number of logs to keep in memory.
|
||||
# @SEMANTICS: logging, initialization, buffer
|
||||
def __init__(self, capacity: int = 1000):
|
||||
super().__init__()
|
||||
self.log_buffer: deque[LogEntry] = deque(maxlen=capacity)
|
||||
@@ -152,6 +156,7 @@ class WebSocketLogHandler(logging.Handler):
|
||||
# @PRE: record is a logging.LogRecord.
|
||||
# @POST: Log is added to the log_buffer.
|
||||
# @PARAM: record (logging.LogRecord) - The log record to emit.
|
||||
# @SEMANTICS: logging, handler, buffer
|
||||
def emit(self, record: logging.LogRecord):
|
||||
try:
|
||||
log_entry = LogEntry(
|
||||
@@ -179,6 +184,7 @@ class WebSocketLogHandler(logging.Handler):
|
||||
# @PRE: None.
|
||||
# @POST: Returns list of LogEntry objects.
|
||||
# @RETURN: List[LogEntry] - List of buffered log entries.
|
||||
# @SEMANTICS: logging, buffer, retrieval
|
||||
def get_recent_logs(self) -> List[LogEntry]:
|
||||
"""
|
||||
Returns a list of recent log entries from the buffer.
|
||||
@@ -196,12 +202,18 @@ logger = logging.getLogger("superset_tools_app")
|
||||
# [DEF:believed:Function]
|
||||
# @PURPOSE: A decorator that wraps a function in a belief scope.
|
||||
# @PARAM: anchor_id (str) - The identifier for the semantic block.
|
||||
# @PRE: anchor_id must be a string.
|
||||
# @POST: Returns a decorator function.
|
||||
def believed(anchor_id: str):
|
||||
# [DEF:decorator:Function]
|
||||
# @PURPOSE: Internal decorator for belief scope.
|
||||
# @PRE: func must be a callable.
|
||||
# @POST: Returns the wrapped function.
|
||||
def decorator(func):
|
||||
# [DEF:wrapper:Function]
|
||||
# @PURPOSE: Internal wrapper that enters belief scope.
|
||||
# @PRE: None.
|
||||
# @POST: Executes the function within a belief scope.
|
||||
def wrapper(*args, **kwargs):
|
||||
with belief_scope(anchor_id):
|
||||
return func(*args, **kwargs)
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import Dict, Any
|
||||
from typing import Dict, Any, Optional
|
||||
from .logger import belief_scope
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
@@ -68,6 +68,21 @@ class PluginBase(ABC):
|
||||
pass
|
||||
# [/DEF:version:Function]
|
||||
|
||||
@property
|
||||
# [DEF:ui_route:Function]
|
||||
# @PURPOSE: Returns the frontend route for the plugin's UI, if applicable.
|
||||
# @PRE: Plugin instance exists.
|
||||
# @POST: Returns string route or None.
|
||||
# @RETURN: Optional[str] - Frontend route.
|
||||
def ui_route(self) -> Optional[str]:
|
||||
"""
|
||||
The frontend route for the plugin's UI.
|
||||
Returns None if the plugin does not have a dedicated UI page.
|
||||
"""
|
||||
with belief_scope("ui_route"):
|
||||
return None
|
||||
# [/DEF:ui_route:Function]
|
||||
|
||||
@abstractmethod
|
||||
# [DEF:get_schema:Function]
|
||||
# @PURPOSE: Returns the JSON schema for the plugin's input parameters.
|
||||
@@ -111,5 +126,6 @@ class PluginConfig(BaseModel):
|
||||
name: str = Field(..., description="Human-readable name for the plugin")
|
||||
description: str = Field(..., description="Brief description of what the plugin does")
|
||||
version: str = Field(..., description="Version of the plugin")
|
||||
ui_route: Optional[str] = Field(None, description="Frontend route for the plugin UI")
|
||||
input_schema: Dict[str, Any] = Field(..., description="JSON schema for input parameters", alias="schema")
|
||||
# [/DEF:PluginConfig:Class]
|
||||
@@ -50,9 +50,18 @@ class PluginLoader:
|
||||
sys.path.insert(0, plugin_parent_dir)
|
||||
|
||||
for filename in os.listdir(self.plugin_dir):
|
||||
file_path = os.path.join(self.plugin_dir, filename)
|
||||
|
||||
# Handle directory-based plugins (packages)
|
||||
if os.path.isdir(file_path):
|
||||
init_file = os.path.join(file_path, "__init__.py")
|
||||
if os.path.exists(init_file):
|
||||
self._load_module(filename, init_file)
|
||||
continue
|
||||
|
||||
# Handle single-file plugins
|
||||
if filename.endswith(".py") and filename != "__init__.py":
|
||||
module_name = filename[:-3]
|
||||
file_path = os.path.join(self.plugin_dir, filename)
|
||||
self._load_module(module_name, file_path)
|
||||
# [/DEF:_load_plugins:Function]
|
||||
|
||||
@@ -132,6 +141,7 @@ class PluginLoader:
|
||||
name=plugin_instance.name,
|
||||
description=plugin_instance.description,
|
||||
version=plugin_instance.version,
|
||||
ui_route=plugin_instance.ui_route,
|
||||
schema=schema,
|
||||
)
|
||||
# The following line is commented out because it requires a schema to be passed to validate against.
|
||||
|
||||
31
backend/src/models/storage.py
Normal file
31
backend/src/models/storage.py
Normal file
@@ -0,0 +1,31 @@
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
from typing import Optional
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
# [DEF:FileCategory:Class]
|
||||
# @PURPOSE: Enumeration of supported file categories in the storage system.
|
||||
class FileCategory(str, Enum):
|
||||
BACKUP = "backups"
|
||||
REPOSITORY = "repositorys"
|
||||
# [/DEF:FileCategory:Class]
|
||||
|
||||
# [DEF:StorageConfig:Class]
|
||||
# @PURPOSE: Configuration model for the storage system, defining paths and naming patterns.
|
||||
class StorageConfig(BaseModel):
|
||||
root_path: str = Field(default="backups", description="Absolute path to the storage root directory.")
|
||||
backup_structure_pattern: str = Field(default="{category}/", description="Pattern for backup directory structure.")
|
||||
repo_structure_pattern: str = Field(default="{category}/", description="Pattern for repository directory structure.")
|
||||
filename_pattern: str = Field(default="{name}_{timestamp}", description="Pattern for filenames.")
|
||||
# [/DEF:StorageConfig:Class]
|
||||
|
||||
# [DEF:StoredFile:Class]
|
||||
# @PURPOSE: Data model representing metadata for a file stored in the system.
|
||||
class StoredFile(BaseModel):
|
||||
name: str = Field(..., description="Name of the file (including extension).")
|
||||
path: str = Field(..., description="Relative path from storage root.")
|
||||
size: int = Field(..., ge=0, description="Size of the file in bytes.")
|
||||
created_at: datetime = Field(..., description="Creation timestamp.")
|
||||
category: FileCategory = Field(..., description="Category of the file.")
|
||||
mime_type: Optional[str] = Field(None, description="MIME type of the file.")
|
||||
# [/DEF:StoredFile:Class]
|
||||
@@ -75,6 +75,15 @@ class BackupPlugin(PluginBase):
|
||||
return "1.0.0"
|
||||
# [/DEF:version:Function]
|
||||
|
||||
@property
|
||||
# [DEF:ui_route:Function]
|
||||
# @PURPOSE: Returns the frontend route for the backup plugin.
|
||||
# @RETURN: str - "/tools/backups"
|
||||
def ui_route(self) -> str:
|
||||
with belief_scope("ui_route"):
|
||||
return "/tools/backups"
|
||||
# [/DEF:ui_route:Function]
|
||||
|
||||
# [DEF:get_schema:Function]
|
||||
# @PURPOSE: Returns the JSON schema for backup plugin parameters.
|
||||
# @PRE: Plugin instance exists.
|
||||
@@ -84,7 +93,7 @@ class BackupPlugin(PluginBase):
|
||||
with belief_scope("get_schema"):
|
||||
config_manager = get_config_manager()
|
||||
envs = [e.name for e in config_manager.get_environments()]
|
||||
default_path = config_manager.get_config().settings.backup_path
|
||||
default_path = config_manager.get_config().settings.storage.root_path
|
||||
|
||||
return {
|
||||
"type": "object",
|
||||
@@ -95,14 +104,8 @@ class BackupPlugin(PluginBase):
|
||||
"description": "The Superset environment to back up.",
|
||||
"enum": envs if envs else [],
|
||||
},
|
||||
"backup_path": {
|
||||
"type": "string",
|
||||
"title": "Backup Path",
|
||||
"description": "The root directory to save backups to.",
|
||||
"default": default_path
|
||||
}
|
||||
},
|
||||
"required": ["env", "backup_path"],
|
||||
"required": ["env"],
|
||||
}
|
||||
# [/DEF:get_schema:Function]
|
||||
|
||||
@@ -126,8 +129,9 @@ class BackupPlugin(PluginBase):
|
||||
if not env:
|
||||
raise KeyError("env")
|
||||
|
||||
backup_path_str = params.get("backup_path") or config_manager.get_config().settings.backup_path
|
||||
backup_path = Path(backup_path_str)
|
||||
storage_settings = config_manager.get_config().settings.storage
|
||||
# Use 'backups' subfolder within the storage root
|
||||
backup_path = Path(storage_settings.root_path) / "backups"
|
||||
|
||||
from ..core.logger import logger as app_logger
|
||||
app_logger.info(f"[BackupPlugin][Entry] Starting backup for {env}.")
|
||||
|
||||
@@ -63,6 +63,15 @@ class DebugPlugin(PluginBase):
|
||||
return "1.0.0"
|
||||
# [/DEF:version:Function]
|
||||
|
||||
@property
|
||||
# [DEF:ui_route:Function]
|
||||
# @PURPOSE: Returns the frontend route for the debug plugin.
|
||||
# @RETURN: str - "/tools/debug"
|
||||
def ui_route(self) -> str:
|
||||
with belief_scope("ui_route"):
|
||||
return "/tools/debug"
|
||||
# [/DEF:ui_route:Function]
|
||||
|
||||
# [DEF:get_schema:Function]
|
||||
# @PURPOSE: Returns the JSON schema for the debug plugin parameters.
|
||||
# @PRE: Plugin instance exists.
|
||||
|
||||
@@ -99,6 +99,15 @@ class GitPlugin(PluginBase):
|
||||
return "0.1.0"
|
||||
# [/DEF:version:Function]
|
||||
|
||||
@property
|
||||
# [DEF:ui_route:Function]
|
||||
# @PURPOSE: Returns the frontend route for the git plugin.
|
||||
# @RETURN: str - "/git"
|
||||
def ui_route(self) -> str:
|
||||
with belief_scope("GitPlugin.ui_route"):
|
||||
return "/git"
|
||||
# [/DEF:ui_route:Function]
|
||||
|
||||
# [DEF:get_schema:Function]
|
||||
# @PURPOSE: Возвращает JSON-схему параметров для выполнения задач плагина.
|
||||
# @PRE: GitPlugin is initialized.
|
||||
|
||||
@@ -66,6 +66,15 @@ class MapperPlugin(PluginBase):
|
||||
return "1.0.0"
|
||||
# [/DEF:version:Function]
|
||||
|
||||
@property
|
||||
# [DEF:ui_route:Function]
|
||||
# @PURPOSE: Returns the frontend route for the mapper plugin.
|
||||
# @RETURN: str - "/tools/mapper"
|
||||
def ui_route(self) -> str:
|
||||
with belief_scope("ui_route"):
|
||||
return "/tools/mapper"
|
||||
# [/DEF:ui_route:Function]
|
||||
|
||||
# [DEF:get_schema:Function]
|
||||
# @PURPOSE: Returns the JSON schema for the mapper plugin parameters.
|
||||
# @PRE: Plugin instance exists.
|
||||
|
||||
@@ -71,6 +71,15 @@ class MigrationPlugin(PluginBase):
|
||||
return "1.0.0"
|
||||
# [/DEF:version:Function]
|
||||
|
||||
@property
|
||||
# [DEF:ui_route:Function]
|
||||
# @PURPOSE: Returns the frontend route for the migration plugin.
|
||||
# @RETURN: str - "/migration"
|
||||
def ui_route(self) -> str:
|
||||
with belief_scope("ui_route"):
|
||||
return "/migration"
|
||||
# [/DEF:ui_route:Function]
|
||||
|
||||
# [DEF:get_schema:Function]
|
||||
# @PURPOSE: Returns the JSON schema for migration plugin parameters.
|
||||
# @PRE: Config manager is available.
|
||||
|
||||
@@ -64,6 +64,15 @@ class SearchPlugin(PluginBase):
|
||||
return "1.0.0"
|
||||
# [/DEF:version:Function]
|
||||
|
||||
@property
|
||||
# [DEF:ui_route:Function]
|
||||
# @PURPOSE: Returns the frontend route for the search plugin.
|
||||
# @RETURN: str - "/tools/search"
|
||||
def ui_route(self) -> str:
|
||||
with belief_scope("ui_route"):
|
||||
return "/tools/search"
|
||||
# [/DEF:ui_route:Function]
|
||||
|
||||
# [DEF:get_schema:Function]
|
||||
# @PURPOSE: Returns the JSON schema for the search plugin parameters.
|
||||
# @PRE: Plugin instance exists.
|
||||
|
||||
3
backend/src/plugins/storage/__init__.py
Normal file
3
backend/src/plugins/storage/__init__.py
Normal file
@@ -0,0 +1,3 @@
|
||||
from .plugin import StoragePlugin
|
||||
|
||||
__all__ = ["StoragePlugin"]
|
||||
333
backend/src/plugins/storage/plugin.py
Normal file
333
backend/src/plugins/storage/plugin.py
Normal file
@@ -0,0 +1,333 @@
|
||||
# [DEF:StoragePlugin:Module]
|
||||
#
|
||||
# @SEMANTICS: storage, files, filesystem, plugin
|
||||
# @PURPOSE: Provides core filesystem operations for managing backups and repositories.
|
||||
# @LAYER: App
|
||||
# @RELATION: IMPLEMENTS -> PluginBase
|
||||
# @RELATION: DEPENDS_ON -> backend.src.models.storage
|
||||
#
|
||||
# @INVARIANT: All file operations must be restricted to the configured storage root.
|
||||
|
||||
# [SECTION: IMPORTS]
|
||||
import os
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
from typing import Dict, Any, List, Optional
|
||||
from fastapi import UploadFile
|
||||
|
||||
from ...core.plugin_base import PluginBase
|
||||
from ...core.logger import belief_scope, logger
|
||||
from ...models.storage import StoredFile, FileCategory, StorageConfig
|
||||
from ...dependencies import get_config_manager
|
||||
# [/SECTION]
|
||||
|
||||
# [DEF:StoragePlugin:Class]
|
||||
# @PURPOSE: Implementation of the storage management plugin.
|
||||
class StoragePlugin(PluginBase):
|
||||
"""
|
||||
Plugin for managing local file storage for backups and repositories.
|
||||
"""
|
||||
|
||||
# [DEF:__init__:Function]
|
||||
# @PURPOSE: Initializes the StoragePlugin and ensures required directories exist.
|
||||
# @PRE: Configuration manager must be accessible.
|
||||
# @POST: Storage root and category directories are created on disk.
|
||||
def __init__(self):
|
||||
with belief_scope("StoragePlugin:init"):
|
||||
self.ensure_directories()
|
||||
# [/DEF:__init__:Function]
|
||||
|
||||
@property
|
||||
# [DEF:id:Function]
|
||||
# @PURPOSE: Returns the unique identifier for the storage plugin.
|
||||
# @PRE: None.
|
||||
# @POST: Returns the plugin ID string.
|
||||
# @RETURN: str - "storage-manager"
|
||||
def id(self) -> str:
|
||||
with belief_scope("StoragePlugin:id"):
|
||||
return "storage-manager"
|
||||
# [/DEF:id:Function]
|
||||
|
||||
@property
|
||||
# [DEF:name:Function]
|
||||
# @PURPOSE: Returns the human-readable name of the storage plugin.
|
||||
# @PRE: None.
|
||||
# @POST: Returns the plugin name string.
|
||||
# @RETURN: str - "Storage Manager"
|
||||
def name(self) -> str:
|
||||
with belief_scope("StoragePlugin:name"):
|
||||
return "Storage Manager"
|
||||
# [/DEF:name:Function]
|
||||
|
||||
@property
|
||||
# [DEF:description:Function]
|
||||
# @PURPOSE: Returns a description of the storage plugin.
|
||||
# @PRE: None.
|
||||
# @POST: Returns the plugin description string.
|
||||
# @RETURN: str - Plugin description.
|
||||
def description(self) -> str:
|
||||
with belief_scope("StoragePlugin:description"):
|
||||
return "Manages local file storage for backups and repositories."
|
||||
# [/DEF:description:Function]
|
||||
|
||||
@property
|
||||
# [DEF:version:Function]
|
||||
# @PURPOSE: Returns the version of the storage plugin.
|
||||
# @PRE: None.
|
||||
# @POST: Returns the version string.
|
||||
# @RETURN: str - "1.0.0"
|
||||
def version(self) -> str:
|
||||
with belief_scope("StoragePlugin:version"):
|
||||
return "1.0.0"
|
||||
# [/DEF:version:Function]
|
||||
|
||||
@property
|
||||
# [DEF:ui_route:Function]
|
||||
# @PURPOSE: Returns the frontend route for the storage plugin.
|
||||
# @RETURN: str - "/tools/storage"
|
||||
def ui_route(self) -> str:
|
||||
with belief_scope("StoragePlugin:ui_route"):
|
||||
return "/tools/storage"
|
||||
# [/DEF:ui_route:Function]
|
||||
|
||||
# [DEF:get_schema:Function]
|
||||
# @PURPOSE: Returns the JSON schema for storage plugin parameters.
|
||||
# @PRE: None.
|
||||
# @POST: Returns a dictionary representing the JSON schema.
|
||||
# @RETURN: Dict[str, Any] - JSON schema.
|
||||
def get_schema(self) -> Dict[str, Any]:
|
||||
with belief_scope("StoragePlugin:get_schema"):
|
||||
return {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"category": {
|
||||
"type": "string",
|
||||
"enum": [c.value for c in FileCategory],
|
||||
"title": "Category"
|
||||
}
|
||||
},
|
||||
"required": ["category"]
|
||||
}
|
||||
# [/DEF:get_schema:Function]
|
||||
|
||||
# [DEF:execute:Function]
|
||||
# @PURPOSE: Executes storage-related tasks (placeholder for PluginBase compliance).
|
||||
# @PRE: params must match the plugin schema.
|
||||
# @POST: Task is executed and logged.
|
||||
async def execute(self, params: Dict[str, Any]):
|
||||
with belief_scope("StoragePlugin:execute"):
|
||||
logger.info(f"[StoragePlugin][Action] Executing with params: {params}")
|
||||
# [/DEF:execute:Function]
|
||||
|
||||
# [DEF:get_storage_root:Function]
|
||||
# @PURPOSE: Resolves the absolute path to the storage root.
|
||||
# @PRE: Settings must define a storage root path.
|
||||
# @POST: Returns a Path object representing the storage root.
|
||||
def get_storage_root(self) -> Path:
|
||||
with belief_scope("StoragePlugin:get_storage_root"):
|
||||
config_manager = get_config_manager()
|
||||
global_settings = config_manager.get_config().settings
|
||||
|
||||
# Use storage.root_path as the source of truth for storage UI
|
||||
root = Path(global_settings.storage.root_path)
|
||||
|
||||
if not root.is_absolute():
|
||||
# Resolve relative to the backend directory
|
||||
# Path(__file__) is backend/src/plugins/storage/plugin.py
|
||||
# parents[3] is the project root (ss-tools)
|
||||
# We need to ensure it's relative to where backend/ is
|
||||
project_root = Path(__file__).parents[3]
|
||||
root = (project_root / root).resolve()
|
||||
return root
|
||||
# [/DEF:get_storage_root:Function]
|
||||
|
||||
# [DEF:resolve_path:Function]
|
||||
# @PURPOSE: Resolves a dynamic path pattern using provided variables.
|
||||
# @PARAM: pattern (str) - The path pattern to resolve.
|
||||
# @PARAM: variables (Dict[str, str]) - Variables to substitute in the pattern.
|
||||
# @PRE: pattern must be a valid format string.
|
||||
# @POST: Returns the resolved path string.
|
||||
# @RETURN: str - The resolved path.
|
||||
def resolve_path(self, pattern: str, variables: Dict[str, str]) -> str:
|
||||
with belief_scope("StoragePlugin:resolve_path"):
|
||||
# Add common variables
|
||||
vars_with_defaults = {
|
||||
"timestamp": datetime.now().strftime("%Y%m%dT%H%M%S"),
|
||||
**variables
|
||||
}
|
||||
try:
|
||||
resolved = pattern.format(**vars_with_defaults)
|
||||
# Clean up any double slashes or leading/trailing slashes for relative path
|
||||
return os.path.normpath(resolved).strip("/")
|
||||
except KeyError as e:
|
||||
logger.warning(f"[StoragePlugin][Coherence:Failed] Missing variable for path resolution: {e}")
|
||||
# Fallback to literal pattern if formatting fails partially (or handle as needed)
|
||||
return pattern.replace("{", "").replace("}", "")
|
||||
# [/DEF:resolve_path:Function]
|
||||
|
||||
# [DEF:ensure_directories:Function]
|
||||
# @PURPOSE: Creates the storage root and category subdirectories if they don't exist.
|
||||
# @PRE: Storage root must be resolvable.
|
||||
# @POST: Directories are created on the filesystem.
|
||||
# @SIDE_EFFECT: Creates directories on the filesystem.
|
||||
def ensure_directories(self):
|
||||
with belief_scope("StoragePlugin:ensure_directories"):
|
||||
root = self.get_storage_root()
|
||||
for category in FileCategory:
|
||||
# Use singular name for consistency with BackupPlugin and GitService
|
||||
path = root / category.value
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
logger.debug(f"[StoragePlugin][Action] Ensured directory: {path}")
|
||||
# [/DEF:ensure_directories:Function]
|
||||
|
||||
# [DEF:validate_path:Function]
|
||||
# @PURPOSE: Prevents path traversal attacks by ensuring the path is within the storage root.
|
||||
# @PRE: path must be a Path object.
|
||||
# @POST: Returns the resolved absolute path if valid, otherwise raises ValueError.
|
||||
def validate_path(self, path: Path) -> Path:
|
||||
with belief_scope("StoragePlugin:validate_path"):
|
||||
root = self.get_storage_root().resolve()
|
||||
resolved = path.resolve()
|
||||
try:
|
||||
resolved.relative_to(root)
|
||||
except ValueError:
|
||||
logger.error(f"[StoragePlugin][Coherence:Failed] Path traversal detected: {resolved} is not under {root}")
|
||||
raise ValueError("Access denied: Path is outside of storage root.")
|
||||
return resolved
|
||||
# [/DEF:validate_path:Function]
|
||||
|
||||
# [DEF:list_files:Function]
|
||||
# @PURPOSE: Lists all files and directories in a specific category and subpath.
|
||||
# @PARAM: category (Optional[FileCategory]) - The category to list.
|
||||
# @PARAM: subpath (Optional[str]) - Nested path within the category.
|
||||
# @PRE: Storage root must exist.
|
||||
# @POST: Returns a list of StoredFile objects.
|
||||
# @RETURN: List[StoredFile] - List of file and directory metadata objects.
|
||||
def list_files(self, category: Optional[FileCategory] = None, subpath: Optional[str] = None) -> List[StoredFile]:
|
||||
with belief_scope("StoragePlugin:list_files"):
|
||||
root = self.get_storage_root()
|
||||
logger.info(f"[StoragePlugin][Action] Listing files in root: {root}, category: {category}, subpath: {subpath}")
|
||||
files = []
|
||||
|
||||
categories = [category] if category else list(FileCategory)
|
||||
|
||||
for cat in categories:
|
||||
# Scan the category subfolder + optional subpath
|
||||
base_dir = root / cat.value
|
||||
if subpath:
|
||||
target_dir = self.validate_path(base_dir / subpath)
|
||||
else:
|
||||
target_dir = base_dir
|
||||
|
||||
if not target_dir.exists():
|
||||
continue
|
||||
|
||||
logger.debug(f"[StoragePlugin][Action] Scanning directory: {target_dir}")
|
||||
|
||||
# Use os.scandir for better performance and to distinguish files vs dirs
|
||||
with os.scandir(target_dir) as it:
|
||||
for entry in it:
|
||||
# Skip logs
|
||||
if "Logs" in entry.path:
|
||||
continue
|
||||
|
||||
stat = entry.stat()
|
||||
is_dir = entry.is_dir()
|
||||
|
||||
files.append(StoredFile(
|
||||
name=entry.name,
|
||||
path=str(Path(entry.path).relative_to(root)),
|
||||
size=stat.st_size if not is_dir else 0,
|
||||
created_at=datetime.fromtimestamp(stat.st_ctime),
|
||||
category=cat,
|
||||
mime_type="directory" if is_dir else None
|
||||
))
|
||||
|
||||
# Sort: directories first, then by name
|
||||
return sorted(files, key=lambda x: (x.mime_type != "directory", x.name))
|
||||
# [/DEF:list_files:Function]
|
||||
|
||||
# [DEF:save_file:Function]
|
||||
# @PURPOSE: Saves an uploaded file to the specified category and optional subpath.
|
||||
# @PARAM: file (UploadFile) - The uploaded file.
|
||||
# @PARAM: category (FileCategory) - The target category.
|
||||
# @PARAM: subpath (Optional[str]) - The target subpath.
|
||||
# @PRE: file must be a valid UploadFile; category must be valid.
|
||||
# @POST: File is written to disk and metadata is returned.
|
||||
# @RETURN: StoredFile - Metadata of the saved file.
|
||||
# @SIDE_EFFECT: Writes file to disk.
|
||||
async def save_file(self, file: UploadFile, category: FileCategory, subpath: Optional[str] = None) -> StoredFile:
|
||||
with belief_scope("StoragePlugin:save_file"):
|
||||
root = self.get_storage_root()
|
||||
dest_dir = root / category.value
|
||||
if subpath:
|
||||
dest_dir = dest_dir / subpath
|
||||
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
dest_path = self.validate_path(dest_dir / file.filename)
|
||||
|
||||
with dest_path.open("wb") as buffer:
|
||||
shutil.copyfileobj(file.file, buffer)
|
||||
|
||||
stat = dest_path.stat()
|
||||
return StoredFile(
|
||||
name=dest_path.name,
|
||||
path=str(dest_path.relative_to(root)),
|
||||
size=stat.st_size,
|
||||
created_at=datetime.fromtimestamp(stat.st_ctime),
|
||||
category=category,
|
||||
mime_type=file.content_type
|
||||
)
|
||||
# [/DEF:save_file:Function]
|
||||
|
||||
# [DEF:delete_file:Function]
|
||||
# @PURPOSE: Deletes a file or directory from the specified category and path.
|
||||
# @PARAM: category (FileCategory) - The category.
|
||||
# @PARAM: path (str) - The relative path of the file or directory.
|
||||
# @PRE: path must belong to the specified category and exist on disk.
|
||||
# @POST: The file or directory is removed from disk.
|
||||
# @SIDE_EFFECT: Removes item from disk.
|
||||
def delete_file(self, category: FileCategory, path: str):
|
||||
with belief_scope("StoragePlugin:delete_file"):
|
||||
root = self.get_storage_root()
|
||||
# path is relative to root, but we ensure it starts with category
|
||||
full_path = self.validate_path(root / path)
|
||||
|
||||
if not str(Path(path)).startswith(category.value):
|
||||
raise ValueError(f"Path {path} does not belong to category {category}")
|
||||
|
||||
if full_path.exists():
|
||||
if full_path.is_dir():
|
||||
shutil.rmtree(full_path)
|
||||
else:
|
||||
full_path.unlink()
|
||||
logger.info(f"[StoragePlugin][Action] Deleted: {full_path}")
|
||||
else:
|
||||
raise FileNotFoundError(f"Item {path} not found")
|
||||
# [/DEF:delete_file:Function]
|
||||
|
||||
# [DEF:get_file_path:Function]
|
||||
# @PURPOSE: Returns the absolute path of a file for download.
|
||||
# @PARAM: category (FileCategory) - The category.
|
||||
# @PARAM: path (str) - The relative path of the file.
|
||||
# @PRE: path must belong to the specified category and be a file.
|
||||
# @POST: Returns the absolute Path to the file.
|
||||
# @RETURN: Path - Absolute path to the file.
|
||||
def get_file_path(self, category: FileCategory, path: str) -> Path:
|
||||
with belief_scope("StoragePlugin:get_file_path"):
|
||||
root = self.get_storage_root()
|
||||
file_path = self.validate_path(root / path)
|
||||
|
||||
if not str(Path(path)).startswith(category.value):
|
||||
raise ValueError(f"Path {path} does not belong to category {category}")
|
||||
|
||||
if not file_path.exists() or file_path.is_dir():
|
||||
raise FileNotFoundError(f"File {path} not found")
|
||||
|
||||
return file_path
|
||||
# [/DEF:get_file_path:Function]
|
||||
|
||||
# [/DEF:StoragePlugin:Class]
|
||||
# [/DEF:StoragePlugin:Module]
|
||||
@@ -31,9 +31,15 @@ class GitService:
|
||||
# @PARAM: base_path (str) - Root directory for all Git clones.
|
||||
# @PRE: base_path is a valid string path.
|
||||
# @POST: GitService is initialized; base_path directory exists.
|
||||
def __init__(self, base_path: str = "backend/git_repos"):
|
||||
def __init__(self, base_path: str = "git_repos"):
|
||||
with belief_scope("GitService.__init__"):
|
||||
self.base_path = base_path
|
||||
# Resolve relative to the backend directory
|
||||
# Path(__file__) is backend/src/services/git_service.py
|
||||
# parents[2] is backend/
|
||||
from pathlib import Path
|
||||
backend_root = Path(__file__).parents[2]
|
||||
|
||||
self.base_path = str((backend_root / base_path).resolve())
|
||||
if not os.path.exists(self.base_path):
|
||||
os.makedirs(self.base_path)
|
||||
# [/DEF:__init__:Function]
|
||||
|
||||
BIN
backend/tasks.db
Normal file
BIN
backend/tasks.db
Normal file
Binary file not shown.
@@ -13,7 +13,7 @@ The settings mechanism allows users to configure multiple Superset environments
|
||||
Configuration is structured using Pydantic models in `backend/src/core/config_models.py`:
|
||||
|
||||
- `Environment`: Represents a Superset instance (URL, credentials). The `base_url` is automatically normalized to include the `/api/v1` suffix if missing.
|
||||
- `GlobalSettings`: Global application parameters (e.g., `backup_path`).
|
||||
- `GlobalSettings`: Global application parameters (e.g., `storage.root_path`).
|
||||
- `AppConfig`: The root configuration object.
|
||||
|
||||
### Configuration Manager
|
||||
@@ -43,4 +43,4 @@ The settings page is located at `frontend/src/pages/Settings.svelte`. It provide
|
||||
|
||||
Existing plugins and utilities use the `ConfigManager` to fetch configuration:
|
||||
- `superset_tool/utils/init_clients.py`: Dynamically initializes Superset clients from the configured environments.
|
||||
- `BackupPlugin`: Uses the configured `backup_path` as the default storage location.
|
||||
- `BackupPlugin`: Uses the configured `storage.root_path` as the default storage location.
|
||||
|
||||
@@ -57,4 +57,4 @@
|
||||
/* Component specific styles */
|
||||
</style>
|
||||
|
||||
<!-- [/DEF:EnvSelector:Component] -->
|
||||
<!-- [/DEF:EnvSelector:Component] -->
|
||||
@@ -25,34 +25,12 @@
|
||||
>
|
||||
{$t.nav.dashboard}
|
||||
</a>
|
||||
<a
|
||||
href="/migration"
|
||||
class="text-gray-600 hover:text-blue-600 font-medium {$page.url.pathname.startsWith('/migration') ? 'text-blue-600 border-b-2 border-blue-600' : ''}"
|
||||
>
|
||||
{$t.nav.migration}
|
||||
</a>
|
||||
<a
|
||||
href="/git"
|
||||
class="text-gray-600 hover:text-blue-600 font-medium {$page.url.pathname.startsWith('/git') ? 'text-blue-600 border-b-2 border-blue-600' : ''}"
|
||||
>
|
||||
{$t.nav.git}
|
||||
</a>
|
||||
<a
|
||||
href="/tasks"
|
||||
class="text-gray-600 hover:text-blue-600 font-medium {$page.url.pathname.startsWith('/tasks') ? 'text-blue-600 border-b-2 border-blue-600' : ''}"
|
||||
>
|
||||
{$t.nav.tasks}
|
||||
</a>
|
||||
<div class="relative inline-block group">
|
||||
<button class="text-gray-600 hover:text-blue-600 font-medium pb-1 {$page.url.pathname.startsWith('/tools') ? 'text-blue-600 border-b-2 border-blue-600' : ''}">
|
||||
{$t.nav.tools}
|
||||
</button>
|
||||
<div class="absolute hidden group-hover:block bg-white shadow-lg rounded-md mt-1 py-2 w-48 z-10 border border-gray-100 before:absolute before:-top-2 before:left-0 before:right-0 before:h-2 before:content-[''] right-0">
|
||||
<a href="/tools/search" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.tools_search}</a>
|
||||
<a href="/tools/mapper" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.tools_mapper}</a>
|
||||
<a href="/tools/debug" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.tools_debug}</a>
|
||||
</div>
|
||||
</div>
|
||||
<div class="relative inline-block group">
|
||||
<button class="text-gray-600 hover:text-blue-600 font-medium pb-1 {$page.url.pathname.startsWith('/settings') ? 'text-blue-600 border-b-2 border-blue-600' : ''}">
|
||||
{$t.nav.settings}
|
||||
@@ -61,7 +39,6 @@
|
||||
<a href="/settings" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.settings_general}</a>
|
||||
<a href="/settings/connections" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.settings_connections}</a>
|
||||
<a href="/settings/git" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.settings_git}</a>
|
||||
<a href="/settings/environments" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.settings_environments}</a>
|
||||
</div>
|
||||
</div>
|
||||
<LanguageSwitcher />
|
||||
|
||||
84
frontend/src/components/backups/BackupList.svelte
Normal file
84
frontend/src/components/backups/BackupList.svelte
Normal file
@@ -0,0 +1,84 @@
|
||||
<!-- [DEF:BackupList:Component] -->
|
||||
<!--
|
||||
@SEMANTICS: backup, list, table
|
||||
@PURPOSE: Displays a list of existing backups.
|
||||
@LAYER: Component
|
||||
@RELATION: USED_BY -> frontend/src/components/backups/BackupManager.svelte
|
||||
-->
|
||||
|
||||
<script lang="ts">
|
||||
// [SECTION: IMPORTS]
|
||||
import { t } from '../../lib/i18n';
|
||||
import { Button } from '../../lib/ui';
|
||||
import type { Backup } from '../../types/backup';
|
||||
import { goto } from '$app/navigation';
|
||||
// [/SECTION]
|
||||
|
||||
// [SECTION: PROPS]
|
||||
/**
|
||||
* @type {Backup[]}
|
||||
* @description Array of backup objects to display.
|
||||
*/
|
||||
export let backups: Backup[] = [];
|
||||
// [/SECTION]
|
||||
|
||||
</script>
|
||||
|
||||
<!-- [SECTION: TEMPLATE] -->
|
||||
<div class="overflow-x-auto rounded-lg border border-gray-200">
|
||||
<table class="min-w-full divide-y divide-gray-200">
|
||||
<thead class="bg-gray-50">
|
||||
<tr>
|
||||
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
|
||||
{$t.storage.table.name}
|
||||
</th>
|
||||
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
|
||||
{$t.tasks.target_env}
|
||||
</th>
|
||||
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
|
||||
{$t.storage.table.created_at}
|
||||
</th>
|
||||
<th class="px-6 py-3 text-right text-xs font-medium text-gray-500 uppercase tracking-wider">
|
||||
{$t.storage.table.actions}
|
||||
</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody class="bg-white divide-y divide-gray-200">
|
||||
{#each backups as backup}
|
||||
<tr class="hover:bg-gray-50 transition-colors">
|
||||
<td class="px-6 py-4 whitespace-nowrap text-sm font-medium text-gray-900">
|
||||
{backup.name}
|
||||
</td>
|
||||
<td class="px-6 py-4 whitespace-nowrap text-sm text-gray-500">
|
||||
{backup.environment}
|
||||
</td>
|
||||
<td class="px-6 py-4 whitespace-nowrap text-sm text-gray-500">
|
||||
{new Date(backup.created_at).toLocaleString()}
|
||||
</td>
|
||||
<td class="px-6 py-4 whitespace-nowrap text-right text-sm font-medium">
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
class="text-blue-600 hover:text-blue-900"
|
||||
on:click={() => goto(`/tools/storage?path=backups/${backup.name}`)}
|
||||
>
|
||||
{$t.storage.table.go_to_storage}
|
||||
</Button>
|
||||
</td>
|
||||
</tr>
|
||||
{:else}
|
||||
<tr>
|
||||
<td colspan="4" class="px-6 py-10 text-center text-gray-500">
|
||||
{$t.storage.no_files}
|
||||
</td>
|
||||
</tr>
|
||||
{/each}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<!-- [/SECTION] -->
|
||||
|
||||
<style>
|
||||
</style>
|
||||
|
||||
<!-- [/DEF:BackupList:Component] -->
|
||||
241
frontend/src/components/backups/BackupManager.svelte
Normal file
241
frontend/src/components/backups/BackupManager.svelte
Normal file
@@ -0,0 +1,241 @@
|
||||
<!-- [DEF:BackupManager:Component] -->
|
||||
<!--
|
||||
@SEMANTICS: backup, manager, orchestrator
|
||||
@PURPOSE: Main container for backup management, handling creation and listing.
|
||||
@LAYER: Feature
|
||||
@RELATION: USES -> BackupList
|
||||
@RELATION: USES -> api
|
||||
|
||||
@INVARIANT: Only one backup task can be triggered at a time from the UI.
|
||||
-->
|
||||
|
||||
<script lang="ts">
|
||||
// [SECTION: IMPORTS]
|
||||
import { onMount } from 'svelte';
|
||||
import { t } from '../../lib/i18n';
|
||||
import { api, requestApi } from '../../lib/api';
|
||||
import { addToast } from '../../lib/toasts';
|
||||
import { Button, Card, Select, Input } from '../../lib/ui';
|
||||
import BackupList from './BackupList.svelte';
|
||||
import type { Backup } from '../../types/backup';
|
||||
// [/SECTION]
|
||||
|
||||
// [SECTION: STATE]
|
||||
let backups: Backup[] = [];
|
||||
let environments: any[] = [];
|
||||
let selectedEnvId = '';
|
||||
let loading = true;
|
||||
let creating = false;
|
||||
let savingSchedule = false;
|
||||
|
||||
// Schedule state for selected environment
|
||||
let scheduleEnabled = false;
|
||||
let cronExpression = '0 0 * * *';
|
||||
|
||||
$: selectedEnv = environments.find(e => e.id === selectedEnvId);
|
||||
$: if (selectedEnv) {
|
||||
scheduleEnabled = selectedEnv.backup_schedule?.enabled ?? false;
|
||||
cronExpression = selectedEnv.backup_schedule?.cron_expression ?? '0 0 * * *';
|
||||
}
|
||||
// [/SECTION]
|
||||
|
||||
// [DEF:loadData:Function]
|
||||
/**
|
||||
* @purpose Loads backups and environments from the backend.
|
||||
*
|
||||
* @pre API must be reachable.
|
||||
* @post environments and backups stores are populated.
|
||||
*
|
||||
* @returns {Promise<void>}
|
||||
* @side_effect Updates local state variables.
|
||||
*/
|
||||
// @RELATION: CALLS -> api.getEnvironmentsList
|
||||
// @RELATION: CALLS -> api.requestApi
|
||||
async function loadData() {
|
||||
console.log("[BackupManager][Entry] Loading data.");
|
||||
loading = true;
|
||||
try {
|
||||
const [envsData, storageData] = await Promise.all([
|
||||
api.getEnvironmentsList(),
|
||||
requestApi('/storage/files?category=backups')
|
||||
]);
|
||||
environments = envsData;
|
||||
|
||||
// Map storage files to Backup type
|
||||
backups = (storageData || []).map((file: any) => ({
|
||||
id: file.name,
|
||||
name: file.name,
|
||||
environment: file.path.split('/')[0] || 'Unknown',
|
||||
created_at: file.created_at,
|
||||
size_bytes: file.size,
|
||||
status: 'success'
|
||||
}));
|
||||
console.log("[BackupManager][Action] Data loaded successfully.");
|
||||
} catch (error) {
|
||||
console.error("[BackupManager][Coherence:Failed] Load failed", error);
|
||||
} finally {
|
||||
loading = false;
|
||||
}
|
||||
}
|
||||
// [/DEF:loadData:Function]
|
||||
|
||||
// [DEF:handleCreateBackup:Function]
|
||||
/**
|
||||
* @purpose Triggers a new backup task for the selected environment.
|
||||
*
|
||||
* @pre selectedEnvId must be a valid environment ID.
|
||||
* @post A new task is created on the backend.
|
||||
*
|
||||
* @returns {Promise<void>}
|
||||
* @side_effect Dispatches a toast notification.
|
||||
*/
|
||||
// @RELATION: CALLS -> api.createTask
|
||||
// [DEF:handleUpdateSchedule:Function]
|
||||
/**
|
||||
* @purpose Updates the backup schedule for the selected environment.
|
||||
* @pre selectedEnvId must be set.
|
||||
* @post Environment config is updated on the backend.
|
||||
*/
|
||||
async function handleUpdateSchedule() {
|
||||
if (!selectedEnvId) return;
|
||||
|
||||
console.log(`[BackupManager][Action] Updating schedule for env: ${selectedEnvId}`);
|
||||
savingSchedule = true;
|
||||
try {
|
||||
await api.updateEnvironmentSchedule(selectedEnvId, {
|
||||
enabled: scheduleEnabled,
|
||||
cron_expression: cronExpression
|
||||
});
|
||||
addToast($t.common.success, 'success');
|
||||
|
||||
// Update local state
|
||||
environments = environments.map(e =>
|
||||
e.id === selectedEnvId
|
||||
? { ...e, backup_schedule: { enabled: scheduleEnabled, cron_expression: cronExpression } }
|
||||
: e
|
||||
);
|
||||
} catch (error) {
|
||||
console.error("[BackupManager][Coherence:Failed] Schedule update failed", error);
|
||||
} finally {
|
||||
savingSchedule = false;
|
||||
}
|
||||
}
|
||||
// [/DEF:handleUpdateSchedule:Function]
|
||||
|
||||
async function handleCreateBackup() {
|
||||
if (!selectedEnvId) {
|
||||
addToast($t.tasks.select_env, 'error');
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`[BackupManager][Action] Triggering backup for env: ${selectedEnvId}`);
|
||||
creating = true;
|
||||
try {
|
||||
await api.createTask('superset-backup', { environment_id: selectedEnvId });
|
||||
addToast($t.common.success, 'success');
|
||||
console.log("[BackupManager][Coherence:OK] Backup task triggered.");
|
||||
} catch (error) {
|
||||
console.error("[BackupManager][Coherence:Failed] Create failed", error);
|
||||
} finally {
|
||||
creating = false;
|
||||
}
|
||||
}
|
||||
// [/DEF:handleCreateBackup:Function]
|
||||
|
||||
onMount(loadData);
|
||||
</script>
|
||||
|
||||
<!-- [SECTION: TEMPLATE] -->
|
||||
<div class="space-y-6">
|
||||
<Card title={$t.tasks.manual_backup}>
|
||||
<div class="space-y-4">
|
||||
<div class="flex items-end gap-4">
|
||||
<div class="flex-1">
|
||||
<Select
|
||||
label={$t.tasks.target_env}
|
||||
bind:value={selectedEnvId}
|
||||
options={[
|
||||
{ value: '', label: $t.tasks.select_env },
|
||||
...environments.map(e => ({ value: e.id, label: e.name }))
|
||||
]}
|
||||
/>
|
||||
</div>
|
||||
<Button
|
||||
variant="primary"
|
||||
on:click={handleCreateBackup}
|
||||
disabled={creating || !selectedEnvId}
|
||||
>
|
||||
{creating ? $t.common.loading : $t.tasks.start_backup}
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{#if selectedEnvId}
|
||||
<div class="pt-6 border-t border-gray-100 mt-4">
|
||||
<h3 class="text-sm font-semibold text-gray-800 mb-4 flex items-center gap-2">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" class="h-4 w-4 text-blue-500" fill="none" viewBox="0 0 24 24" stroke="currentColor">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 8v4l3 3m6-3a9 9 0 11-18 0 9 9 0 0118 0z" />
|
||||
</svg>
|
||||
{$t.tasks.backup_schedule}
|
||||
</h3>
|
||||
|
||||
<div class="bg-gray-50 rounded-lg p-4 border border-gray-200">
|
||||
<div class="flex flex-col md:flex-row md:items-start gap-6">
|
||||
<div class="pt-8">
|
||||
<label class="flex items-center gap-3 cursor-pointer group">
|
||||
<div class="relative inline-flex items-center">
|
||||
<input type="checkbox" bind:checked={scheduleEnabled} class="sr-only peer" />
|
||||
<div class="w-11 h-6 bg-gray-200 peer-focus:outline-none peer-focus:ring-4 peer-focus:ring-blue-300 rounded-full peer peer-checked:after:translate-x-full peer-checked:after:border-white after:content-[''] after:absolute after:top-[2px] after:left-[2px] after:bg-white after:border-gray-300 after:border after:rounded-full after:h-5 after:w-5 after:transition-all peer-checked:bg-blue-600"></div>
|
||||
<span class="ml-3 text-sm font-medium text-gray-700 group-hover:text-gray-900 transition-colors">{$t.tasks.schedule_enabled}</span>
|
||||
</div>
|
||||
</label>
|
||||
</div>
|
||||
|
||||
<div class="flex-1 space-y-2">
|
||||
<Input
|
||||
label={$t.tasks.cron_label}
|
||||
placeholder="0 0 * * *"
|
||||
bind:value={cronExpression}
|
||||
disabled={!scheduleEnabled}
|
||||
/>
|
||||
<p class="text-xs text-gray-500 italic">{$t.tasks.cron_hint}</p>
|
||||
</div>
|
||||
|
||||
<div class="pt-8">
|
||||
<Button
|
||||
variant="secondary"
|
||||
on:click={handleUpdateSchedule}
|
||||
disabled={savingSchedule}
|
||||
class="min-w-[100px]"
|
||||
>
|
||||
{#if savingSchedule}
|
||||
<span class="flex items-center gap-2">
|
||||
<svg class="animate-spin h-4 w-4" viewBox="0 0 24 24">
|
||||
<circle class="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" stroke-width="4" fill="none"></circle>
|
||||
<path class="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"></path>
|
||||
</svg>
|
||||
{$t.common.loading}
|
||||
</span>
|
||||
{:else}
|
||||
{$t.common.save}
|
||||
{/if}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{/if}
|
||||
</div>
|
||||
</Card>
|
||||
|
||||
<div class="space-y-3">
|
||||
<h2 class="text-lg font-semibold text-gray-700">{$t.storage.backups}</h2>
|
||||
{#if loading}
|
||||
<div class="py-10 text-center text-gray-500">{$t.common.loading}</div>
|
||||
{:else}
|
||||
<BackupList {backups} />
|
||||
{/if}
|
||||
</div>
|
||||
</div>
|
||||
<!-- [/SECTION] -->
|
||||
|
||||
<!-- [/DEF:BackupManager:Component] -->
|
||||
134
frontend/src/components/storage/FileList.svelte
Normal file
134
frontend/src/components/storage/FileList.svelte
Normal file
@@ -0,0 +1,134 @@
|
||||
<!-- [DEF:FileList:Component] -->
|
||||
<!--
|
||||
@SEMANTICS: storage, files, list, table
|
||||
@PURPOSE: Displays a table of files with metadata and actions.
|
||||
@LAYER: Component
|
||||
@RELATION: DEPENDS_ON -> storageService
|
||||
|
||||
@PROPS: files (Array) - List of StoredFile objects.
|
||||
@EVENTS: delete (filename) - Dispatched when a file is deleted.
|
||||
-->
|
||||
|
||||
<script lang="ts">
|
||||
// [SECTION: IMPORTS]
|
||||
import { createEventDispatcher } from 'svelte';
|
||||
import { downloadFileUrl } from '../../services/storageService';
|
||||
import { t } from '../../lib/i18n';
|
||||
// [/SECTION: IMPORTS]
|
||||
|
||||
export let files = [];
|
||||
const dispatch = createEventDispatcher();
|
||||
|
||||
// [DEF:isDirectory:Function]
|
||||
/**
|
||||
* @purpose Checks if a file object represents a directory.
|
||||
* @param {Object} file - The file object to check.
|
||||
* @return {boolean} True if it's a directory, false otherwise.
|
||||
*/
|
||||
function isDirectory(file) {
|
||||
return file.mime_type === 'directory';
|
||||
}
|
||||
// [/DEF:isDirectory:Function]
|
||||
|
||||
// [DEF:formatSize:Function]
|
||||
/**
|
||||
* @purpose Formats file size in bytes into a human-readable string.
|
||||
* @param {number} bytes - The size in bytes.
|
||||
* @return {string} Formatted size (e.g., "1.2 MB").
|
||||
*/
|
||||
function formatSize(bytes) {
|
||||
if (bytes === 0) return '0 B';
|
||||
const k = 1024;
|
||||
const sizes = ['B', 'KB', 'MB', 'GB', 'TB'];
|
||||
const i = Math.floor(Math.log(bytes) / Math.log(k));
|
||||
return parseFloat((bytes / Math.pow(k, i)).toFixed(2)) + ' ' + sizes[i];
|
||||
}
|
||||
// [/DEF:formatSize:Function]
|
||||
|
||||
// [DEF:formatDate:Function]
|
||||
/**
|
||||
* @purpose Formats an ISO date string into a localized readable format.
|
||||
* @param {string} dateStr - The date string to format.
|
||||
* @return {string} Localized date and time.
|
||||
*/
|
||||
function formatDate(dateStr) {
|
||||
return new Date(dateStr).toLocaleString();
|
||||
}
|
||||
// [/DEF:formatDate:Function]
|
||||
</script>
|
||||
|
||||
<!-- [SECTION: TEMPLATE] -->
|
||||
<div class="overflow-x-auto">
|
||||
<table class="min-w-full bg-white border border-gray-200">
|
||||
<thead class="bg-gray-50">
|
||||
<tr>
|
||||
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">{$t.storage.table.name}</th>
|
||||
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">{$t.storage.table.category}</th>
|
||||
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">{$t.storage.table.size}</th>
|
||||
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">{$t.storage.table.created_at}</th>
|
||||
<th class="px-6 py-3 text-right text-xs font-medium text-gray-500 uppercase tracking-wider">{$t.storage.table.actions}</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody class="divide-y divide-gray-200">
|
||||
{#each files as file}
|
||||
<tr class="hover:bg-gray-50">
|
||||
<td class="px-6 py-4 whitespace-nowrap text-sm font-medium text-gray-900">
|
||||
{#if isDirectory(file)}
|
||||
<button
|
||||
on:click={() => dispatch('navigate', file.path)}
|
||||
class="flex items-center text-indigo-600 hover:text-indigo-900"
|
||||
>
|
||||
<svg class="h-5 w-5 mr-2 text-yellow-400" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path d="M2 6a2 2 0 012-2h5l2 2h5a2 2 0 012 2v6a2 2 0 01-2 2H4a2 2 0 01-2-2V6z" />
|
||||
</svg>
|
||||
{file.name}
|
||||
</button>
|
||||
{:else}
|
||||
<div class="flex items-center">
|
||||
<svg class="h-5 w-5 mr-2 text-gray-400" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M7 21h10a2 2 0 002-2V9.414a1 1 0 00-.293-.707l-5.414-5.414A1 1 0 0012.586 3H7a2 2 0 00-2 2v14a2 2 0 002 2z" />
|
||||
</svg>
|
||||
{file.name}
|
||||
</div>
|
||||
{/if}
|
||||
</td>
|
||||
<td class="px-6 py-4 whitespace-nowrap text-sm text-gray-500 capitalize">{file.category}</td>
|
||||
<td class="px-6 py-4 whitespace-nowrap text-sm text-gray-500">
|
||||
{isDirectory(file) ? '--' : formatSize(file.size)}
|
||||
</td>
|
||||
<td class="px-6 py-4 whitespace-nowrap text-sm text-gray-500">{formatDate(file.created_at)}</td>
|
||||
<td class="px-6 py-4 whitespace-nowrap text-right text-sm font-medium">
|
||||
{#if !isDirectory(file)}
|
||||
<a
|
||||
href={downloadFileUrl(file.category, file.path)}
|
||||
download={file.name}
|
||||
class="text-indigo-600 hover:text-indigo-900 mr-4"
|
||||
>
|
||||
{$t.storage.table.download}
|
||||
</a>
|
||||
{/if}
|
||||
<button
|
||||
on:click={() => dispatch('delete', { category: file.category, path: file.path, name: file.name })}
|
||||
class="text-red-600 hover:text-red-900"
|
||||
>
|
||||
{$t.storage.table.delete}
|
||||
</button>
|
||||
</td>
|
||||
</tr>
|
||||
{:else}
|
||||
<tr>
|
||||
<td colspan="5" class="px-6 py-10 text-center text-sm text-gray-500">
|
||||
{$t.storage.no_files}
|
||||
</td>
|
||||
</tr>
|
||||
{/each}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<!-- [/SECTION: TEMPLATE] -->
|
||||
|
||||
<style>
|
||||
/* ... */
|
||||
</style>
|
||||
|
||||
<!-- [/DEF:FileList:Component] -->
|
||||
134
frontend/src/components/storage/FileUpload.svelte
Normal file
134
frontend/src/components/storage/FileUpload.svelte
Normal file
@@ -0,0 +1,134 @@
|
||||
<!-- [DEF:FileUpload:Component] -->
|
||||
<!--
|
||||
@SEMANTICS: storage, upload, files
|
||||
@PURPOSE: Provides a form for uploading files to a specific category.
|
||||
@LAYER: Component
|
||||
@RELATION: DEPENDS_ON -> storageService
|
||||
|
||||
@PROPS: None
|
||||
@EVENTS: uploaded - Dispatched when a file is successfully uploaded.
|
||||
-->
|
||||
|
||||
<script lang="ts">
|
||||
// [SECTION: IMPORTS]
|
||||
import { createEventDispatcher } from 'svelte';
|
||||
import { uploadFile } from '../../services/storageService';
|
||||
import { addToast } from '../../lib/toasts';
|
||||
import { t } from '../../lib/i18n';
|
||||
// [/SECTION: IMPORTS]
|
||||
|
||||
// [DEF:handleUpload:Function]
|
||||
/**
|
||||
* @purpose Handles the file upload process.
|
||||
* @pre A file must be selected in the file input.
|
||||
* @post The file is uploaded to the server and a success toast is shown.
|
||||
*/
|
||||
const dispatch = createEventDispatcher();
|
||||
let fileInput;
|
||||
export let category = 'backups';
|
||||
export let path = '';
|
||||
let isUploading = false;
|
||||
let dragOver = false;
|
||||
|
||||
async function handleUpload() {
|
||||
const file = fileInput.files[0];
|
||||
if (!file) return;
|
||||
|
||||
isUploading = true;
|
||||
try {
|
||||
// path is relative to root, but upload endpoint expects path within category
|
||||
// FileList.path is like "backup/folder", we need just "folder"
|
||||
const subpath = path.startsWith(category)
|
||||
? path.substring(category.length).replace(/^\/+/, '')
|
||||
: path;
|
||||
|
||||
await uploadFile(file, category, subpath);
|
||||
addToast($t.storage.messages.upload_success.replace('{name}', file.name), 'success');
|
||||
fileInput.value = '';
|
||||
dispatch('uploaded');
|
||||
} catch (error) {
|
||||
addToast($t.storage.messages.upload_failed.replace('{error}', error.message), 'error');
|
||||
} finally {
|
||||
isUploading = false;
|
||||
}
|
||||
}
|
||||
// [/DEF:handleUpload:Function]
|
||||
|
||||
// [DEF:handleDrop:Function]
|
||||
/**
|
||||
* @purpose Handles the file drop event for drag-and-drop.
|
||||
* @param {DragEvent} event - The drop event.
|
||||
*/
|
||||
function handleDrop(event) {
|
||||
event.preventDefault();
|
||||
dragOver = false;
|
||||
const files = event.dataTransfer.files;
|
||||
if (files.length > 0) {
|
||||
fileInput.files = files;
|
||||
handleUpload();
|
||||
}
|
||||
}
|
||||
// [/DEF:handleDrop:Function]
|
||||
</script>
|
||||
|
||||
<!-- [SECTION: TEMPLATE] -->
|
||||
<div class="bg-white p-6 rounded-lg border border-gray-200 shadow-sm">
|
||||
<h2 class="text-lg font-semibold mb-4">{$t.storage.upload_title}</h2>
|
||||
|
||||
<div class="space-y-4">
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-gray-700 mb-1">{$t.storage.target_category}</label>
|
||||
<select
|
||||
bind:value={category}
|
||||
class="block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm"
|
||||
>
|
||||
<option value="backups">{$t.storage.backups}</option>
|
||||
<option value="repositorys">{$t.storage.repositories}</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<div
|
||||
class="mt-1 flex justify-center px-6 pt-5 pb-6 border-2 border-dashed rounded-md transition-colors
|
||||
{dragOver ? 'border-indigo-500 bg-indigo-50' : 'border-gray-300'}"
|
||||
on:dragover|preventDefault={() => dragOver = true}
|
||||
on:dragleave|preventDefault={() => dragOver = false}
|
||||
on:drop|preventDefault={handleDrop}
|
||||
>
|
||||
<div class="space-y-1 text-center">
|
||||
<svg class="mx-auto h-12 w-12 text-gray-400" stroke="currentColor" fill="none" viewBox="0 0 48 48" aria-hidden="true">
|
||||
<path d="M28 8H12a4 4 0 00-4 4v20m32-12v8m0 0v8a4 4 0 01-4 4H12a4 4 0 01-4-4v-4m32-4l-3.172-3.172a4 4 0 00-5.656 0L28 28M8 32l9.172-9.172a4 4 0 015.656 0L28 28m0 0l4 4m4-24h8m-4-4v8m-12 4h.02" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" />
|
||||
</svg>
|
||||
<div class="flex text-sm text-gray-600">
|
||||
<label for="file-upload" class="relative cursor-pointer bg-white rounded-md font-medium text-indigo-600 hover:text-indigo-500 focus-within:outline-none focus-within:ring-2 focus-within:ring-offset-2 focus-within:ring-indigo-500">
|
||||
<span>{$t.storage.upload_button}</span>
|
||||
<input
|
||||
id="file-upload"
|
||||
name="file-upload"
|
||||
type="file"
|
||||
class="sr-only"
|
||||
bind:this={fileInput}
|
||||
on:change={handleUpload}
|
||||
disabled={isUploading}
|
||||
>
|
||||
</label>
|
||||
<p class="pl-1">{$t.storage.drag_drop}</p>
|
||||
</div>
|
||||
<p class="text-xs text-gray-500">{$t.storage.supported_formats}</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{#if isUploading}
|
||||
<div class="flex items-center justify-center space-x-2 text-indigo-600">
|
||||
<div class="animate-spin rounded-full h-4 w-4 border-b-2 border-indigo-600"></div>
|
||||
<span class="text-sm font-medium">{$t.storage.uploading}</span>
|
||||
</div>
|
||||
{/if}
|
||||
</div>
|
||||
</div>
|
||||
<!-- [/SECTION: TEMPLATE] -->
|
||||
|
||||
<style>
|
||||
/* ... */
|
||||
</style>
|
||||
|
||||
<!-- [/DEF:FileUpload:Component] -->
|
||||
@@ -13,6 +13,8 @@
|
||||
import { getConnections } from '../../services/connectionService.js';
|
||||
import { selectedTask } from '../../lib/stores.js';
|
||||
import { addToast } from '../../lib/toasts.js';
|
||||
import { t } from '../../lib/i18n';
|
||||
import { Button, Card, Select, Input } from '../../lib/ui';
|
||||
// [/SECTION]
|
||||
|
||||
let envs = [];
|
||||
@@ -36,7 +38,7 @@
|
||||
envs = await envsRes.json();
|
||||
connections = await getConnections();
|
||||
} catch (e) {
|
||||
addToast('Failed to fetch data', 'error');
|
||||
addToast($t.mapper.errors.fetch_failed, 'error');
|
||||
}
|
||||
}
|
||||
// [/DEF:fetchData:Function]
|
||||
@@ -47,17 +49,17 @@
|
||||
// @POST: Mapper task is started and selectedTask is updated.
|
||||
async function handleRunMapper() {
|
||||
if (!selectedEnv || !datasetId) {
|
||||
addToast('Please fill in required fields', 'warning');
|
||||
addToast($t.mapper.errors.required_fields, 'warning');
|
||||
return;
|
||||
}
|
||||
|
||||
if (source === 'postgres' && (!selectedConnection || !tableName)) {
|
||||
addToast('Connection and Table Name are required for postgres source', 'warning');
|
||||
addToast($t.mapper.errors.postgres_required, 'warning');
|
||||
return;
|
||||
}
|
||||
|
||||
if (source === 'excel' && !excelPath) {
|
||||
addToast('Excel path is required for excel source', 'warning');
|
||||
addToast($t.mapper.errors.excel_required, 'warning');
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -75,7 +77,7 @@
|
||||
});
|
||||
|
||||
selectedTask.set(task);
|
||||
addToast('Mapper task started', 'success');
|
||||
addToast($t.mapper.success.started, 'success');
|
||||
} catch (e) {
|
||||
addToast(e.message, 'error');
|
||||
} finally {
|
||||
@@ -88,78 +90,94 @@
|
||||
</script>
|
||||
|
||||
<!-- [SECTION: TEMPLATE] -->
|
||||
<div class="bg-white p-6 rounded-lg shadow-sm border border-gray-200">
|
||||
<h3 class="text-lg font-medium text-gray-900 mb-4">Dataset Column Mapper</h3>
|
||||
<div class="space-y-4">
|
||||
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">
|
||||
<div>
|
||||
<label for="mapper-env" class="block text-sm font-medium text-gray-700">Environment</label>
|
||||
<select id="mapper-env" bind:value={selectedEnv} class="mt-1 block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
|
||||
<option value="" disabled>-- Select Environment --</option>
|
||||
{#each envs as env}
|
||||
<option value={env.id}>{env.name}</option>
|
||||
{/each}
|
||||
</select>
|
||||
</div>
|
||||
<div>
|
||||
<label for="mapper-ds-id" class="block text-sm font-medium text-gray-700">Dataset ID</label>
|
||||
<input type="number" id="mapper-ds-id" bind:value={datasetId} class="mt-1 block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" />
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-gray-700">Mapping Source</label>
|
||||
<div class="mt-2 flex space-x-4">
|
||||
<label class="inline-flex items-center">
|
||||
<input type="radio" bind:group={source} value="postgres" class="focus:ring-indigo-500 h-4 w-4 text-indigo-600 border-gray-300" />
|
||||
<span class="ml-2 text-sm text-gray-700">PostgreSQL</span>
|
||||
</label>
|
||||
<label class="inline-flex items-center">
|
||||
<input type="radio" bind:group={source} value="excel" class="focus:ring-indigo-500 h-4 w-4 text-indigo-600 border-gray-300" />
|
||||
<span class="ml-2 text-sm text-gray-700">Excel</span>
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{#if source === 'postgres'}
|
||||
<div class="space-y-4 p-4 bg-gray-50 rounded-md border border-gray-100">
|
||||
<div class="space-y-6">
|
||||
<Card title={$t.mapper.title}>
|
||||
<div class="space-y-4">
|
||||
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">
|
||||
<div>
|
||||
<label for="mapper-conn" class="block text-sm font-medium text-gray-700">Saved Connection</label>
|
||||
<select id="mapper-conn" bind:value={selectedConnection} class="mt-1 block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
|
||||
<option value="" disabled>-- Select Connection --</option>
|
||||
{#each connections as conn}
|
||||
<option value={conn.id}>{conn.name}</option>
|
||||
{/each}
|
||||
</select>
|
||||
<Select
|
||||
label={$t.mapper.environment}
|
||||
bind:value={selectedEnv}
|
||||
options={[
|
||||
{ value: '', label: $t.mapper.select_env },
|
||||
...envs.map(e => ({ value: e.id, label: e.name }))
|
||||
]}
|
||||
/>
|
||||
</div>
|
||||
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">
|
||||
<div>
|
||||
<label for="mapper-table" class="block text-sm font-medium text-gray-700">Table Name</label>
|
||||
<input type="text" id="mapper-table" bind:value={tableName} class="mt-1 block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" />
|
||||
</div>
|
||||
<div>
|
||||
<label for="mapper-schema" class="block text-sm font-medium text-gray-700">Table Schema</label>
|
||||
<input type="text" id="mapper-schema" bind:value={tableSchema} class="mt-1 block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" />
|
||||
</div>
|
||||
<div>
|
||||
<Input
|
||||
label={$t.mapper.dataset_id}
|
||||
type="number"
|
||||
bind:value={datasetId}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
{:else}
|
||||
<div class="p-4 bg-gray-50 rounded-md border border-gray-100">
|
||||
<label for="mapper-excel" class="block text-sm font-medium text-gray-700">Excel File Path</label>
|
||||
<input type="text" id="mapper-excel" bind:value={excelPath} placeholder="/path/to/mapping.xlsx" class="mt-1 block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" />
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
<div class="flex justify-end">
|
||||
<button
|
||||
on:click={handleRunMapper}
|
||||
disabled={isRunning}
|
||||
class="inline-flex items-center px-4 py-2 border border-transparent text-sm font-medium rounded-md shadow-sm text-white bg-indigo-600 hover:bg-indigo-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-indigo-500 disabled:opacity-50"
|
||||
>
|
||||
{isRunning ? 'Starting...' : 'Run Mapper'}
|
||||
</button>
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-gray-700 mb-2">{$t.mapper.source}</label>
|
||||
<div class="flex space-x-4">
|
||||
<label class="inline-flex items-center">
|
||||
<input type="radio" bind:group={source} value="postgres" class="focus:ring-blue-500 h-4 w-4 text-blue-600 border-gray-300" />
|
||||
<span class="ml-2 text-sm text-gray-700">{$t.mapper.source_postgres}</span>
|
||||
</label>
|
||||
<label class="inline-flex items-center">
|
||||
<input type="radio" bind:group={source} value="excel" class="focus:ring-blue-500 h-4 w-4 text-blue-600 border-gray-300" />
|
||||
<span class="ml-2 text-sm text-gray-700">{$t.mapper.source_excel}</span>
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{#if source === 'postgres'}
|
||||
<div class="space-y-4 p-4 bg-gray-50 rounded-md border border-gray-100">
|
||||
<div>
|
||||
<Select
|
||||
label={$t.mapper.connection}
|
||||
bind:value={selectedConnection}
|
||||
options={[
|
||||
{ value: '', label: $t.mapper.select_connection },
|
||||
...connections.map(c => ({ value: c.id, label: c.name }))
|
||||
]}
|
||||
/>
|
||||
</div>
|
||||
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">
|
||||
<div>
|
||||
<Input
|
||||
label={$t.mapper.table_name}
|
||||
type="text"
|
||||
bind:value={tableName}
|
||||
/>
|
||||
</div>
|
||||
<div>
|
||||
<Input
|
||||
label={$t.mapper.table_schema}
|
||||
type="text"
|
||||
bind:value={tableSchema}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{:else}
|
||||
<div class="p-4 bg-gray-50 rounded-md border border-gray-100">
|
||||
<Input
|
||||
label={$t.mapper.excel_path}
|
||||
type="text"
|
||||
bind:value={excelPath}
|
||||
placeholder="/path/to/mapping.xlsx"
|
||||
/>
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
<div class="flex justify-end pt-2">
|
||||
<Button
|
||||
variant="primary"
|
||||
on:click={handleRunMapper}
|
||||
disabled={isRunning}
|
||||
>
|
||||
{isRunning ? $t.mapper.starting : $t.mapper.run}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</Card>
|
||||
</div>
|
||||
<!-- [/SECTION] -->
|
||||
<!-- [/DEF:MapperTool:Component] -->
|
||||
@@ -1,186 +0,0 @@
|
||||
<!-- [DEF:SearchTool:Component] -->
|
||||
<!--
|
||||
@SEMANTICS: search, tool, dataset, regex
|
||||
@PURPOSE: UI component for searching datasets using the SearchPlugin.
|
||||
@LAYER: UI
|
||||
@RELATION: USES -> frontend/src/services/toolsService.js
|
||||
-->
|
||||
<script>
|
||||
// [SECTION: IMPORTS]
|
||||
import { onMount } from 'svelte';
|
||||
import { runTask, getTaskStatus } from '../../services/toolsService.js';
|
||||
import { selectedTask } from '../../lib/stores.js';
|
||||
import { addToast } from '../../lib/toasts.js';
|
||||
// [/SECTION]
|
||||
|
||||
let envs = [];
|
||||
let selectedEnv = '';
|
||||
let searchQuery = '';
|
||||
let isRunning = false;
|
||||
let results = null;
|
||||
let pollInterval;
|
||||
|
||||
// [DEF:fetchEnvironments:Function]
|
||||
// @PURPOSE: Fetches the list of available environments.
|
||||
// @PRE: None.
|
||||
// @POST: envs array is populated.
|
||||
async function fetchEnvironments() {
|
||||
try {
|
||||
const res = await fetch('/api/environments');
|
||||
envs = await res.json();
|
||||
} catch (e) {
|
||||
addToast('Failed to fetch environments', 'error');
|
||||
}
|
||||
}
|
||||
// [/DEF:fetchEnvironments:Function]
|
||||
|
||||
// [DEF:handleSearch:Function]
|
||||
// @PURPOSE: Triggers the SearchPlugin task.
|
||||
// @PRE: selectedEnv and searchQuery must be set.
|
||||
// @POST: Task is started and polling begins.
|
||||
async function handleSearch() {
|
||||
if (!selectedEnv || !searchQuery) {
|
||||
addToast('Please select environment and enter query', 'warning');
|
||||
return;
|
||||
}
|
||||
|
||||
isRunning = true;
|
||||
results = null;
|
||||
try {
|
||||
// Find the environment name from ID
|
||||
const env = envs.find(e => e.id === selectedEnv);
|
||||
const task = await runTask('search-datasets', {
|
||||
env: env.name,
|
||||
query: searchQuery
|
||||
});
|
||||
|
||||
selectedTask.set(task);
|
||||
startPolling(task.id);
|
||||
} catch (e) {
|
||||
isRunning = false;
|
||||
addToast(e.message, 'error');
|
||||
}
|
||||
}
|
||||
// [/DEF:handleSearch:Function]
|
||||
|
||||
// [DEF:startPolling:Function]
|
||||
// @PURPOSE: Polls for task completion and results.
|
||||
// @PRE: taskId is provided.
|
||||
// @POST: pollInterval is set and results are updated on success.
|
||||
function startPolling(taskId) {
|
||||
if (pollInterval) clearInterval(pollInterval);
|
||||
|
||||
pollInterval = setInterval(async () => {
|
||||
try {
|
||||
const task = await getTaskStatus(taskId);
|
||||
selectedTask.set(task);
|
||||
|
||||
if (task.status === 'SUCCESS') {
|
||||
clearInterval(pollInterval);
|
||||
isRunning = false;
|
||||
results = task.result;
|
||||
addToast('Search completed', 'success');
|
||||
} else if (task.status === 'FAILED') {
|
||||
clearInterval(pollInterval);
|
||||
isRunning = false;
|
||||
addToast('Search failed', 'error');
|
||||
}
|
||||
} catch (e) {
|
||||
clearInterval(pollInterval);
|
||||
isRunning = false;
|
||||
addToast('Error polling task status', 'error');
|
||||
}
|
||||
}, 2000);
|
||||
}
|
||||
// [/DEF:startPolling:Function]
|
||||
|
||||
onMount(fetchEnvironments);
|
||||
</script>
|
||||
|
||||
<!-- [SECTION: TEMPLATE] -->
|
||||
<div class="space-y-6">
|
||||
<div class="bg-white p-6 rounded-lg shadow-sm border border-gray-200">
|
||||
<h3 class="text-lg font-medium text-gray-900 mb-4">Search Dataset Metadata</h3>
|
||||
<div class="grid grid-cols-1 md:grid-cols-2 gap-4 items-end">
|
||||
<div>
|
||||
<label for="env-select" class="block text-sm font-medium text-gray-700">Environment</label>
|
||||
<select
|
||||
id="env-select"
|
||||
bind:value={selectedEnv}
|
||||
class="mt-1 block w-full pl-3 pr-10 py-2 text-base border-gray-300 focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm rounded-md"
|
||||
>
|
||||
<option value="" disabled>-- Select Environment --</option>
|
||||
{#each envs as env}
|
||||
<option value={env.id}>{env.name}</option>
|
||||
{/each}
|
||||
</select>
|
||||
</div>
|
||||
<div>
|
||||
<label for="search-query" class="block text-sm font-medium text-gray-700">Regex Pattern</label>
|
||||
<input
|
||||
type="text"
|
||||
id="search-query"
|
||||
bind:value={searchQuery}
|
||||
placeholder="e.g. from dm.*\.account"
|
||||
class="mt-1 block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<div class="mt-4 flex justify-end">
|
||||
<button
|
||||
on:click={handleSearch}
|
||||
disabled={isRunning}
|
||||
class="inline-flex items-center px-4 py-2 border border-transparent text-sm font-medium rounded-md shadow-sm text-white bg-indigo-600 hover:bg-indigo-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-indigo-500 disabled:opacity-50"
|
||||
>
|
||||
{#if isRunning}
|
||||
<svg class="animate-spin -ml-1 mr-3 h-5 w-5 text-white" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24">
|
||||
<circle class="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" stroke-width="4"></circle>
|
||||
<path class="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"></path>
|
||||
</svg>
|
||||
Searching...
|
||||
{:else}
|
||||
Search
|
||||
{/if}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{#if results}
|
||||
<div class="bg-white shadow overflow-hidden sm:rounded-md border border-gray-200">
|
||||
<div class="px-4 py-5 sm:px-6 flex justify-between items-center bg-gray-50 border-b border-gray-200">
|
||||
<h3 class="text-lg leading-6 font-medium text-gray-900">
|
||||
Search Results
|
||||
</h3>
|
||||
<span class="inline-flex items-center px-2.5 py-0.5 rounded-full text-xs font-medium bg-blue-100 text-blue-800">
|
||||
{results.count} matches
|
||||
</span>
|
||||
</div>
|
||||
<ul class="divide-y divide-gray-200">
|
||||
{#each results.results as item}
|
||||
<li class="p-4 hover:bg-gray-50">
|
||||
<div class="flex items-center justify-between">
|
||||
<div class="text-sm font-medium text-indigo-600 truncate">
|
||||
{item.dataset_name} (ID: {item.dataset_id})
|
||||
</div>
|
||||
<div class="ml-2 flex-shrink-0 flex">
|
||||
<p class="px-2 inline-flex text-xs leading-5 font-semibold rounded-full bg-green-100 text-green-800">
|
||||
Field: {item.field}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="mt-2">
|
||||
<pre class="text-xs text-gray-500 bg-gray-50 p-2 rounded border border-gray-100 overflow-x-auto">{item.match_context}</pre>
|
||||
</div>
|
||||
</li>
|
||||
{/each}
|
||||
{#if results.count === 0}
|
||||
<li class="p-8 text-center text-gray-500 italic">
|
||||
No matches found for the given pattern.
|
||||
</li>
|
||||
{/if}
|
||||
</ul>
|
||||
</div>
|
||||
{/if}
|
||||
</div>
|
||||
<!-- [/SECTION] -->
|
||||
<!-- [/DEF:SearchTool:Component] -->
|
||||
@@ -95,7 +95,10 @@ async function requestApi(endpoint, method = 'GET', body = null) {
|
||||
const response = await fetch(`${API_BASE_URL}${endpoint}`, options);
|
||||
if (!response.ok) {
|
||||
const errorData = await response.json().catch(() => ({}));
|
||||
throw new Error(errorData.detail || `API request failed with status ${response.status}`);
|
||||
const message = errorData.detail
|
||||
? (typeof errorData.detail === 'string' ? errorData.detail : JSON.stringify(errorData.detail))
|
||||
: `API request failed with status ${response.status}`;
|
||||
throw new Error(message);
|
||||
}
|
||||
return await response.json();
|
||||
} catch (error) {
|
||||
@@ -123,6 +126,8 @@ export const api = {
|
||||
deleteEnvironment: (id) => requestApi(`/settings/environments/${id}`, 'DELETE'),
|
||||
testEnvironmentConnection: (id) => postApi(`/settings/environments/${id}/test`, {}),
|
||||
updateEnvironmentSchedule: (id, schedule) => requestApi(`/environments/${id}/schedule`, 'PUT', schedule),
|
||||
getStorageSettings: () => fetchApi('/settings/storage'),
|
||||
updateStorageSettings: (storage) => requestApi('/settings/storage', 'PUT', storage),
|
||||
getEnvironmentsList: () => fetchApi('/environments'),
|
||||
};
|
||||
// [/DEF:api:Data]
|
||||
@@ -130,6 +135,7 @@ export const api = {
|
||||
// [/DEF:api_module:Module]
|
||||
|
||||
// Export individual functions for easier use in components
|
||||
export { requestApi };
|
||||
export const getPlugins = api.getPlugins;
|
||||
export const getTasks = api.getTasks;
|
||||
export const getTask = api.getTask;
|
||||
@@ -143,3 +149,5 @@ export const deleteEnvironment = api.deleteEnvironment;
|
||||
export const testEnvironmentConnection = api.testEnvironmentConnection;
|
||||
export const updateEnvironmentSchedule = api.updateEnvironmentSchedule;
|
||||
export const getEnvironmentsList = api.getEnvironmentsList;
|
||||
export const getStorageSettings = api.getStorageSettings;
|
||||
export const updateStorageSettings = api.updateStorageSettings;
|
||||
|
||||
@@ -20,7 +20,6 @@
|
||||
let settings = {
|
||||
environments: [],
|
||||
settings: {
|
||||
backup_path: '',
|
||||
default_environment_id: null,
|
||||
logging: {
|
||||
level: 'INFO',
|
||||
@@ -204,12 +203,6 @@
|
||||
|
||||
<section class="mb-8 bg-white p-6 rounded shadow">
|
||||
<h2 class="text-xl font-semibold mb-4">Global Settings</h2>
|
||||
<div class="grid grid-cols-1 gap-4">
|
||||
<div>
|
||||
<label for="backup_path" class="block text-sm font-medium text-gray-700">Backup Storage Path</label>
|
||||
<input type="text" id="backup_path" bind:value={settings.settings.backup_path} class="mt-1 block w-full border border-gray-300 rounded-md shadow-sm p-2" />
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<h3 class="text-lg font-medium mb-4 mt-6">Logging Configuration</h3>
|
||||
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">
|
||||
|
||||
@@ -23,10 +23,8 @@
|
||||
*/
|
||||
function selectPlugin(plugin) {
|
||||
console.log(`[Dashboard][Action] Selecting plugin: ${plugin.id}`);
|
||||
if (plugin.id === 'superset-migration') {
|
||||
goto('/migration');
|
||||
} else if (plugin.id === 'git-integration') {
|
||||
goto('/git');
|
||||
if (plugin.ui_route) {
|
||||
goto(plugin.ui_route);
|
||||
} else {
|
||||
selectedPlugin.set(plugin);
|
||||
}
|
||||
@@ -82,7 +80,7 @@
|
||||
{/if}
|
||||
|
||||
<div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-6">
|
||||
{#each data.plugins as plugin}
|
||||
{#each data.plugins.filter(p => p.id !== 'superset-search') as plugin}
|
||||
<div
|
||||
on:click={() => selectPlugin(plugin)}
|
||||
role="button"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
<script>
|
||||
import { onMount } from 'svelte';
|
||||
import { updateGlobalSettings, addEnvironment, updateEnvironment, deleteEnvironment, testEnvironmentConnection } from '../../lib/api';
|
||||
import { updateGlobalSettings, addEnvironment, updateEnvironment, deleteEnvironment, testEnvironmentConnection, updateStorageSettings } from '../../lib/api';
|
||||
import { addToast } from '../../lib/toasts';
|
||||
import { t } from '$lib/i18n';
|
||||
import { Button, Input, Card, PageHeader } from '$lib/ui';
|
||||
@@ -41,6 +41,24 @@
|
||||
}
|
||||
// [/DEF:handleSaveGlobal:Function]
|
||||
|
||||
// [DEF:handleSaveStorage:Function]
|
||||
/* @PURPOSE: Saves storage-specific settings.
|
||||
@PRE: settings.settings.storage must contain valid configuration.
|
||||
@POST: Storage settings are updated via API.
|
||||
*/
|
||||
async function handleSaveStorage() {
|
||||
try {
|
||||
console.log("[Settings.handleSaveStorage][Action] Saving storage settings.");
|
||||
await updateStorageSettings(settings.settings.storage);
|
||||
addToast('Storage settings saved', 'success');
|
||||
console.log("[Settings.handleSaveStorage][Coherence:OK] Storage settings saved.");
|
||||
} catch (error) {
|
||||
console.error("[Settings.handleSaveStorage][Coherence:Failed] Failed to save storage settings:", error);
|
||||
addToast(error.message || 'Failed to save storage settings', 'error');
|
||||
}
|
||||
}
|
||||
// [/DEF:handleSaveStorage:Function]
|
||||
|
||||
// [DEF:handleAddOrUpdateEnv:Function]
|
||||
/* @PURPOSE: Adds a new environment or updates an existing one.
|
||||
@PRE: newEnv must contain valid environment details.
|
||||
@@ -152,14 +170,37 @@
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
|
||||
<div class="mb-8">
|
||||
<Card title={$t.settings?.global_title || "Global Settings"}>
|
||||
<div class="grid grid-cols-1 gap-6">
|
||||
<Card title={$t.settings?.storage_title || "File Storage Configuration"}>
|
||||
<div class="grid grid-cols-1 md:grid-cols-2 gap-6">
|
||||
<div class="md:col-span-2">
|
||||
<Input
|
||||
label={$t.settings?.storage_root || "Storage Root Path"}
|
||||
bind:value={settings.settings.storage.root_path}
|
||||
/>
|
||||
</div>
|
||||
<Input
|
||||
label={$t.settings?.backup_path || "Backup Storage Path"}
|
||||
bind:value={settings.settings.backup_path}
|
||||
label={$t.settings?.storage_backup_pattern || "Backup Directory Pattern"}
|
||||
bind:value={settings.settings.storage.backup_structure_pattern}
|
||||
/>
|
||||
<Button on:click={handleSaveGlobal}>
|
||||
<Input
|
||||
label={$t.settings?.storage_repo_pattern || "Repository Directory Pattern"}
|
||||
bind:value={settings.settings.storage.repo_structure_pattern}
|
||||
/>
|
||||
<Input
|
||||
label={$t.settings?.storage_filename_pattern || "Filename Pattern"}
|
||||
bind:value={settings.settings.storage.filename_pattern}
|
||||
/>
|
||||
<div class="bg-gray-50 p-4 rounded border border-gray-200">
|
||||
<span class="block text-xs font-semibold text-gray-500 uppercase mb-2">{$t.settings?.storage_preview || "Path Preview"}</span>
|
||||
<code class="text-sm text-indigo-600">
|
||||
{settings.settings.storage.root_path}/backups/sample_backup.zip
|
||||
</code>
|
||||
</div>
|
||||
</div>
|
||||
<div class="mt-6">
|
||||
<Button on:click={handleSaveStorage}>
|
||||
{$t.common.save}
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
@@ -18,7 +18,6 @@ export async function load() {
|
||||
settings: {
|
||||
environments: [],
|
||||
settings: {
|
||||
backup_path: '',
|
||||
default_environment_id: null
|
||||
}
|
||||
},
|
||||
|
||||
@@ -1,40 +0,0 @@
|
||||
<script>
|
||||
import { onMount } from 'svelte';
|
||||
import { gitService } from '../../../services/gitService';
|
||||
import { addToast as toast } from '../../../lib/toasts.js';
|
||||
|
||||
let environments = [];
|
||||
|
||||
onMount(async () => {
|
||||
try {
|
||||
environments = await gitService.getEnvironments();
|
||||
} catch (e) {
|
||||
toast(e.message, 'error');
|
||||
}
|
||||
});
|
||||
</script>
|
||||
|
||||
<div class="p-6">
|
||||
<h1 class="text-2xl font-bold mb-6">Deployment Environments</h1>
|
||||
|
||||
<div class="bg-white p-6 rounded shadow">
|
||||
<h2 class="text-xl font-semibold mb-4">Target Environments</h2>
|
||||
{#if environments.length === 0}
|
||||
<p class="text-gray-500">No deployment environments configured.</p>
|
||||
{:else}
|
||||
<ul class="divide-y">
|
||||
{#each environments as env}
|
||||
<li class="py-3 flex justify-between items-center">
|
||||
<div>
|
||||
<span class="font-medium">{env.name}</span>
|
||||
<div class="text-xs text-gray-400">{env.superset_url}</div>
|
||||
</div>
|
||||
<span class="px-2 py-1 text-xs rounded {env.is_active ? 'bg-green-100 text-green-800' : 'bg-gray-100 text-gray-800'}">
|
||||
{env.is_active ? 'Active' : 'Inactive'}
|
||||
</span>
|
||||
</li>
|
||||
{/each}
|
||||
</ul>
|
||||
{/if}
|
||||
</div>
|
||||
</div>
|
||||
@@ -116,13 +116,7 @@
|
||||
</script>
|
||||
|
||||
<div class="container mx-auto p-4 max-w-6xl">
|
||||
<PageHeader title={$t.tasks.management}>
|
||||
<div slot="actions">
|
||||
<Button on:click={() => showBackupModal = true}>
|
||||
{$t.tasks.run_backup}
|
||||
</Button>
|
||||
</div>
|
||||
</PageHeader>
|
||||
<PageHeader title={$t.tasks.management} />
|
||||
|
||||
<div class="grid grid-cols-1 lg:grid-cols-3 gap-6">
|
||||
<div class="lg:col-span-1">
|
||||
|
||||
27
frontend/src/routes/tools/backups/+page.svelte
Normal file
27
frontend/src/routes/tools/backups/+page.svelte
Normal file
@@ -0,0 +1,27 @@
|
||||
<!-- [DEF:BackupPage:Component] -->
|
||||
<!--
|
||||
@SEMANTICS: backup, page, tools
|
||||
@PURPOSE: Entry point for the Backup Management interface.
|
||||
@LAYER: Page
|
||||
@RELATION: USES -> BackupManager
|
||||
-->
|
||||
|
||||
<script lang="ts">
|
||||
// [SECTION: IMPORTS]
|
||||
import { t } from '../../../lib/i18n';
|
||||
import { PageHeader } from '../../../lib/ui';
|
||||
import BackupManager from '../../../components/backups/BackupManager.svelte';
|
||||
// [/SECTION]
|
||||
</script>
|
||||
|
||||
<!-- [SECTION: TEMPLATE] -->
|
||||
<div class="container mx-auto p-4 max-w-6xl">
|
||||
<PageHeader title={$t.nav.tools_backups} />
|
||||
|
||||
<div class="mt-6">
|
||||
<BackupManager />
|
||||
</div>
|
||||
</div>
|
||||
<!-- [/SECTION] -->
|
||||
|
||||
<!-- [/DEF:BackupPage:Component] -->
|
||||
@@ -1,25 +0,0 @@
|
||||
<!-- [DEF:SearchPage:Component] -->
|
||||
<!--
|
||||
@SEMANTICS: search, page, tool
|
||||
@PURPOSE: Page for the dataset search tool.
|
||||
@LAYER: UI
|
||||
-->
|
||||
<script>
|
||||
import SearchTool from '../../../components/tools/SearchTool.svelte';
|
||||
import TaskRunner from '../../../components/TaskRunner.svelte';
|
||||
import { PageHeader } from '$lib/ui';
|
||||
</script>
|
||||
|
||||
<div class="max-w-7xl mx-auto p-6">
|
||||
<PageHeader title="Dataset Search" />
|
||||
|
||||
<div class="grid grid-cols-1 lg:grid-cols-3 gap-8">
|
||||
<div class="lg:col-span-2">
|
||||
<SearchTool />
|
||||
</div>
|
||||
<div class="lg:col-span-1">
|
||||
<TaskRunner />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- [/DEF:SearchPage:Component] -->
|
||||
212
frontend/src/routes/tools/storage/+page.svelte
Normal file
212
frontend/src/routes/tools/storage/+page.svelte
Normal file
@@ -0,0 +1,212 @@
|
||||
<!-- [DEF:StoragePage:Component] -->
|
||||
<!--
|
||||
@SEMANTICS: storage, files, management
|
||||
@PURPOSE: Main page for file storage management.
|
||||
@LAYER: Feature
|
||||
@RELATION: DEPENDS_ON -> storageService
|
||||
@RELATION: CONTAINS -> FileList
|
||||
@RELATION: CONTAINS -> FileUpload
|
||||
|
||||
@INVARIANT: Always displays tabs for Backups and Repositories.
|
||||
-->
|
||||
|
||||
<script lang="ts">
|
||||
// [SECTION: IMPORTS]
|
||||
import { onMount } from 'svelte';
|
||||
import { page } from '$app/stores';
|
||||
import { listFiles, deleteFile } from '../../../services/storageService';
|
||||
import { addToast } from '../../../lib/toasts';
|
||||
import { t } from '../../../lib/i18n';
|
||||
import FileList from '../../../components/storage/FileList.svelte';
|
||||
import FileUpload from '../../../components/storage/FileUpload.svelte';
|
||||
// [/SECTION: IMPORTS]
|
||||
|
||||
// [DEF:loadFiles:Function]
|
||||
/**
|
||||
* @purpose Fetches the list of files from the server.
|
||||
* @post Updates the `files` array with the latest data.
|
||||
*/
|
||||
let files = [];
|
||||
let isLoading = false;
|
||||
let activeTab = 'backups';
|
||||
let currentPath = 'backups'; // Relative to storage root
|
||||
|
||||
async function loadFiles() {
|
||||
isLoading = true;
|
||||
try {
|
||||
const category = activeTab;
|
||||
|
||||
// If we have a currentPath, we use it.
|
||||
// But if user switched tabs, we should reset currentPath to category root
|
||||
let effectivePath = currentPath;
|
||||
if (category && !currentPath.startsWith(category)) {
|
||||
effectivePath = category;
|
||||
currentPath = category;
|
||||
}
|
||||
|
||||
// API expects path relative to category root if category is provided
|
||||
const subpath = (category && effectivePath.startsWith(category))
|
||||
? effectivePath.substring(category.length).replace(/^\/+/, '')
|
||||
: effectivePath;
|
||||
|
||||
files = await listFiles(category, subpath);
|
||||
} catch (error) {
|
||||
addToast($t.storage.messages.load_failed.replace('{error}', error.message), 'error');
|
||||
} finally {
|
||||
isLoading = false;
|
||||
}
|
||||
}
|
||||
// [/DEF:loadFiles:Function]
|
||||
|
||||
// [DEF:handleDelete:Function]
|
||||
/**
|
||||
* @purpose Handles the file deletion process.
|
||||
* @param {CustomEvent} event - The delete event containing category and path.
|
||||
*/
|
||||
async function handleDelete(event) {
|
||||
const { category, path, name } = event.detail;
|
||||
if (!confirm($t.storage.messages.delete_confirm.replace('{name}', name))) return;
|
||||
|
||||
try {
|
||||
await deleteFile(category, path);
|
||||
addToast($t.storage.messages.delete_success.replace('{name}', name), 'success');
|
||||
await loadFiles();
|
||||
} catch (error) {
|
||||
addToast($t.storage.messages.delete_failed.replace('{error}', error.message), 'error');
|
||||
}
|
||||
}
|
||||
// [/DEF:handleDelete:Function]
|
||||
|
||||
// [DEF:handleNavigate:Function]
|
||||
/**
|
||||
* @purpose Updates the current path and reloads files when navigating into a directory.
|
||||
* @param {CustomEvent} event - The navigation event containing the new path.
|
||||
*/
|
||||
function handleNavigate(event) {
|
||||
currentPath = event.detail;
|
||||
loadFiles();
|
||||
}
|
||||
// [/DEF:handleNavigate:Function]
|
||||
|
||||
// [DEF:navigateUp:Function]
|
||||
/**
|
||||
* @purpose Navigates one level up in the directory structure.
|
||||
* @pre currentPath is set and deeper than activeTab root.
|
||||
* @post currentPath is moved up one directory level.
|
||||
*/
|
||||
function navigateUp() {
|
||||
if (!currentPath || currentPath === activeTab) return;
|
||||
const parts = currentPath.split('/');
|
||||
parts.pop();
|
||||
currentPath = parts.join('/') || '';
|
||||
loadFiles();
|
||||
}
|
||||
// [/DEF:navigateUp:Function]
|
||||
|
||||
onMount(() => {
|
||||
const pathParam = $page.url.searchParams.get('path');
|
||||
if (pathParam) {
|
||||
currentPath = pathParam;
|
||||
if (pathParam.startsWith('repositorys')) {
|
||||
activeTab = 'repositorys';
|
||||
} else {
|
||||
activeTab = 'backups';
|
||||
}
|
||||
}
|
||||
loadFiles();
|
||||
});
|
||||
|
||||
$: if (activeTab) {
|
||||
// Reset path when switching tabs
|
||||
if (!currentPath.startsWith(activeTab)) {
|
||||
currentPath = activeTab;
|
||||
}
|
||||
loadFiles();
|
||||
}
|
||||
</script>
|
||||
|
||||
<!-- [SECTION: TEMPLATE] -->
|
||||
<div class="container mx-auto p-4 max-w-6xl">
|
||||
<div class="mb-6">
|
||||
<h1 class="text-2xl font-bold text-gray-900">{$t.storage.management}</h1>
|
||||
{#if currentPath}
|
||||
<div class="flex items-center mt-2 text-sm text-gray-500">
|
||||
<button on:click={() => { currentPath = activeTab; loadFiles(); }} class="hover:text-indigo-600">{$t.storage.root}</button>
|
||||
{#each currentPath.split('/').slice(1) as part, i}
|
||||
<span class="mx-2">/</span>
|
||||
<button
|
||||
on:click={() => { currentPath = currentPath.split('/').slice(0, i + 1).join('/'); loadFiles(); }}
|
||||
class="hover:text-indigo-600 capitalize"
|
||||
>
|
||||
{part}
|
||||
</button>
|
||||
{/each}
|
||||
</div>
|
||||
{/if}
|
||||
</div>
|
||||
|
||||
<div class="flex justify-end mb-4">
|
||||
<button
|
||||
on:click={loadFiles}
|
||||
class="inline-flex items-center px-4 py-2 border border-gray-300 rounded-md shadow-sm text-sm font-medium text-gray-700 bg-white hover:bg-gray-50"
|
||||
disabled={isLoading}
|
||||
>
|
||||
{isLoading ? $t.storage.refreshing : $t.storage.refresh}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div class="grid grid-cols-1 lg:grid-cols-3 gap-6">
|
||||
<!-- Main Content: File List -->
|
||||
<div class="lg:col-span-2 space-y-4">
|
||||
<!-- Tabs -->
|
||||
<div class="border-b border-gray-200">
|
||||
<nav class="-mb-px flex space-x-8">
|
||||
<button
|
||||
on:click={() => activeTab = 'backups'}
|
||||
class="py-4 px-1 border-b-2 font-medium text-sm {activeTab === 'backups' ? 'border-indigo-500 text-indigo-600' : 'border-transparent text-gray-500 hover:text-gray-700 hover:border-gray-300'}"
|
||||
>
|
||||
{$t.storage.backups}
|
||||
</button>
|
||||
<button
|
||||
on:click={() => activeTab = 'repositorys'}
|
||||
class="py-4 px-1 border-b-2 font-medium text-sm {activeTab === 'repositorys' ? 'border-indigo-500 text-indigo-600' : 'border-transparent text-gray-500 hover:text-gray-700 hover:border-gray-300'}"
|
||||
>
|
||||
{$t.storage.repositories}
|
||||
</button>
|
||||
</nav>
|
||||
</div>
|
||||
|
||||
<div class="flex items-center mb-2">
|
||||
{#if currentPath && currentPath !== activeTab}
|
||||
<button
|
||||
on:click={navigateUp}
|
||||
class="mr-4 inline-flex items-center px-3 py-1 border border-gray-300 shadow-sm text-xs font-medium rounded text-gray-700 bg-white hover:bg-gray-50"
|
||||
>
|
||||
<svg class="h-4 w-4 mr-1" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M10 19l-7-7m0 0l7-7m-7 7h18" />
|
||||
</svg>
|
||||
Back
|
||||
</button>
|
||||
{/if}
|
||||
</div>
|
||||
|
||||
<FileList {files} on:delete={handleDelete} on:navigate={handleNavigate} />
|
||||
</div>
|
||||
|
||||
<!-- Sidebar: Upload -->
|
||||
<div class="lg:col-span-1">
|
||||
<FileUpload
|
||||
category={activeTab}
|
||||
path={currentPath}
|
||||
on:uploaded={loadFiles}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- [/SECTION: TEMPLATE] -->
|
||||
|
||||
<style>
|
||||
/* ... */
|
||||
</style>
|
||||
|
||||
<!-- [/DEF:StoragePage:Component] -->
|
||||
109
frontend/src/services/storageService.js
Normal file
109
frontend/src/services/storageService.js
Normal file
@@ -0,0 +1,109 @@
|
||||
// [DEF:storageService:Module]
|
||||
/**
|
||||
* @purpose Frontend API client for file storage management.
|
||||
* @layer Service
|
||||
* @relation DEPENDS_ON -> backend.api.storage
|
||||
* @SEMANTICS: storage, api, client
|
||||
*/
|
||||
|
||||
const API_BASE = '/api/storage';
|
||||
|
||||
// [DEF:listFiles:Function]
|
||||
/**
|
||||
* @purpose Fetches the list of files for a given category and subpath.
|
||||
* @param {string} [category] - Optional category filter.
|
||||
* @param {string} [path] - Optional subpath filter.
|
||||
* @returns {Promise<Array>}
|
||||
* @PRE category and path should be valid strings if provided.
|
||||
* @POST Returns a promise resolving to an array of StoredFile objects.
|
||||
*/
|
||||
export async function listFiles(category, path) {
|
||||
const params = new URLSearchParams();
|
||||
if (category) {
|
||||
params.append('category', category);
|
||||
}
|
||||
if (path) {
|
||||
params.append('path', path);
|
||||
}
|
||||
const response = await fetch(`${API_BASE}/files?${params.toString()}`);
|
||||
if (!response.ok) {
|
||||
throw new Error(`Failed to fetch files: ${response.statusText}`);
|
||||
}
|
||||
return await response.json();
|
||||
}
|
||||
// [/DEF:listFiles:Function]
|
||||
|
||||
// [DEF:uploadFile:Function]
|
||||
/**
|
||||
* @purpose Uploads a file to the storage system.
|
||||
* @param {File} file - The file to upload.
|
||||
* @param {string} category - Target category.
|
||||
* @param {string} [path] - Target subpath.
|
||||
* @returns {Promise<Object>}
|
||||
* @PRE file must be a valid File object; category must be specified.
|
||||
* @POST Returns a promise resolving to the metadata of the uploaded file.
|
||||
*/
|
||||
export async function uploadFile(file, category, path) {
|
||||
const formData = new FormData();
|
||||
formData.append('file', file);
|
||||
formData.append('category', category);
|
||||
if (path) {
|
||||
formData.append('path', path);
|
||||
}
|
||||
|
||||
const response = await fetch(`${API_BASE}/upload`, {
|
||||
method: 'POST',
|
||||
body: formData
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorData = await response.json().catch(() => ({}));
|
||||
throw new Error(errorData.detail || `Failed to upload file: ${response.statusText}`);
|
||||
}
|
||||
return await response.json();
|
||||
}
|
||||
// [/DEF:uploadFile:Function]
|
||||
|
||||
// [DEF:deleteFile:Function]
|
||||
/**
|
||||
* @purpose Deletes a file or directory from storage.
|
||||
* @param {string} category - File category.
|
||||
* @param {string} path - Relative path of the item.
|
||||
* @returns {Promise<void>}
|
||||
* @PRE category and path must identify an existing file or directory.
|
||||
* @POST The specified file or directory is removed from storage.
|
||||
*/
|
||||
export async function deleteFile(category, path) {
|
||||
const response = await fetch(`${API_BASE}/files/${category}/${path}`, {
|
||||
method: 'DELETE'
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorData = await response.json().catch(() => ({}));
|
||||
throw new Error(errorData.detail || `Failed to delete: ${response.statusText}`);
|
||||
}
|
||||
}
|
||||
// [/DEF:deleteFile:Function]
|
||||
|
||||
// [DEF:downloadFileUrl:Function]
|
||||
/**
|
||||
* @purpose Returns the URL for downloading a file.
|
||||
* @param {string} category - File category.
|
||||
* @param {string} path - Relative path of the file.
|
||||
* @returns {string}
|
||||
* @PRE category and path must identify an existing file.
|
||||
* @POST Returns a valid API URL for file download.
|
||||
*/
|
||||
export function downloadFileUrl(category, path) {
|
||||
return `${API_BASE}/download/${category}/${path}`;
|
||||
}
|
||||
// [/DEF:downloadFileUrl:Function]
|
||||
|
||||
export default {
|
||||
listFiles,
|
||||
uploadFile,
|
||||
deleteFile,
|
||||
downloadFileUrl
|
||||
};
|
||||
|
||||
// [/DEF:storageService:Module]
|
||||
22
frontend/src/types/backup.ts
Normal file
22
frontend/src/types/backup.ts
Normal file
@@ -0,0 +1,22 @@
|
||||
/**
|
||||
* [DEF:BackupTypes:Module]
|
||||
* @SEMANTICS: types, backup, interface
|
||||
* @PURPOSE: Defines types and interfaces for the Backup Management UI.
|
||||
*/
|
||||
|
||||
export interface Backup {
|
||||
id: string;
|
||||
name: string;
|
||||
environment: string;
|
||||
created_at: string;
|
||||
size_bytes?: number;
|
||||
status: 'success' | 'failed' | 'in_progress';
|
||||
}
|
||||
|
||||
export interface BackupCreateRequest {
|
||||
environment_id: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* [/DEF:BackupTypes:Module]
|
||||
*/
|
||||
@@ -130,7 +130,8 @@ class SemanticEntity:
|
||||
self.compliance_issues.append(f"Missing Mandatory Tag: @{req_tag}")
|
||||
|
||||
# 3. Check for Belief State Logging (Python only)
|
||||
if self.type == "Function" and self.file_path.endswith(".py"):
|
||||
# Skip check for logger.py to avoid circular dependencies
|
||||
if self.type == "Function" and self.file_path.endswith(".py") and "backend/src/core/logger.py" not in self.file_path:
|
||||
if not getattr(self, 'has_belief_scope', False):
|
||||
self.compliance_issues.append("Missing Belief State Logging: Function should use belief_scope context manager.")
|
||||
|
||||
|
||||
124
semantics/reports/semantic_report_20260126_112020.md
Normal file
124
semantics/reports/semantic_report_20260126_112020.md
Normal file
File diff suppressed because one or more lines are too long
117
semantics/reports/semantic_report_20260126_114128.md
Normal file
117
semantics/reports/semantic_report_20260126_114128.md
Normal file
@@ -0,0 +1,117 @@
|
||||
# Semantic Compliance Report
|
||||
|
||||
**Generated At:** 2026-01-26T11:41:28.355350
|
||||
**Global Compliance Score:** 99.2%
|
||||
**Scanned Files:** 108
|
||||
|
||||
## File Compliance Status
|
||||
| File | Score | Issues |
|
||||
|------|-------|--------|
|
||||
| frontend/src/components/storage/FileList.svelte | 🟡 75% | [isDirectory] Missing Mandatory Tag: @PRE<br>[isDirectory] Missing Mandatory Tag: @POST<br>[isDirectory] Missing Mandatory Tag: @PRE<br>[isDirectory] Missing Mandatory Tag: @POST<br>[formatSize] Missing Mandatory Tag: @PRE<br>[formatSize] Missing Mandatory Tag: @POST<br>[formatSize] Missing Mandatory Tag: @PRE<br>[formatSize] Missing Mandatory Tag: @POST<br>[formatDate] Missing Mandatory Tag: @PRE<br>[formatDate] Missing Mandatory Tag: @POST<br>[formatDate] Missing Mandatory Tag: @PRE<br>[formatDate] Missing Mandatory Tag: @POST |
|
||||
| frontend/src/routes/tools/storage/+page.svelte | 🟡 77% | [loadFiles] Missing Mandatory Tag: @PRE<br>[loadFiles] Missing Mandatory Tag: @PRE<br>[handleDelete] Missing Mandatory Tag: @PRE<br>[handleDelete] Missing Mandatory Tag: @POST<br>[handleDelete] Missing Mandatory Tag: @PRE<br>[handleDelete] Missing Mandatory Tag: @POST<br>[handleNavigate] Missing Mandatory Tag: @PRE<br>[handleNavigate] Missing Mandatory Tag: @POST<br>[handleNavigate] Missing Mandatory Tag: @PRE<br>[handleNavigate] Missing Mandatory Tag: @POST<br>[navigateUp] Missing Mandatory Tag: @PRE<br>[navigateUp] Missing Mandatory Tag: @POST<br>[navigateUp] Missing Mandatory Tag: @PRE<br>[navigateUp] Missing Mandatory Tag: @POST |
|
||||
| frontend/src/components/storage/FileUpload.svelte | 🟡 89% | [handleDrop] Missing Mandatory Tag: @PRE<br>[handleDrop] Missing Mandatory Tag: @POST<br>[handleDrop] Missing Mandatory Tag: @PRE<br>[handleDrop] Missing Mandatory Tag: @POST |
|
||||
| frontend/src/components/git/CommitModal.svelte | 🟡 94% | [loadStatus] Missing Mandatory Tag: @POST<br>[loadStatus] Missing Mandatory Tag: @POST |
|
||||
| frontend/src/components/DashboardGrid.svelte | 🟡 94% | [openGit] Missing Mandatory Tag: @PRE<br>[openGit] Missing Mandatory Tag: @POST<br>[openGit] Missing Mandatory Tag: @PRE<br>[openGit] Missing Mandatory Tag: @POST |
|
||||
| backend/src/api/routes/settings.py | 🟡 95% | [get_storage_settings] Missing Mandatory Tag: @PRE<br>[get_storage_settings] Missing Mandatory Tag: @POST<br>[get_storage_settings] Missing Mandatory Tag: @PRE<br>[get_storage_settings] Missing Mandatory Tag: @POST<br>[update_storage_settings] Missing Mandatory Tag: @PRE<br>[update_storage_settings] Missing Mandatory Tag: @PRE |
|
||||
| frontend/src/components/git/DeploymentModal.svelte | 🟡 96% | [loadEnvironments] Missing Mandatory Tag: @PRE<br>[loadEnvironments] Missing Mandatory Tag: @PRE |
|
||||
| frontend/src/components/git/BranchSelector.svelte | 🟡 97% | [handleCheckout] Missing Mandatory Tag: @PRE<br>[handleCheckout] Missing Mandatory Tag: @PRE |
|
||||
| backend/src/core/utils/dataset_mapper.py | 🟡 97% | [__init__] Missing Mandatory Tag: @PRE<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__init__] Missing Mandatory Tag: @PRE<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__init__] Missing Mandatory Tag: @PRE<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager. |
|
||||
| generate_semantic_map.py | 🟢 100% | [__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__enter__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__enter__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__exit__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__exit__] Missing Belief State Logging: Function should use belief_scope context manager. |
|
||||
| frontend/src/lib/stores.js | 🟢 100% | OK |
|
||||
| frontend/src/lib/toasts.js | 🟢 100% | OK |
|
||||
| frontend/src/lib/api.js | 🟢 100% | OK |
|
||||
| frontend/src/lib/ui/Select.svelte | 🟢 100% | OK |
|
||||
| frontend/src/lib/ui/index.ts | 🟢 100% | OK |
|
||||
| frontend/src/lib/ui/PageHeader.svelte | 🟢 100% | OK |
|
||||
| frontend/src/lib/ui/Card.svelte | 🟢 100% | OK |
|
||||
| frontend/src/lib/ui/Button.svelte | 🟢 100% | OK |
|
||||
| frontend/src/lib/ui/Input.svelte | 🟢 100% | OK |
|
||||
| frontend/src/lib/ui/LanguageSwitcher.svelte | 🟢 100% | OK |
|
||||
| frontend/src/lib/i18n/index.ts | 🟢 100% | OK |
|
||||
| frontend/src/routes/+page.svelte | 🟢 100% | OK |
|
||||
| frontend/src/routes/+page.ts | 🟢 100% | OK |
|
||||
| frontend/src/routes/tasks/+page.svelte | 🟢 100% | OK |
|
||||
| frontend/src/routes/migration/+page.svelte | 🟢 100% | OK |
|
||||
| frontend/src/routes/migration/mappings/+page.svelte | 🟢 100% | OK |
|
||||
| frontend/src/routes/tools/search/+page.svelte | 🟢 100% | OK |
|
||||
| frontend/src/routes/tools/mapper/+page.svelte | 🟢 100% | OK |
|
||||
| frontend/src/routes/tools/debug/+page.svelte | 🟢 100% | OK |
|
||||
| frontend/src/routes/settings/+page.svelte | 🟢 100% | OK |
|
||||
| frontend/src/routes/settings/+page.ts | 🟢 100% | OK |
|
||||
| frontend/src/routes/settings/connections/+page.svelte | 🟢 100% | OK |
|
||||
| frontend/src/routes/settings/git/+page.svelte | 🟢 100% | OK |
|
||||
| frontend/src/routes/git/+page.svelte | 🟢 100% | OK |
|
||||
| frontend/src/pages/Dashboard.svelte | 🟢 100% | OK |
|
||||
| frontend/src/pages/Settings.svelte | 🟢 100% | OK |
|
||||
| frontend/src/services/connectionService.js | 🟢 100% | OK |
|
||||
| frontend/src/services/gitService.js | 🟢 100% | OK |
|
||||
| frontend/src/services/toolsService.js | 🟢 100% | OK |
|
||||
| frontend/src/services/taskService.js | 🟢 100% | OK |
|
||||
| frontend/src/services/storageService.js | 🟢 100% | OK |
|
||||
| frontend/src/components/PasswordPrompt.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/MappingTable.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/TaskLogViewer.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/Footer.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/MissingMappingModal.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/Navbar.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/TaskHistory.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/Toast.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/TaskRunner.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/TaskList.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/DynamicForm.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/EnvSelector.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/tools/ConnectionForm.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/tools/ConnectionList.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/tools/MapperTool.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/tools/DebugTool.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/tools/SearchTool.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/git/CommitHistory.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/git/ConflictResolver.svelte | 🟢 100% | OK |
|
||||
| frontend/src/components/git/GitManager.svelte | 🟢 100% | OK |
|
||||
| backend/delete_running_tasks.py | 🟢 100% | [delete_running_tasks] Missing Belief State Logging: Function should use belief_scope context manager.<br>[delete_running_tasks] Missing Belief State Logging: Function should use belief_scope context manager. |
|
||||
| backend/src/app.py | 🟢 100% | OK |
|
||||
| backend/src/dependencies.py | 🟢 100% | OK |
|
||||
| backend/src/core/superset_client.py | 🟢 100% | OK |
|
||||
| backend/src/core/config_manager.py | 🟢 100% | OK |
|
||||
| backend/src/core/scheduler.py | 🟢 100% | OK |
|
||||
| backend/src/core/config_models.py | 🟢 100% | OK |
|
||||
| backend/src/core/database.py | 🟢 100% | OK |
|
||||
| backend/src/core/logger.py | 🟢 100% | OK |
|
||||
| backend/src/core/plugin_loader.py | 🟢 100% | OK |
|
||||
| backend/src/core/migration_engine.py | 🟢 100% | [_transform_yaml] Missing Belief State Logging: Function should use belief_scope context manager.<br>[_transform_yaml] Missing Belief State Logging: Function should use belief_scope context manager.<br>[_transform_yaml] Missing Belief State Logging: Function should use belief_scope context manager. |
|
||||
| backend/src/core/plugin_base.py | 🟢 100% | OK |
|
||||
| backend/src/core/utils/fileio.py | 🟢 100% | [replacer] Missing Belief State Logging: Function should use belief_scope context manager.<br>[replacer] Missing Belief State Logging: Function should use belief_scope context manager.<br>[replacer] Missing Belief State Logging: Function should use belief_scope context manager. |
|
||||
| backend/src/core/utils/network.py | 🟢 100% | OK |
|
||||
| backend/src/core/utils/matching.py | 🟢 100% | [suggest_mappings] Missing Belief State Logging: Function should use belief_scope context manager.<br>[suggest_mappings] Missing Belief State Logging: Function should use belief_scope context manager. |
|
||||
| backend/src/core/task_manager/persistence.py | 🟢 100% | OK |
|
||||
| backend/src/core/task_manager/manager.py | 🟢 100% | OK |
|
||||
| backend/src/core/task_manager/models.py | 🟢 100% | [__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager. |
|
||||
| backend/src/core/task_manager/cleanup.py | 🟢 100% | [__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager. |
|
||||
| backend/src/core/task_manager/__init__.py | 🟢 100% | OK |
|
||||
| backend/src/api/auth.py | 🟢 100% | [get_current_user] Missing Belief State Logging: Function should use belief_scope context manager.<br>[get_current_user] Missing Belief State Logging: Function should use belief_scope context manager. |
|
||||
| backend/src/api/routes/git.py | 🟢 100% | OK |
|
||||
| backend/src/api/routes/connections.py | 🟢 100% | OK |
|
||||
| backend/src/api/routes/environments.py | 🟢 100% | OK |
|
||||
| backend/src/api/routes/migration.py | 🟢 100% | OK |
|
||||
| backend/src/api/routes/plugins.py | 🟢 100% | OK |
|
||||
| backend/src/api/routes/mappings.py | 🟢 100% | OK |
|
||||
| backend/src/api/routes/git_schemas.py | 🟢 100% | OK |
|
||||
| backend/src/api/routes/storage.py | 🟢 100% | OK |
|
||||
| backend/src/api/routes/tasks.py | 🟢 100% | OK |
|
||||
| backend/src/models/git.py | 🟢 100% | OK |
|
||||
| backend/src/models/task.py | 🟢 100% | OK |
|
||||
| backend/src/models/connection.py | 🟢 100% | OK |
|
||||
| backend/src/models/mapping.py | 🟢 100% | OK |
|
||||
| backend/src/models/storage.py | 🟢 100% | OK |
|
||||
| backend/src/models/dashboard.py | 🟢 100% | OK |
|
||||
| backend/src/services/git_service.py | 🟢 100% | OK |
|
||||
| backend/src/services/mapping_service.py | 🟢 100% | OK |
|
||||
| backend/src/plugins/backup.py | 🟢 100% | OK |
|
||||
| backend/src/plugins/debug.py | 🟢 100% | OK |
|
||||
| backend/src/plugins/search.py | 🟢 100% | OK |
|
||||
| backend/src/plugins/mapper.py | 🟢 100% | OK |
|
||||
| backend/src/plugins/git_plugin.py | 🟢 100% | OK |
|
||||
| backend/src/plugins/migration.py | 🟢 100% | OK |
|
||||
| backend/src/plugins/storage/plugin.py | 🟢 100% | OK |
|
||||
| backend/tests/test_models.py | 🟢 100% | OK |
|
||||
| backend/tests/test_logger.py | 🟢 100% | OK |
|
||||
File diff suppressed because it is too large
Load Diff
@@ -35,7 +35,7 @@
|
||||
|
||||
## Final Phase: Polish & Cross-cutting concerns
|
||||
- [ ] T022 Implement task cleanup/retention policy (e.g., delete tasks older than 30 days)
|
||||
- [ ] T023 Add real-time updates for task status using WebSockets (optional/refinement)
|
||||
- [x] T023 Add real-time updates for task status using WebSockets (optional/refinement)
|
||||
- [x] T024 Ensure consistent error handling and logging across scheduler and task manager
|
||||
|
||||
## Dependencies
|
||||
|
||||
34
specs/014-file-storage-ui/checklists/requirements.md
Normal file
34
specs/014-file-storage-ui/checklists/requirements.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Specification Quality Checklist: File Storage Management & UI
|
||||
|
||||
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
||||
**Created**: 2026-01-24
|
||||
**Feature**: [Link to spec.md](../spec.md)
|
||||
|
||||
## Content Quality
|
||||
|
||||
- [x] No implementation details (languages, frameworks, APIs)
|
||||
- [x] Focused on user value and business needs
|
||||
- [x] Written for non-technical stakeholders
|
||||
- [x] All mandatory sections completed
|
||||
|
||||
## Requirement Completeness
|
||||
|
||||
- [x] No [NEEDS CLARIFICATION] markers remain
|
||||
- [x] Requirements are testable and unambiguous
|
||||
- [x] Success criteria are measurable
|
||||
- [x] Success criteria are technology-agnostic (no implementation details)
|
||||
- [x] All acceptance scenarios are defined
|
||||
- [x] Edge cases are identified
|
||||
- [x] Scope is clearly bounded
|
||||
- [x] Dependencies and assumptions identified
|
||||
|
||||
## Feature Readiness
|
||||
|
||||
- [x] All functional requirements have clear acceptance criteria
|
||||
- [x] User scenarios cover primary flows
|
||||
- [x] Feature meets measurable outcomes defined in Success Criteria
|
||||
- [x] No implementation details leak into specification
|
||||
|
||||
## Notes
|
||||
|
||||
- Items marked incomplete require spec updates before `/speckit.clarify` or `/speckit.plan`
|
||||
35
specs/014-file-storage-ui/checklists/ux.md
Normal file
35
specs/014-file-storage-ui/checklists/ux.md
Normal file
@@ -0,0 +1,35 @@
|
||||
# Checklist: File Storage UX & Configuration
|
||||
|
||||
**Purpose**: Validate implementation of User Experience and Configuration Flexibility requirements.
|
||||
**Created**: 2026-01-24
|
||||
**Feature**: [File Storage Management & UI](../spec.md)
|
||||
|
||||
## User Experience (File Management)
|
||||
|
||||
- [x] CHK001 Are loading states displayed while fetching the file list? [Completeness]
|
||||
- [x] CHK002 Is visual feedback provided immediately after file upload starts? [Clarity]
|
||||
- [x] CHK003 Are error messages user-friendly when upload fails (e.g., "File too large" vs "Error 413")? [Clarity]
|
||||
- [x] CHK004 Is a confirmation modal shown before permanently deleting a file? [Safety]
|
||||
- [x] CHK005 Does the UI clearly distinguish between "Backups" and "Repositories" tabs? [Clarity]
|
||||
- [x] CHK006 Is the file list sortable by Date and Name? [Usability]
|
||||
- [x] CHK007 Are file sizes formatted in human-readable units (KB, MB, GB)? [Usability]
|
||||
- [x] CHK008 Is the download action easily accessible for each file item? [Accessibility]
|
||||
- [x] CHK009 Does the upload component support drag-and-drop interactions? [Usability]
|
||||
- [x] CHK010 Is the "Upload" button disabled or hidden when no category is selected? [Consistency]
|
||||
|
||||
## Configuration Flexibility
|
||||
|
||||
- [x] CHK011 Can the storage root path be configured to any writable directory on the server? [Flexibility]
|
||||
- [x] CHK012 Does the system support defining custom directory structures using variables like `{environment}`? [Flexibility]
|
||||
- [x] CHK013 Does the system support defining custom filename patterns using variables like `{timestamp}`? [Flexibility]
|
||||
- [ ] CHK014 Are the supported pattern variables (e.g., `{dashboard_name}`) clearly documented in the UI? [Clarity]
|
||||
- [x] CHK015 Can the configuration be updated without restarting the application? [Usability]
|
||||
- [ ] CHK016 Is the current resolved path shown as a preview when editing patterns? [Usability]
|
||||
- [ ] CHK017 Does the system allow reverting configuration to default values? [Recovery]
|
||||
|
||||
## Edge Cases & Error Handling
|
||||
|
||||
- [x] CHK018 Is the UI behavior defined for an empty file list (zero state)? [Coverage]
|
||||
- [ ] CHK019 Is the behavior defined when the configured storage path becomes inaccessible? [Resilience]
|
||||
- [x] CHK020 Are long filenames handled gracefully in the UI (e.g., truncation with tooltip)? [Layout]
|
||||
- [x] CHK021 Is the behavior defined for uploading a file with a duplicate name? [Conflict Resolution]
|
||||
74
specs/014-file-storage-ui/contracts/api.md
Normal file
74
specs/014-file-storage-ui/contracts/api.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# API Contracts: File Storage Management & UI
|
||||
|
||||
## Endpoints
|
||||
|
||||
### GET /api/storage/files
|
||||
List all files in the storage system.
|
||||
|
||||
**Query Parameters:**
|
||||
- `category` (optional): Filter by category (`backup` or `repository`).
|
||||
|
||||
**Response:**
|
||||
- `200 OK`: List of `StoredFile` objects.
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"name": "dashboard_backup_20260124.zip",
|
||||
"path": "backups/dashboard_backup_20260124.zip",
|
||||
"size": 102400,
|
||||
"created_at": "2026-01-24T12:00:00Z",
|
||||
"category": "backup",
|
||||
"mime_type": "application/zip"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
### POST /api/storage/upload
|
||||
Upload a file to the storage system.
|
||||
|
||||
**Form Data:**
|
||||
- `file`: The file content.
|
||||
- `category`: Target category (`backup` or `repository`).
|
||||
|
||||
**Response:**
|
||||
- `201 Created`: The uploaded `StoredFile` object.
|
||||
- `400 Bad Request`: Invalid category or file.
|
||||
|
||||
### DELETE /api/storage/files/{category}/{filename}
|
||||
Delete a file from storage.
|
||||
|
||||
**Path Parameters:**
|
||||
- `category`: `backup` or `repository`.
|
||||
- `filename`: Name of the file to delete.
|
||||
|
||||
**Response:**
|
||||
- `204 No Content`: File deleted successfully.
|
||||
- `404 Not Found`: File does not exist.
|
||||
|
||||
### GET /api/storage/download/{category}/{filename}
|
||||
Download a file.
|
||||
|
||||
**Path Parameters:**
|
||||
- `category`: `backup` or `repository`.
|
||||
- `filename`: Name of the file to download.
|
||||
|
||||
**Response:**
|
||||
- `200 OK`: File stream.
|
||||
- `404 Not Found`: File does not exist.
|
||||
|
||||
### GET /api/settings/storage
|
||||
Get current storage configuration.
|
||||
|
||||
**Response:**
|
||||
- `200 OK`: `StorageConfig` object.
|
||||
|
||||
### PUT /api/settings/storage
|
||||
Update storage configuration.
|
||||
|
||||
**Body:**
|
||||
- `StorageConfig` object.
|
||||
|
||||
**Response:**
|
||||
- `200 OK`: Updated `StorageConfig`.
|
||||
- `400 Bad Request`: Invalid path or not writable.
|
||||
34
specs/014-file-storage-ui/data-model.md
Normal file
34
specs/014-file-storage-ui/data-model.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Data Model: File Storage Management & UI
|
||||
|
||||
## Entities
|
||||
|
||||
### StorageConfig
|
||||
*Configuration for the storage system.*
|
||||
|
||||
| Field | Type | Description | Constraints |
|
||||
|---|---|---|---|
|
||||
| `root_path` | `string` | Absolute path to the storage root directory. | Must be a valid, writable path. Default: `../ss-tools-storage` |
|
||||
| `backup_structure_pattern` | `string` | Pattern for backup directory structure. | Default: `{category}/` |
|
||||
| `repo_structure_pattern` | `string` | Pattern for repository directory structure. | Default: `{category}/` |
|
||||
| `filename_pattern` | `string` | Pattern for filenames. | Default: `{name}_{timestamp}` |
|
||||
|
||||
### StoredFile
|
||||
*Representation of a file in the storage system.*
|
||||
|
||||
| Field | Type | Description | Constraints |
|
||||
|---|---|---|---|
|
||||
| `name` | `string` | Name of the file (including extension). | No path separators. |
|
||||
| `path` | `string` | Relative path from storage root. | |
|
||||
| `size` | `integer` | Size of the file in bytes. | >= 0 |
|
||||
| `created_at` | `datetime` | Creation timestamp. | |
|
||||
| `category` | `enum` | Category of the file. | `backup`, `repository` |
|
||||
| `mime_type` | `string` | MIME type of the file. | Optional |
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```text
|
||||
{root_path}/
|
||||
├── backups/
|
||||
│ └── {filename}.zip
|
||||
└── repositories/
|
||||
└── {filename}.zip
|
||||
106
specs/014-file-storage-ui/plan.md
Normal file
106
specs/014-file-storage-ui/plan.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# Implementation Plan: File Storage Management & UI
|
||||
|
||||
**Branch**: `014-file-storage-ui` | **Date**: 2026-01-24 | **Spec**: [specs/014-file-storage-ui/spec.md](../014-file-storage-ui/spec.md)
|
||||
**Input**: Feature specification from `specs/014-file-storage-ui/spec.md`
|
||||
|
||||
## Summary
|
||||
|
||||
This feature implements a managed file storage system for dashboard backups and exported repositories. It introduces a configurable storage root (defaulting to outside the workspace) and a Web UI to list, upload, download, and delete files. The system enforces a structured layout with `backups/` and `repositories/` subdirectories to keep artifacts organized. The UI supports hierarchical folder navigation (e.g., `backups/SS2/DashboardName`), allowing users to browse, download, and manage files within nested directories.
|
||||
|
||||
## Technical Context
|
||||
|
||||
**Language/Version**: Python 3.9+ (Backend), Node.js 18+ (Frontend)
|
||||
**Primary Dependencies**: FastAPI (Backend), SvelteKit (Frontend)
|
||||
**Storage**: Local Filesystem (for artifacts), Config (for storage path)
|
||||
**Testing**: pytest (Backend), vitest/playwright (Frontend - implied)
|
||||
**Target Platform**: Linux server
|
||||
**Project Type**: Web application
|
||||
**Performance Goals**: File list load < 1s for 100 files, supports 50MB+ uploads
|
||||
**Constraints**: Must prevent path traversal, must not pollute git repo
|
||||
**Scale/Scope**: ~2-3 backend endpoints, 1-2 frontend pages/components
|
||||
|
||||
## Constitution Check
|
||||
|
||||
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
|
||||
|
||||
- **I. Semantic Protocol Compliance**:
|
||||
- **Status**: PASSED
|
||||
- **Check**: Will use `[DEF]` anchors and `@RELATION` tags.
|
||||
- **Check**: Will follow File Structure Standard.
|
||||
|
||||
- **II. Causal Validity (Contracts First)**:
|
||||
- **Status**: PASSED
|
||||
- **Check**: Contracts will be defined in `specs/014-file-storage-ui/contracts/` before implementation.
|
||||
|
||||
- **III. Immutability of Architecture**:
|
||||
- **Status**: PASSED
|
||||
- **Check**: No changes to immutable architectural constraints expected.
|
||||
|
||||
- **IV. Design by Contract (DbC)**:
|
||||
- **Status**: PASSED
|
||||
- **Check**: Functions will define `@PRE` and `@POST` conditions.
|
||||
|
||||
- **V. Belief State Logging**:
|
||||
- **Status**: PASSED
|
||||
- **Check**: Will use standard logging patterns.
|
||||
|
||||
- **VI. Fractal Complexity Limit**:
|
||||
- **Status**: PASSED
|
||||
- **Check**: Feature scope is small, unlikely to exceed complexity limits.
|
||||
|
||||
- **VII. Everything is a Plugin**:
|
||||
- **Status**: PASSED
|
||||
- **Check**: New functionality will be implemented as a `StoragePlugin` (or similar) inheriting from `PluginBase`.
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Documentation (this feature)
|
||||
|
||||
```text
|
||||
specs/014-file-storage-ui/
|
||||
├── plan.md # This file
|
||||
├── research.md # Phase 0 output
|
||||
├── data-model.md # Phase 1 output
|
||||
├── quickstart.md # Phase 1 output
|
||||
├── contracts/ # Phase 1 output
|
||||
└── tasks.md # Phase 2 output
|
||||
```
|
||||
|
||||
### Source Code (repository root)
|
||||
|
||||
```text
|
||||
backend/
|
||||
├── src/
|
||||
│ ├── api/
|
||||
│ │ └── routes/
|
||||
│ │ └── storage.py # New route handler
|
||||
│ ├── plugins/
|
||||
│ │ └── storage.py # New plugin implementation
|
||||
│ └── models/
|
||||
│ └── storage.py # Pydantic models (StoredFile, StorageConfig)
|
||||
└── tests/
|
||||
└── test_storage.py # Backend tests
|
||||
|
||||
frontend/
|
||||
├── src/
|
||||
│ ├── routes/
|
||||
│ │ └── storage/
|
||||
│ │ └── +page.svelte # Main storage UI
|
||||
│ ├── components/
|
||||
│ │ └── storage/
|
||||
│ │ ├── FileList.svelte # Component for listing files and folders (explorer view)
|
||||
│ │ ├── Breadcrumbs.svelte # Component for navigation
|
||||
│ │ └── FileUpload.svelte # Component for uploading
|
||||
│ └── services/
|
||||
│ └── storageService.js # Frontend API client
|
||||
```
|
||||
|
||||
**Structure Decision**: Standard Web Application structure (Backend + Frontend) with Plugin architecture for the backend logic.
|
||||
|
||||
## Complexity Tracking
|
||||
|
||||
> **Fill ONLY if Constitution Check has violations that must be justified**
|
||||
|
||||
| Violation | Why Needed | Simpler Alternative Rejected Because |
|
||||
|-----------|------------|-------------------------------------|
|
||||
| N/A | | |
|
||||
31
specs/014-file-storage-ui/quickstart.md
Normal file
31
specs/014-file-storage-ui/quickstart.md
Normal file
@@ -0,0 +1,31 @@
|
||||
# Quickstart: File Storage Management & UI
|
||||
|
||||
## Usage Guide
|
||||
|
||||
1. **Access File Storage**: Navigate to the "Tools" > "File Storage" section in the main navigation.
|
||||
2. **View Files**: You will see two tabs: "Backups" and "Repositories". Click on a tab to view files in that category.
|
||||
3. **Upload File**:
|
||||
- Click the "Upload" button.
|
||||
- Select a file from your computer.
|
||||
- Choose the target category (Backup or Repository).
|
||||
- Click "Upload" to start the transfer.
|
||||
4. **Download File**: Click the "Download" icon next to any file in the list.
|
||||
5. **Delete File**: Click the "Trash" icon next to any file to delete it permanently.
|
||||
6. **Configure Storage Path**:
|
||||
- Go to "Settings".
|
||||
- Locate the "File Storage" section.
|
||||
- Enter a new absolute path for the storage root.
|
||||
- Click "Save". The system will verify write access to the new path.
|
||||
|
||||
## Development
|
||||
|
||||
### Backend
|
||||
|
||||
- **Plugin**: `backend/src/plugins/storage.py`
|
||||
- **API Routes**: `backend/src/api/routes/storage.py`
|
||||
- **Models**: `backend/src/models/storage.py`
|
||||
|
||||
### Frontend
|
||||
|
||||
- **Page**: `frontend/src/routes/tools/storage/+page.svelte`
|
||||
- **Components**: `frontend/src/components/storage/*`
|
||||
17
specs/014-file-storage-ui/research.md
Normal file
17
specs/014-file-storage-ui/research.md
Normal file
@@ -0,0 +1,17 @@
|
||||
# Research: File Storage Management & UI
|
||||
|
||||
**Decision**: Use Python's built-in `pathlib` and `shutil` for filesystem operations.
|
||||
**Rationale**: Standard library, robust, cross-platform (though target is Linux), and sufficient for local file management. No external dependencies needed.
|
||||
**Alternatives considered**: `os` module (lower level, less ergonomic), `pyfilesystem2` (external dependency, unnecessary overhead for simple local storage).
|
||||
|
||||
**Decision**: Use `pydantic` for configuration and file metadata models.
|
||||
**Rationale**: Already used in the project, provides validation and serialization.
|
||||
|
||||
**Decision**: Use `multipart/form-data` for file uploads.
|
||||
**Rationale**: Standard web practice for file uploads. FastAPI supports `UploadFile` natively.
|
||||
|
||||
**Decision**: Default storage path strategy.
|
||||
**Rationale**: The default path will be `../ss-tools-storage` (relative to workspace root) or similar to ensure it sits outside the git repository by default, but allows configuration override.
|
||||
|
||||
**Decision**: Path Traversal Prevention.
|
||||
**Rationale**: Will use `os.path.commonpath` or `pathlib.Path.resolve()` to strictly validate that any accessed file path is a child of the configured storage root.
|
||||
97
specs/014-file-storage-ui/spec.md
Normal file
97
specs/014-file-storage-ui/spec.md
Normal file
@@ -0,0 +1,97 @@
|
||||
# Feature Specification: File Storage Management & UI
|
||||
|
||||
**Feature Branch**: `014-file-storage-ui`
|
||||
**Created**: 2026-01-24
|
||||
**Status**: Draft
|
||||
**Input**: User description: "Я хочу проработать механизм хранения файлов и доступа к ним - бекапов дашбордов и репозиториев дашбордов. Во первых, нужно иметь указывать место хранения, по умолчанию оно должно быть за файловой системой сервера (чтобы не влиять на git репозиторий. Во вторых, нужен web ui для базового доступа ко всем файлам - возможность скачивания/удаления, информация о датах создания, возможность загрузки"
|
||||
|
||||
## User Scenarios & Testing *(mandatory)*
|
||||
|
||||
### User Story 1 - File Management Dashboard (Priority: P1)
|
||||
|
||||
Users need a visual interface to manage the artifacts generated by the system (dashboard backups, exported repositories) without needing direct server access. Users must be able to navigate through the folder structure (e.g., `backups/SS2/Sales Dashboard`) to locate specific files.
|
||||
|
||||
**Why this priority**: Core functionality requested. Without the UI, the storage mechanism is opaque and hard to use.
|
||||
|
||||
**Independent Test**: Can be fully tested by opening the new "File Storage" page, navigating into a subdirectory, uploading a test file, verifying it appears in the list, downloading it, and then deleting it.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** the File Storage page is open, **When** I view the list, **Then** I see the top-level folders (e.g., `backups`, `repositories`) or files.
|
||||
2. **Given** I am viewing a folder, **When** I click a subfolder name, **Then** the view updates to show the contents of that subfolder.
|
||||
3. **Given** I am in a subfolder, **When** I click "Download" on a file, **Then** the file is downloaded to my local machine.
|
||||
4. **Given** a file exists in the list, **When** I click "Delete" and confirm, **Then** the file is removed from the list and the server filesystem.
|
||||
5. **Given** I have a file locally, **When** I drag and drop it or use the "Upload" button, **Then** the file is uploaded to the current directory and appears in the list.
|
||||
|
||||
---
|
||||
|
||||
### User Story 2 - Storage Location Configuration (Priority: P2)
|
||||
|
||||
Administrators need to control where potentially large or sensitive files are stored. Crucially, these files must not accidentally pollute the source code repository or the application's working directory.
|
||||
|
||||
**Why this priority**: Essential for system stability and cleanliness (preventing git pollution), but the system could theoretically start with a hardcoded safe default.
|
||||
|
||||
**Independent Test**: Change the storage path in Settings, generate a file (or upload one), and verify it exists in the new location on the server disk.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** I am in the Settings page, **When** I enter a new absolute path for "File Storage Path" and save, **Then** the system updates the configuration.
|
||||
2. **Given** the default configuration, **When** the system starts, **Then** the storage path defaults to a location outside the project's git scope (or is properly ignored).
|
||||
3. **Given** an invalid path (e.g., no write permissions), **When** I try to save, **Then** the system shows an error message.
|
||||
|
||||
---
|
||||
|
||||
### Edge Cases
|
||||
|
||||
- **File Name Conflicts**: What happens when uploading a file that already exists? (System should likely rename or ask to overwrite).
|
||||
- **Storage Quota/Disk Space**: What happens if the disk is full during upload?
|
||||
- **Path Traversal**: Ensure users cannot access files outside the configured storage directory via the API.
|
||||
- **Large Files**: Handling uploads/downloads of large backup archives (e.g., > 100MB).
|
||||
|
||||
## Clarifications
|
||||
|
||||
### Session 2026-01-24
|
||||
|
||||
- Q: Should the system enforce a structure to keep backups and repositories separate? → A: **Structured**: System creates and enforces `backups/` and `repositories/` folders; UI separates them.
|
||||
|
||||
### Session 2026-01-24 (Update)
|
||||
|
||||
- Q: Should the system allow advanced configuration of file structure and naming conventions? → A: **Yes**: Users should be able to configure the directory structure (e.g., include dashboard/environment names) and file naming patterns (e.g., include timestamps).
|
||||
|
||||
## Requirements *(mandatory)*
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
- **FR-001**: System MUST allow configuring a local filesystem root path for storing artifacts.
|
||||
- **FR-002**: The default storage path MUST be configured such that it does not interfere with the application's git repository (e.g., a directory outside the workspace or explicitly git-ignored).
|
||||
- **FR-003**: System MUST enforce a directory structure within the storage root: `backups/` for dashboard backups and `repositories/` for exported repositories.
|
||||
- **FR-004**: System MUST provide a Web UI to list files and folders, organized by their type (Backup vs Repository).
|
||||
- **FR-005**: System MUST display file metadata in the UI: Filename, Size, Creation Date.
|
||||
- **FR-006**: System MUST allow users to download files from the storage directory (including subdirectories) via the Web UI.
|
||||
- **FR-007**: System MUST allow users to delete files from the storage directory via the Web UI.
|
||||
- **FR-008**: System MUST allow users to upload files to the specific folder in the storage directory via the Web UI.
|
||||
- **FR-013**: System MUST support navigating through the directory hierarchy within the allowed categories.
|
||||
- **FR-014**: System MUST display breadcrumbs or similar navigation aid to show current path.
|
||||
- **FR-009**: System MUST validate that the configured storage path is accessible and writable.
|
||||
- **FR-010**: System MUST prevent access to files outside the configured storage directory (Path Traversal protection).
|
||||
- **FR-011**: System MUST allow configuring the directory structure pattern for backups and repositories (e.g., `{environment}/{dashboard_name}/`).
|
||||
- **FR-012**: System MUST allow configuring the filename pattern for generated files (e.g., `{dashboard_name}_{timestamp}.zip`).
|
||||
|
||||
### Key Entities
|
||||
|
||||
- **StorageConfig**: Settings defining the root directory path.
|
||||
- **StoredFile**: Conceptual representation of a file (Name, Path, Size, CreatedAt, MimeType).
|
||||
|
||||
### Assumptions
|
||||
|
||||
- The application server has access to a writable local filesystem.
|
||||
- Users utilizing this feature have appropriate permissions within the application to manage system-wide storage settings and files.
|
||||
|
||||
## Success Criteria *(mandatory)*
|
||||
|
||||
### Measurable Outcomes
|
||||
|
||||
- **SC-001**: Users can successfully upload and then download a file of at least 50MB size.
|
||||
- **SC-002**: Files created or uploaded via the system do not appear in the application's `git status` output by default.
|
||||
- **SC-003**: File list loads in under 1 second for a directory containing 100 files.
|
||||
- **SC-004**: Users can delete a file via UI and confirm it is physically removed from the disk.
|
||||
96
specs/014-file-storage-ui/tasks.md
Normal file
96
specs/014-file-storage-ui/tasks.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# Tasks: File Storage Management & UI
|
||||
|
||||
**Branch**: `014-file-storage-ui` | **Spec**: [specs/014-file-storage-ui/spec.md](../014-file-storage-ui/spec.md)
|
||||
|
||||
## Phase 1: Setup
|
||||
*Goal: Initialize backend plugin structure and frontend route scaffolding.*
|
||||
|
||||
- [x] T001 Create storage plugin directory and `__init__.py` in `backend/src/plugins/storage/`
|
||||
- [x] T002 Create storage models file `backend/src/models/storage.py` with `StorageConfig` and `StoredFile` Pydantic models
|
||||
- [x] T003 Create empty storage route handler `backend/src/api/routes/storage.py` and register in `backend/src/api/routes/__init__.py`
|
||||
- [x] T004 Create frontend storage route directory `frontend/src/routes/tools/storage/` and empty `+page.svelte`
|
||||
- [x] T005 Create frontend service `frontend/src/services/storageService.js` stub
|
||||
|
||||
## Phase 2: Foundational
|
||||
*Goal: Implement core backend logic for storage management, configuration, and security.*
|
||||
|
||||
- [x] T006 Implement `StoragePlugin` class in `backend/src/plugins/storage/plugin.py` inheriting from `PluginBase`
|
||||
- [x] T007 Implement `get_storage_root()` method in `StoragePlugin` with default path logic (`../ss-tools-storage`)
|
||||
- [x] T008 Implement `ensure_directories()` method to create `backups/` and `repositories/` subfolders on init
|
||||
- [x] T009 Implement path traversal protection helper `validate_path(path)` in `StoragePlugin`
|
||||
- [x] T010 Implement `list_files(category)` method in `StoragePlugin` returning `StoredFile` objects
|
||||
- [x] T011 Implement `save_file(file, category)` method in `StoragePlugin` handling uploads
|
||||
- [x] T012 Implement `delete_file(category, filename)` method in `StoragePlugin`
|
||||
- [x] T013 Implement `get_file_path(category, filename)` method in `StoragePlugin` for downloads
|
||||
- [x] T014 Register `StoragePlugin` in `backend/src/core/plugin_loader.py` (if manual registration needed)
|
||||
|
||||
## Phase 3: User Story 1 - File Management Dashboard (Priority: P1)
|
||||
*Goal: Enable users to list, upload, download, and delete files via Web UI.*
|
||||
|
||||
### Backend Endpoints
|
||||
- [x] T015 [US1] Implement `GET /api/storage/files` endpoint in `backend/src/api/routes/storage.py` using `StoragePlugin.list_files`
|
||||
- [x] T016 [US1] Implement `POST /api/storage/upload` endpoint in `backend/src/api/routes/storage.py` using `StoragePlugin.save_file`
|
||||
- [x] T017 [US1] Implement `DELETE /api/storage/files/{category}/{filename}` endpoint in `backend/src/api/routes/storage.py`
|
||||
- [x] T018 [US1] Implement `GET /api/storage/download/{category}/{filename}` endpoint in `backend/src/api/routes/storage.py`
|
||||
|
||||
### Frontend Implementation
|
||||
- [x] T019 [US1] Implement `listFiles`, `uploadFile`, `deleteFile`, `downloadFileUrl` in `frontend/src/services/storageService.js`
|
||||
- [x] T020 [US1] Create `frontend/src/components/storage/FileList.svelte` to display files in a table with metadata
|
||||
- [x] T021 [US1] Create `frontend/src/components/storage/FileUpload.svelte` with category selection and drag-drop support
|
||||
- [x] T022 [US1] Implement main logic in `frontend/src/routes/tools/storage/+page.svelte` to fetch files and handle tabs (Backups vs Repositories)
|
||||
- [x] T023 [US1] Integrate `FileList` and `FileUpload` components into `+page.svelte`
|
||||
|
||||
## Phase 4: User Story 2 - Storage Location Configuration (Priority: P2)
|
||||
*Goal: Allow administrators to configure the storage root path via Settings.*
|
||||
|
||||
### Backend
|
||||
- [x] T024 [US2] Add `storage_path` field to main configuration model in `backend/src/core/config_models.py` (if not using separate storage config)
|
||||
- [x] T025 [US2] Implement `GET /api/settings/storage` and `PUT /api/settings/storage` endpoints in `backend/src/api/routes/settings.py` (or `storage.py`)
|
||||
- [x] T026 [US2] Update `StoragePlugin` to read root path from global configuration instead of hardcoded default
|
||||
- [x] T027 [US2] Add validation logic to `PUT` endpoint to ensure new path is writable
|
||||
|
||||
### Frontend
|
||||
- [x] T028 [US2] Add `getStorageConfig` and `updateStorageConfig` to `frontend/src/services/storageService.js`
|
||||
- [x] T029 [US2] Create configuration section in `frontend/src/routes/settings/+page.svelte` (or dedicated Storage Settings component)
|
||||
- [x] T030 [US2] Implement form to update storage path with validation feedback
|
||||
- [x] T031 [US2] Add configuration fields for directory structure and filename patterns in `backend/src/models/storage.py` and `frontend/src/routes/settings/+page.svelte`
|
||||
- [x] T032 [US2] Implement logic in `StoragePlugin` to resolve dynamic paths based on configured patterns
|
||||
|
||||
## Phase 5: Polish & Cross-Cutting
|
||||
*Goal: Finalize UI/UX and ensure robustness.*
|
||||
|
||||
- [x] T033 Add link to "File Storage" in main navigation `frontend/src/components/Navbar.svelte`
|
||||
- [x] T034 Add error handling toasts for failed uploads or file operations
|
||||
- [x] T035 Verify large file upload support (50MB+) in Nginx/FastAPI config if applicable
|
||||
- [x] T036 Add confirmation modal for file deletion
|
||||
|
||||
## Phase 6: Folder Structure Support (Refactor)
|
||||
*Goal: Enable hierarchical navigation, nested file management, and downloading.*
|
||||
|
||||
- [x] T037 Refactor `StoragePlugin.list_files` in `backend/src/plugins/storage/plugin.py` to accept `subpath` and return directories/files
|
||||
- [x] T038 Refactor `StoragePlugin` methods (`save_file`, `delete_file`, `get_file_path`) to support nested paths
|
||||
- [x] T039 Update backend endpoints in `backend/src/api/routes/storage.py` (`GET /files`, `POST /upload`, `DELETE /files`, `GET /download`) to accept `path` parameter
|
||||
- [x] T040 Update `frontend/src/services/storageService.js` to pass `path` argument in all API calls
|
||||
- [x] T041 Update `frontend/src/components/storage/FileList.svelte` to display folder icons, handle navigation events, and show breadcrumbs
|
||||
- [x] T042 Update `frontend/src/components/storage/FileUpload.svelte` to upload to the currently active directory
|
||||
- [x] T043 Update `frontend/src/routes/tools/storage/+page.svelte` to manage current path state and handle navigation logic
|
||||
|
||||
## Dependencies
|
||||
|
||||
1. **Phase 1 (Setup)**: No dependencies.
|
||||
2. **Phase 2 (Foundational)**: Depends on Phase 1.
|
||||
3. **Phase 3 (US1)**: Depends on Phase 2.
|
||||
4. **Phase 4 (US2)**: Depends on Phase 2. Can run parallel to Phase 3.
|
||||
5. **Phase 5 (Polish)**: Depends on Phase 3 and 4.
|
||||
6. **Phase 6 (Refactor)**: Depends on Phase 3.
|
||||
|
||||
## Parallel Execution Examples
|
||||
|
||||
- **Backend/Frontend Split**: T015-T018 (Backend Endpoints) can be developed in parallel with T020-T021 (Frontend Components) using mock data.
|
||||
- **Story Split**: US1 (File Management) and US2 (Configuration) are largely independent after Phase 2 is complete.
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
1. **MVP**: Complete Phases 1, 2, and 3. This delivers a working file manager with a default storage location.
|
||||
2. **Full Feature**: Complete Phase 4 to allow path configuration.
|
||||
3. **Polish**: Complete Phase 5 for better UX.
|
||||
34
specs/015-frontend-nav-redesign/checklists/requirements.md
Normal file
34
specs/015-frontend-nav-redesign/checklists/requirements.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Specification Quality Checklist: Frontend Navigation Redesign
|
||||
|
||||
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
||||
**Created**: 2026-01-26
|
||||
**Feature**: [specs/015-frontend-nav-redesign/spec.md](../spec.md)
|
||||
|
||||
## Content Quality
|
||||
|
||||
- [x] No implementation details (languages, frameworks, APIs)
|
||||
- [x] Focused on user value and business needs
|
||||
- [x] Written for non-technical stakeholders
|
||||
- [x] All mandatory sections completed
|
||||
|
||||
## Requirement Completeness
|
||||
|
||||
- [x] No [NEEDS CLARIFICATION] markers remain
|
||||
- [x] Requirements are testable and unambiguous
|
||||
- [x] Success criteria are measurable
|
||||
- [x] Success criteria are technology-agnostic (no implementation details)
|
||||
- [x] All acceptance scenarios are defined
|
||||
- [x] Edge cases are identified
|
||||
- [x] Scope is clearly bounded
|
||||
- [x] Dependencies and assumptions identified
|
||||
|
||||
## Feature Readiness
|
||||
|
||||
- [x] All functional requirements have clear acceptance criteria
|
||||
- [x] User scenarios cover primary flows
|
||||
- [x] Feature meets measurable outcomes defined in Success Criteria
|
||||
- [x] No implementation details leak into specification
|
||||
|
||||
## Notes
|
||||
|
||||
- Items marked incomplete require spec updates before `/speckit.clarify` or `/speckit.plan`
|
||||
@@ -0,0 +1,18 @@
|
||||
# Backup Contracts
|
||||
|
||||
## Component: BackupManager
|
||||
|
||||
### Props
|
||||
None (Top-level page component)
|
||||
|
||||
### Events
|
||||
- `on:backup-create`: Triggered when user requests a new backup.
|
||||
- `on:backup-restore`: Triggered when user requests a restore.
|
||||
|
||||
### Data Dependencies
|
||||
- `GET /api/environments`: Fetch list of available environments.
|
||||
- `GET /api/storage/files?category=backups`: Fetch list of backup files.
|
||||
- `POST /api/tasks`: Create new backup task.
|
||||
- Body: `{ plugin_id: 'superset-backup', params: { environment_id: string } }`
|
||||
- `PUT /api/environments/{id}/schedule`: Update backup schedule.
|
||||
- Body: `{ enabled: boolean, cron_expression: string }`
|
||||
47
specs/015-frontend-nav-redesign/data-model.md
Normal file
47
specs/015-frontend-nav-redesign/data-model.md
Normal file
@@ -0,0 +1,47 @@
|
||||
# Data Model: Frontend Navigation Redesign
|
||||
|
||||
## Plugin Configuration
|
||||
|
||||
The `PluginConfig` model is extended to support backend-driven navigation.
|
||||
|
||||
```python
|
||||
class PluginConfig(BaseModel):
|
||||
"""Pydantic model for plugin configuration."""
|
||||
id: str = Field(..., description="Unique identifier for the plugin")
|
||||
name: str = Field(..., description="Human-readable name for the plugin")
|
||||
description: str = Field(..., description="Brief description of what the plugin does")
|
||||
version: str = Field(..., description="Version of the plugin")
|
||||
ui_route: Optional[str] = Field(None, description="Frontend route for the plugin UI")
|
||||
input_schema: Dict[str, Any] = Field(..., description="JSON schema for input parameters", alias="schema")
|
||||
```
|
||||
|
||||
### ui_route
|
||||
|
||||
- **Type**: `Optional[str]`
|
||||
- **Description**: Specifies the client-side route (URL path) where the plugin's custom UI is hosted.
|
||||
- **Behavior**:
|
||||
- If `None` (default): The dashboard will open the plugin using the generic `DynamicForm` modal.
|
||||
- If set (e.g., `"/tools/mapper"`): The dashboard will navigate (`goto`) to this route when the plugin card is clicked.
|
||||
|
||||
## Backup Management (New)
|
||||
|
||||
### Backup Types
|
||||
|
||||
```typescript
|
||||
// frontend/src/types/backup.ts
|
||||
|
||||
export interface BackupFile {
|
||||
name: string; // e.g., "prod-dashboard-export-2024.zip"
|
||||
path: string; // Relative path in storage
|
||||
size: number; // Bytes
|
||||
created_at: string; // ISO Date
|
||||
category: 'backups'; // Fixed category
|
||||
mime_type?: string;
|
||||
}
|
||||
|
||||
export interface BackupState {
|
||||
isLoading: boolean;
|
||||
files: BackupFile[];
|
||||
error: string | null;
|
||||
selectedBackup: BackupFile | null;
|
||||
}
|
||||
88
specs/015-frontend-nav-redesign/plan.md
Normal file
88
specs/015-frontend-nav-redesign/plan.md
Normal file
@@ -0,0 +1,88 @@
|
||||
# Implementation Plan: Frontend Navigation Redesign
|
||||
|
||||
**Branch**: `015-frontend-nav-redesign` | **Date**: 2026-01-26 | **Spec**: [specs/015-frontend-nav-redesign/spec.md](../spec.md)
|
||||
**Input**: Feature specification from `specs/015-frontend-nav-redesign/spec.md`
|
||||
|
||||
**Note**: This template is filled in by the `/speckit.plan` command. See `.specify/templates/commands/plan.md` for the execution workflow.
|
||||
|
||||
## Summary
|
||||
|
||||
This feature redesigns the frontend navigation to shift from a Navbar-heavy approach to a Dashboard-centric model. Key changes include moving tool access (Mapper, Storage, Backups) to the Dashboard, simplifying the Navbar to global contexts (Tasks, Settings), removing deprecated features (Dataset Search, Environments), and implementing a dedicated Backup Management UI based on backend capabilities from feature 009.
|
||||
|
||||
Additionally, the navigation architecture is refactored to be backend-driven. Plugins now expose a `ui_route` property, allowing the frontend to dynamically determine the correct navigation path without hardcoded mapping.
|
||||
|
||||
## Technical Context
|
||||
|
||||
**Language/Version**: Python 3.9+ (Backend), Node.js 18+ (Frontend)
|
||||
**Primary Dependencies**: FastAPI (Backend), SvelteKit + Tailwind CSS (Frontend)
|
||||
**Storage**: N/A (UI reorganization and API integration)
|
||||
**Testing**: Playwright (E2E - if available), Vitest (Unit)
|
||||
**Target Platform**: Web Browser
|
||||
**Project Type**: Web Application (Frontend + Backend)
|
||||
**Performance Goals**: Instant navigation (<100ms), fast dashboard load
|
||||
**Constraints**: Must maintain responsive design; Backup UI must interface with existing backend endpoints
|
||||
**Scale/Scope**: ~5-10 file modifications, 1 new major component (BackupManager)
|
||||
|
||||
## Constitution Check
|
||||
|
||||
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
|
||||
|
||||
- [x] **Semantic Protocol Compliance**: All new components will follow Svelte component header standards.
|
||||
- [x] **Causal Validity**: Contracts (props/events) will be defined before implementation.
|
||||
- [x] **Immutability of Architecture**: No core architectural changes; only UI reorganization.
|
||||
- [x] **Design by Contract**: New Backup component will define clear interface contracts.
|
||||
- [x] **Everything is a Plugin**: N/A (Frontend changes primarily, backend remains plugin-based).
|
||||
- [x] **Unified Frontend Experience**: All new UI components will use standardized components and internationalization (i18n).
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Documentation (this feature)
|
||||
|
||||
```text
|
||||
specs/015-frontend-nav-redesign/
|
||||
├── plan.md # This file
|
||||
├── research.md # Phase 0 output
|
||||
├── data-model.md # Phase 1 output
|
||||
├── quickstart.md # Phase 1 output
|
||||
├── contracts/ # Phase 1 output
|
||||
└── tasks.md # Phase 2 output
|
||||
```
|
||||
|
||||
### Source Code (repository root)
|
||||
|
||||
```text
|
||||
backend/
|
||||
├── src/
|
||||
│ ├── core/
|
||||
│ │ ├── plugin_base.py # (Modify: Add ui_route property)
|
||||
│ │ └── plugin_loader.py # (Modify: Populate ui_route in PluginConfig)
|
||||
│ ├── plugins/ # (Modify: Implement ui_route in all plugins)
|
||||
│ └── api/routes/ # (Verify backup routes exist)
|
||||
|
||||
frontend/
|
||||
├── src/
|
||||
│ ├── components/
|
||||
│ │ ├── Navbar.svelte # (Modify: Simplify items)
|
||||
│ │ ├── DashboardGrid.svelte # (Modify: Add tool links)
|
||||
│ │ └── backups/ # (New: Backup UI)
|
||||
│ │ ├── BackupManager.svelte
|
||||
│ │ └── BackupList.svelte
|
||||
│ ├── pages/
|
||||
│ │ └── Dashboard.svelte # (Modify: Layout updates)
|
||||
│ └── routes/
|
||||
│ ├── +layout.svelte # (Check global nav injection)
|
||||
│ ├── +page.svelte # (Modify: Use plugin.ui_route for navigation)
|
||||
│ └── tools/
|
||||
│ └── backups/ # (New Route)
|
||||
│ └── +page.svelte
|
||||
```
|
||||
|
||||
**Structure Decision**: Standard SvelteKit structure. New `backups` component directory for the complex backup UI. Route added under `tools/` to match existing pattern (mapper, storage).
|
||||
|
||||
## Complexity Tracking
|
||||
|
||||
> **Fill ONLY if Constitution Check has violations that must be justified**
|
||||
|
||||
| Violation | Why Needed | Simpler Alternative Rejected Because |
|
||||
|-----------|------------|-------------------------------------|
|
||||
| N/A | | |
|
||||
29
specs/015-frontend-nav-redesign/quickstart.md
Normal file
29
specs/015-frontend-nav-redesign/quickstart.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# Quickstart: Frontend Navigation Redesign
|
||||
|
||||
## Overview
|
||||
This feature reorganizes the application navigation to be dashboard-centric and introduces a dedicated UI for Backup Management.
|
||||
|
||||
## New Routes
|
||||
- `/tools/backups`: The new Backup Management interface.
|
||||
- `/`: Dashboard (updated with new tool links).
|
||||
|
||||
## Removed Routes
|
||||
- `/tools/search`: Deprecated and removed.
|
||||
- `/settings/environments`: Deprecated and removed.
|
||||
|
||||
## Development
|
||||
|
||||
### Running the Backup UI
|
||||
1. Ensure the backend is running: `cd backend && uvicorn src.app:app --reload`
|
||||
2. Start the frontend: `cd frontend && npm run dev`
|
||||
3. Navigate to `http://localhost:5173/tools/backups`
|
||||
|
||||
### Key Components
|
||||
- `frontend/src/components/backups/BackupManager.svelte`: Main container for backup operations.
|
||||
- `frontend/src/components/DashboardGrid.svelte`: Updated grid with new tool cards.
|
||||
- `frontend/src/components/Navbar.svelte`: Simplified navigation bar.
|
||||
|
||||
## Verification
|
||||
1. Check Dashboard: Should see cards for Mapper, Storage, and Backups.
|
||||
2. Check Navbar: Should ONLY show Tasks and Settings.
|
||||
3. Check Backup Tool: Should load and display backup status/controls.
|
||||
23
specs/015-frontend-nav-redesign/research.md
Normal file
23
specs/015-frontend-nav-redesign/research.md
Normal file
@@ -0,0 +1,23 @@
|
||||
# Research: Frontend Navigation Redesign
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. Backup Management UI Strategy
|
||||
**Decision**: Create a dedicated `BackupManager` component in `frontend/src/components/backups/`.
|
||||
**Rationale**: The requirement is to have a "full component" accessible from the dashboard. Separating it into its own directory ensures modularity and keeps the Dashboard component clean. It will consume the existing backup APIs (likely `/api/tasks` with specific backup types or a dedicated backup endpoint if one exists - *to be verified in Phase 1*).
|
||||
**Alternatives considered**: Embedding backup controls directly in the Dashboard (rejected: clutters the main view), reusing the TaskRunner component (rejected: need specific backup context/history view).
|
||||
|
||||
### 2. Navigation State Management
|
||||
**Decision**: Use SvelteKit's layout system (`+layout.svelte`) and simple component props for Navbar state.
|
||||
**Rationale**: The Navbar changes are global. Removing items is a static change to the `Navbar.svelte` component. No complex state management (stores) is needed for this structural change.
|
||||
**Alternatives considered**: Dynamic config-based menu (rejected: overkill for this specific redesign).
|
||||
|
||||
### 3. Deprecation Strategy
|
||||
**Decision**: Hard removal of "Dataset Search" and "Deployment Environments" components and routes.
|
||||
**Rationale**: The spec explicitly calls for removal. Keeping dead code increases maintenance burden.
|
||||
**Alternatives considered**: Hiding behind a feature flag (rejected: requirement is explicit removal).
|
||||
|
||||
### 4. Dashboard Grid Layout
|
||||
**Decision**: Update `DashboardGrid.svelte` to include new cards for Mapper, Storage, and Backups.
|
||||
**Rationale**: Reusing the existing grid component maintains consistency.
|
||||
**Alternatives considered**: Creating a separate "Tools" page (rejected: spec requires access from main Dashboard).
|
||||
97
specs/015-frontend-nav-redesign/spec.md
Normal file
97
specs/015-frontend-nav-redesign/spec.md
Normal file
@@ -0,0 +1,97 @@
|
||||
# Feature Specification: Frontend Navigation Redesign
|
||||
|
||||
**Feature Branch**: `015-frontend-nav-redesign`
|
||||
**Created**: 2026-01-26
|
||||
**Status**: Draft
|
||||
**Input**: User description: "Я хочу провести редизайн фронта в части навигации. 1. Удалить Dataset Search (из Navbar и дашборда), Deployment Environments 2. Вкладку Tasks оставить для просмотра всех задач - убрать оттуда кнопку Run backup 3. Должен быть полноценный компонент бэкапов, как мы разрабатывали в 009-backup-scheduler. Доступ - из дашборда главного 4. Перенести ссылку на Dataset mapper из Navbar на дашборд 5. Перенести ссылку на Storage manager на дашборд Общая логика - на дашборде должны быть ссылки на полноценные инструменты, навбар - для настроек и общей Tasks"
|
||||
|
||||
## Clarifications
|
||||
|
||||
### Session 2026-01-26
|
||||
- Q: Do I need to build the Backup Management UI from scratch? → A: Yes, create the UI for backup using data from task 009-backup-scheduler.
|
||||
|
||||
## User Scenarios & Testing *(mandatory)*
|
||||
|
||||
### User Story 1 - Centralized Tool Access via Dashboard (Priority: P1)
|
||||
|
||||
As a user, I want to access all main tools (Backups, Mapper, Storage) from the main Dashboard so that I have a central hub for operations.
|
||||
|
||||
**Why this priority**: This is the core of the redesign, shifting the navigation paradigm to a dashboard-centric model for tools.
|
||||
|
||||
**Independent Test**: Can be tested by verifying the Dashboard contains links/cards for Backups, Mapper, and Storage, and that clicking them navigates to the correct full-page tools.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** I am on the main Dashboard, **When** I look at the available tools, **Then** I see options for "Backup Manager", "Dataset Mapper", and "Storage Manager".
|
||||
2. **Given** I am on the Dashboard, **When** I click "Backup Manager", **Then** I am taken to the full Backup management interface.
|
||||
3. **Given** I am on the Dashboard, **When** I click "Dataset Mapper", **Then** I am taken to the Mapper tool.
|
||||
4. **Given** I am on the Dashboard, **When** I click "Storage Manager", **Then** I am taken to the Storage tool.
|
||||
|
||||
---
|
||||
|
||||
### User Story 2 - Simplified Navigation Bar (Priority: P1)
|
||||
|
||||
As a user, I want a clean Navbar containing only global context items (Tasks, Settings) so that the interface is less cluttered and navigation is distinct from tool usage.
|
||||
|
||||
**Why this priority**: Enforces the separation of concerns between "Global Status/Settings" (Navbar) and "Operational Tools" (Dashboard).
|
||||
|
||||
**Independent Test**: Can be tested by inspecting the Navbar across the application to ensure removed items (Search, Mapper, Environments) are gone and only Tasks and Settings remain.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** I am on any page, **When** I view the Navbar, **Then** I do NOT see links for "Dataset Search", "Dataset Mapper", or "Deployment Environments".
|
||||
2. **Given** I am on any page, **When** I view the Navbar, **Then** I see links for "Tasks" and "Settings".
|
||||
3. **Given** I am on the Tasks page, **When** I look for the "Run backup" button, **Then** it is NOT present (as it belongs in the Backup tool).
|
||||
|
||||
---
|
||||
|
||||
### User Story 3 - Deprecation of Unused Features (Priority: P2)
|
||||
|
||||
As a user, I want removed features (Dataset Search, Deployment Environments) to be inaccessible so that I don't use deprecated workflows.
|
||||
|
||||
**Why this priority**: Cleans up the UI and prevents confusion with features that are being removed or hidden.
|
||||
|
||||
**Independent Test**: Verify that UI elements for Dataset Search and Deployment Environments are removed from both Navbar and Dashboard.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** I am on the Dashboard, **When** I look for "Dataset Search", **Then** it is not visible.
|
||||
2. **Given** I am on the Dashboard or Navbar, **When** I look for "Deployment Environments", **Then** it is not visible.
|
||||
|
||||
### Edge Cases
|
||||
|
||||
- **Direct URL Access**: If a user attempts to access the URL of a removed page (e.g., `/search` or `/environments`) via bookmark or history, they should be redirected to the Dashboard or shown a 404 page (standard app behavior).
|
||||
- **Mobile View**: The simplified Navbar must remain responsive; with fewer items, it should likely avoid collapsing into a hamburger menu unless necessary on very small screens.
|
||||
|
||||
### Assumptions
|
||||
|
||||
- The backend logic and UI components for the "Backup Scheduler" (Feature 009) are available and ready to be integrated into the main Dashboard view.
|
||||
- Existing tools (Dataset Mapper, Storage Manager) function independently of the Navbar context and will work correctly when accessed via the Dashboard.
|
||||
|
||||
## Requirements *(mandatory)*
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
- **FR-001**: System MUST display "Dataset Mapper" entry point on the main Dashboard.
|
||||
- **FR-002**: System MUST display "Storage Manager" entry point on the main Dashboard.
|
||||
- **FR-003**: System MUST display "Backup Scheduler" (or similar name) entry point on the main Dashboard.
|
||||
- **FR-004**: System MUST provide access to the full Backup Management component (newly created based on feature 009 data) via the Dashboard link.
|
||||
- **FR-005**: Navbar MUST NOT contain links to "Dataset Search", "Dataset Mapper", or "Deployment Environments".
|
||||
- **FR-006**: Dashboard MUST NOT contain "Dataset Search" widget or link.
|
||||
- **FR-007**: Tasks page MUST NOT show the "Run backup" button (backup initiation moves to Backup tool).
|
||||
- **FR-008**: Navbar MUST retain "Tasks" and "Settings" links.
|
||||
- **FR-009**: Backup Manager MUST support configuring automated backup schedules (enabled/disabled, cron expression) per environment.
|
||||
- **FR-010**: Backup List MUST provide a "Go to Storage" action that navigates to the Storage Manager with the correct path selected.
|
||||
|
||||
### Key Entities
|
||||
|
||||
- **Dashboard**: The main landing page serving as the registry for tools.
|
||||
- **Navbar**: The persistent top navigation for global application state/config.
|
||||
|
||||
## Success Criteria *(mandatory)*
|
||||
|
||||
### Measurable Outcomes
|
||||
|
||||
- **SC-001**: Users can navigate to Dataset Mapper, Storage Manager, and Backup Manager within 1 click from the Dashboard.
|
||||
- **SC-002**: Navbar contains strictly 0 links to operational tools (Mapper, Search, Storage), containing only Tasks and Settings.
|
||||
- **SC-003**: "Run backup" action is successfully performed via the new Dashboard -> Backup route.
|
||||
116
specs/015-frontend-nav-redesign/tasks.md
Normal file
116
specs/015-frontend-nav-redesign/tasks.md
Normal file
@@ -0,0 +1,116 @@
|
||||
# Tasks: Frontend Navigation Redesign
|
||||
|
||||
**Feature Branch**: `015-frontend-nav-redesign`
|
||||
**Spec**: [specs/015-frontend-nav-redesign/spec.md](../spec.md)
|
||||
**Plan**: [specs/015-frontend-nav-redesign/plan.md](../plan.md)
|
||||
|
||||
## Phase 1: Setup
|
||||
|
||||
*Goal: Initialize project structure for the new feature.*
|
||||
|
||||
- [x] T001 Create backup component directory structure
|
||||
- Path: `frontend/src/components/backups/`
|
||||
- [x] T002 Create backup page route directory
|
||||
- Path: `frontend/src/routes/tools/backups/`
|
||||
|
||||
## Phase 2: Foundational
|
||||
|
||||
*Goal: Prepare core components for integration and verify backend connectivity.*
|
||||
|
||||
- [x] T003 Verify backend API endpoints for backups (via `009-backup-scheduler`)
|
||||
- Path: `backend/src/api/routes/tasks.py` (or relevant backup route)
|
||||
- [x] T004 Define Backup types and interfaces in frontend
|
||||
- Path: `frontend/src/types/backup.ts`
|
||||
|
||||
## Phase 3: User Story 1 - Centralized Tool Access via Dashboard
|
||||
|
||||
*Goal: Update the main dashboard to include all tool entry points.*
|
||||
|
||||
**User Story**: As a user, I want to access all main tools (Backups, Mapper, Storage) from the main Dashboard so that I have a central hub for operations. (P1)
|
||||
|
||||
- [x] T005 [US1] Update DashboardGrid to include "Backup Manager" card
|
||||
- Path: `frontend/src/components/DashboardGrid.svelte`
|
||||
- [x] T006 [US1] Update DashboardGrid to ensure "Dataset Mapper" and "Storage Manager" cards are present
|
||||
- Path: `frontend/src/components/DashboardGrid.svelte`
|
||||
- [x] T007 [US1] Remove "Dataset Search" card from DashboardGrid
|
||||
- Path: `frontend/src/components/DashboardGrid.svelte`
|
||||
|
||||
## Phase 4: User Story 2 - Simplified Navigation Bar
|
||||
|
||||
*Goal: Clean up the Navbar to show only global context items.*
|
||||
|
||||
**User Story**: As a user, I want a clean Navbar containing only global context items (Tasks, Settings) so that the interface is less cluttered and navigation is distinct from tool usage. (P1)
|
||||
|
||||
- [x] T008 [US2] Remove "Dataset Search" link from Navbar
|
||||
- Path: `frontend/src/components/Navbar.svelte`
|
||||
- [x] T009 [US2] Remove "Dataset Mapper" link from Navbar
|
||||
- Path: `frontend/src/components/Navbar.svelte`
|
||||
- [x] T010 [US2] Remove "Deployment Environments" link from Navbar
|
||||
- Path: `frontend/src/components/Navbar.svelte`
|
||||
- [x] T011 [US2] Verify "Tasks" and "Settings" links remain in Navbar
|
||||
- Path: `frontend/src/components/Navbar.svelte`
|
||||
- [x] T012 [US2] Remove "Run backup" button from Tasks page
|
||||
- Path: `frontend/src/routes/tasks/+page.svelte` (or relevant component)
|
||||
|
||||
## Phase 5: Backup Management UI
|
||||
|
||||
*Goal: Implement the dedicated Backup Management interface.*
|
||||
|
||||
**User Story**: (Implicit P1 from FR-004) System MUST provide access to the full Backup Management component via the Dashboard link.
|
||||
|
||||
- [x] T013 [US1] Create BackupList component to display existing backups (Must use `src/lib/ui` components and `src/lib/i18n`)
|
||||
- Path: `frontend/src/components/backups/BackupList.svelte`
|
||||
- [x] T014 [US1] Create BackupManager main component (container) (Must use `src/lib/ui` components and `src/lib/i18n`)
|
||||
- Path: `frontend/src/components/backups/BackupManager.svelte`
|
||||
- [x] T015 [US1] Implement "Create Backup" functionality in BackupManager (Must use `src/lib/ui` components and `src/lib/i18n`)
|
||||
- Path: `frontend/src/components/backups/BackupManager.svelte`
|
||||
- [x] T016 [US1] Implement "Restore Backup" functionality (if supported by backend) (Must use `src/lib/ui` components and `src/lib/i18n`)
|
||||
- Path: `frontend/src/components/backups/BackupManager.svelte`
|
||||
- [x] T017 [US1] Create Backup page to host the manager (Must use `src/lib/ui` components and `src/lib/i18n`)
|
||||
- Path: `frontend/src/routes/tools/backups/+page.svelte`
|
||||
- [x] T017b [US1] Implement Backup Schedule configuration in BackupManager
|
||||
- Path: `frontend/src/components/backups/BackupManager.svelte`
|
||||
- [x] T017c [US1] Implement "Go to Storage" navigation in BackupList
|
||||
- Path: `frontend/src/components/backups/BackupList.svelte`
|
||||
|
||||
## Phase 6: User Story 3 - Deprecation
|
||||
|
||||
*Goal: Remove deprecated routes and code.*
|
||||
|
||||
**User Story**: As a user, I want removed features (Dataset Search, Deployment Environments) to be inaccessible so that I don't use deprecated workflows. (P2)
|
||||
|
||||
- [x] T018 [US3] Delete Dataset Search route
|
||||
- Path: `frontend/src/routes/tools/search/` (delete directory)
|
||||
- [x] T019 [US3] Delete Deployment Environments route
|
||||
- Path: `frontend/src/routes/settings/environments/` (delete directory)
|
||||
- [x] T020 [US3] Delete Dataset Search component (if not used elsewhere)
|
||||
- Path: `frontend/src/components/tools/SearchTool.svelte`
|
||||
- [x] T021 [US3] Delete EnvSelector component (if not used elsewhere)
|
||||
- Path: `frontend/src/components/EnvSelector.svelte`
|
||||
|
||||
## Phase 7: Polish & Cross-Cutting
|
||||
|
||||
*Goal: Final verification and cleanup.*
|
||||
|
||||
- [x] T022 Verify all navigation links work correctly
|
||||
- Path: `frontend/src/components/Navbar.svelte`
|
||||
- [x] T023 Verify responsive layout of new Dashboard grid
|
||||
- Path: `frontend/src/components/DashboardGrid.svelte`
|
||||
- [x] T024 Ensure i18n strings are extracted for new Backup UI
|
||||
- Path: `frontend/src/lib/i18n/` (or relevant locale files)
|
||||
- [x] T025 Verify "Run backup" action successfully triggers backup job (Manual/E2E check)
|
||||
- Path: `frontend/src/components/backups/BackupManager.svelte`
|
||||
|
||||
## Dependencies
|
||||
|
||||
1. **Phase 1 & 2** must be completed first.
|
||||
2. **Phase 3 (Dashboard)** and **Phase 4 (Navbar)** can be done in parallel.
|
||||
3. **Phase 5 (Backup UI)** depends on Phase 1 & 2.
|
||||
4. **Phase 6 (Deprecation)** should be done last to ensure no regressions before removal.
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
1. **Setup**: Create the new directory structure.
|
||||
2. **Dashboard & Navbar**: Quick wins to reshape the navigation.
|
||||
3. **Backup UI**: The core development effort. Connect to existing backend.
|
||||
4. **Cleanup**: Remove old code once the new flows are verified.
|
||||
34
specs/016-multi-user-auth/checklists/requirements.md
Normal file
34
specs/016-multi-user-auth/checklists/requirements.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Specification Quality Checklist: Multi-User Authentication and Authorization
|
||||
|
||||
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
||||
**Created**: 2026-01-26
|
||||
**Feature**: [Link to spec.md](../spec.md)
|
||||
|
||||
## Content Quality
|
||||
|
||||
- [x] No implementation details (languages, frameworks, APIs)
|
||||
- [x] Focused on user value and business needs
|
||||
- [x] Written for non-technical stakeholders
|
||||
- [x] All mandatory sections completed
|
||||
|
||||
## Requirement Completeness
|
||||
|
||||
- [x] No [NEEDS CLARIFICATION] markers remain
|
||||
- [x] Requirements are testable and unambiguous
|
||||
- [x] Success criteria are measurable
|
||||
- [x] Success criteria are technology-agnostic (no implementation details)
|
||||
- [x] All acceptance scenarios are defined
|
||||
- [x] Edge cases are identified
|
||||
- [x] Scope is clearly bounded
|
||||
- [x] Dependencies and assumptions identified
|
||||
|
||||
## Feature Readiness
|
||||
|
||||
- [x] All functional requirements have clear acceptance criteria
|
||||
- [x] User scenarios cover primary flows
|
||||
- [x] Feature meets measurable outcomes defined in Success Criteria
|
||||
- [x] No implementation details leak into specification
|
||||
|
||||
## Notes
|
||||
|
||||
- Items marked incomplete require spec updates before `/speckit.clarify` or `/speckit.plan`
|
||||
39
specs/016-multi-user-auth/checklists/security.md
Normal file
39
specs/016-multi-user-auth/checklists/security.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# Security Requirements Checklist: Multi-User Auth
|
||||
|
||||
**Purpose**: Validate completeness and rigor of security requirements for authentication and authorization.
|
||||
**Created**: 2026-01-27
|
||||
**Feature**: [Link to spec.md](../spec.md)
|
||||
|
||||
## Authentication Security
|
||||
|
||||
- [x] CHK001 Are password complexity requirements specified for local users? [Completeness, Gap] (Covered by T037)
|
||||
- [x] CHK002 Is the exact hashing algorithm (bcrypt) and work factor specified? [Clarity, Spec §Research] (Covered by T006)
|
||||
- [x] CHK003 Are account lockout policies defined for failed login attempts? [Coverage, Gap] (Covered by T033)
|
||||
- [x] CHK004 Is the behavior for inactive/disabled accounts explicitly defined for both local and ADFS users? [Edge Case, Spec §Edge Cases] (Covered by T044)
|
||||
- [x] CHK005 Are requirements defined for session revocation (e.g., logout, admin action)? [Completeness] (Covered by T043)
|
||||
|
||||
## ADFS & SSO Security
|
||||
|
||||
- [x] CHK006 Are token validation requirements (signature, issuer, audience) specified for ADFS OIDC tokens? [Completeness] (Covered by T007)
|
||||
- [x] CHK007 Is the mapping behavior defined when an ADFS user is removed from a mapped AD group? [Edge Case, Gap] (Covered by T028)
|
||||
- [x] CHK008 Are requirements defined for handling ADFS token expiration and refresh? [Coverage] (Covered by T046)
|
||||
- [x] CHK009 Is the JIT provisioning process secure against privilege escalation (e.g., default role)? [Security, Spec §FR-008] (Covered by T028)
|
||||
|
||||
## Authorization & RBAC
|
||||
|
||||
- [x] CHK010 Are "default deny" requirements specified for plugin access? [Clarity, Spec §SC-002] (Covered by T020)
|
||||
- [x] CHK011 Is the behavior defined when a user has multiple roles with conflicting permissions? [Edge Case, Gap] (Covered by T045)
|
||||
- [x] CHK012 Are requirements specified for preventing admins from removing their own admin privileges (lockout prevention)? [Edge Case] (Covered by T022)
|
||||
- [x] CHK013 Is the scope of "Execute" vs "Read" permission clearly defined for each plugin? [Clarity] (Covered by T019)
|
||||
|
||||
## Data Protection
|
||||
|
||||
- [x] CHK014 Are requirements defined for protecting sensitive data (passwords, tokens) in logs? [Completeness, Spec §Constitution] (Covered by T047)
|
||||
- [x] CHK015 Are HttpOnly and Secure flags required for session cookies? [Clarity, Spec §Research] (Covered by T032)
|
||||
- [x] CHK016 Is the storage mechanism for ADFS client secrets defined securely? [Completeness] (Covered by T002)
|
||||
|
||||
## API Security
|
||||
|
||||
- [x] CHK017 Are authentication requirements enforced on ALL API endpoints (except login)? [Coverage] (Covered by T021)
|
||||
- [x] CHK018 Are rate limiting requirements defined for login endpoints to prevent brute force? [Gap] (Covered by T033)
|
||||
- [x] CHK019 Are error messages required to be generic to avoid username enumeration? [Clarity] (Covered by T034)
|
||||
31
specs/016-multi-user-auth/checklists/technical.md
Normal file
31
specs/016-multi-user-auth/checklists/technical.md
Normal file
@@ -0,0 +1,31 @@
|
||||
# Technical Readiness Checklist: Multi-User Auth
|
||||
|
||||
**Purpose**: Validate technical specifications, schema, and API contracts.
|
||||
**Created**: 2026-01-27
|
||||
**Feature**: [Link to spec.md](../spec.md)
|
||||
|
||||
## Data Model & Schema
|
||||
|
||||
- [x] CHK001 Are all necessary fields defined for the `User` entity (e.g., last_login)? [Completeness, Spec §Data Model] (Covered by T004)
|
||||
- [x] CHK002 Are foreign key constraints explicitly defined for `ADGroupMapping`? [Clarity, Spec §Data Model] (Covered by T027)
|
||||
- [x] CHK003 Is the uniqueness constraint for `username` and `email` specified? [Consistency] (Covered by T004)
|
||||
- [x] CHK004 Are database migration requirements defined for the new `auth.db`? [Completeness, Gap] (Covered by T005)
|
||||
|
||||
## API Contracts
|
||||
|
||||
- [x] CHK005 Are request/response schemas defined for the `login` endpoint? [Completeness, Spec §Contracts] (Covered by T009)
|
||||
- [x] CHK006 Are error response codes (401, 403, 404) standardized across all auth endpoints? [Consistency] (Covered by T012)
|
||||
- [x] CHK007 Is the structure of the JWT payload (claims) explicitly defined? [Clarity, Spec §Research] (Covered by T007)
|
||||
- [x] CHK008 Are pagination requirements defined for the "List Users" admin endpoint? [Gap] (Covered by T023)
|
||||
|
||||
## Dependencies & Integration
|
||||
|
||||
- [x] CHK009 Are version requirements specified for `Authlib` and `Passlib`? [Clarity, Spec §Plan] (Covered by T001)
|
||||
- [x] CHK010 Is the dependency on the existing `TaskManager` for plugin execution defined? [Integration] (Covered by T021)
|
||||
- [x] CHK011 Are requirements defined for the CLI admin creation tool? [Completeness, Spec §FR-009] (Covered by T008)
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
- [x] CHK012 Is the maximum acceptable latency for auth verification specified? [Clarity, Spec §Plan] (Covered by T013)
|
||||
- [x] CHK013 Are concurrency requirements defined for the SQLite `auth.db` (WAL mode)? [Completeness, Spec §Research] (Covered by T003)
|
||||
- [x] CHK014 Are logging requirements defined for audit trails (who did what)? [Completeness] (Covered by T047)
|
||||
26
specs/016-multi-user-auth/checklists/testing.md
Normal file
26
specs/016-multi-user-auth/checklists/testing.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# Testing Requirements Checklist: Multi-User Auth
|
||||
|
||||
**Purpose**: Validate test scenario coverage and strategy.
|
||||
**Created**: 2026-01-27
|
||||
**Feature**: [Link to spec.md](../spec.md)
|
||||
|
||||
## Functional Coverage
|
||||
|
||||
- [x] CHK001 Are positive test scenarios defined for Local Login? [Coverage, Spec §US-1] (Covered by T049)
|
||||
- [x] CHK002 Are positive test scenarios defined for ADFS Login (mocked)? [Coverage, Spec §US-3] (Covered by T050)
|
||||
- [x] CHK003 Are negative test scenarios defined for invalid passwords? [Coverage] (Covered by T049)
|
||||
- [x] CHK004 Are negative test scenarios defined for unauthorized plugin access? [Coverage, Spec §US-2] (Covered by T049)
|
||||
- [x] CHK005 Are test scenarios defined for switching between auth methods on the same screen? [Coverage] (Covered by T050)
|
||||
|
||||
## Edge Cases
|
||||
|
||||
- [x] CHK005 Are test scenarios defined for mixed-case username handling? [Edge Case] (Covered by T049)
|
||||
- [x] CHK006 Are test scenarios defined for ADFS JIT provisioning with missing groups? [Edge Case] (Covered by T050)
|
||||
- [x] CHK007 Are test scenarios defined for accessing the API with an expired token? [Edge Case] (Covered by T049)
|
||||
- [x] CHK008 Are test scenarios defined for concurrent login sessions? [Edge Case] (Covered by T049)
|
||||
|
||||
## Integration & System
|
||||
|
||||
- [x] CHK009 Is the strategy defined for mocking ADFS during CI/CD tests? [Completeness] (Covered by T041)
|
||||
- [x] CHK010 Are end-to-end tests required for the full admin user creation flow? [Coverage] (Covered by T050)
|
||||
- [x] CHK011 Are tests required to verify the CLI admin creation tool? [Coverage] (Covered by T049)
|
||||
31
specs/016-multi-user-auth/checklists/ux.md
Normal file
31
specs/016-multi-user-auth/checklists/ux.md
Normal file
@@ -0,0 +1,31 @@
|
||||
# UX Requirements Checklist: Multi-User Auth
|
||||
|
||||
**Purpose**: Validate user experience and interface requirements.
|
||||
**Created**: 2026-01-27
|
||||
**Feature**: [Link to spec.md](../spec.md)
|
||||
|
||||
## Login Flow
|
||||
|
||||
- [x] CHK001 Are feedback requirements defined for invalid credentials (generic message)? [Clarity, Spec §US-1] (Covered by T016)
|
||||
- [x] CHK002 Is the redirect behavior specified after successful login (dashboard vs deep link)? [Clarity, Spec §US-1] (Covered by T016)
|
||||
- [x] CHK003 Are loading states required during the ADFS redirection process? [Completeness] (Covered by T030)
|
||||
- [x] CHK004 Is the "Session Expired" user flow defined? [Edge Case, Gap] (Covered by T035)
|
||||
- [x] CHK005 Are requirements defined for the dual-mode login screen layout (Form + ADFS Button)? [Clarity, Spec §FR-013] (Covered by T030)
|
||||
|
||||
## Admin Interface
|
||||
|
||||
- [x] CHK005 Are requirements defined for the User Management list view (columns, sorting)? [Completeness] (Covered by T024)
|
||||
- [x] CHK006 Is the feedback mechanism defined for successful/failed user creation? [Clarity] (Covered by T024)
|
||||
- [x] CHK007 Are confirmation dialogs required for deleting users? [Safety, Gap] (Covered by T040)
|
||||
- [x] CHK008 Is the UI behavior defined when assigning roles (dropdown, search)? [Clarity] (Covered by T024)
|
||||
|
||||
## Navigation & Visibility
|
||||
|
||||
- [x] CHK009 Are requirements defined for hiding menu items the user lacks permission for? [Completeness, Spec §FR-006] (Covered by T025)
|
||||
- [x] CHK010 Is the behavior defined if a user tries to access a restricted URL directly? [Edge Case] (Covered by T042)
|
||||
- [x] CHK011 Are user profile/logout controls required to be visible on all pages? [Consistency] (Covered by T025)
|
||||
|
||||
## Accessibility
|
||||
|
||||
- [x] CHK012 Are keyboard navigation requirements defined for the login form? [Coverage] (Covered by T048)
|
||||
- [x] CHK013 Are error message accessibility requirements (ARIA alerts) specified? [Coverage] (Covered by T048)
|
||||
132
specs/016-multi-user-auth/contracts/api.yaml
Normal file
132
specs/016-multi-user-auth/contracts/api.yaml
Normal file
@@ -0,0 +1,132 @@
|
||||
openapi: 3.0.0
|
||||
info:
|
||||
title: Authentication API
|
||||
version: 1.0.0
|
||||
paths:
|
||||
/api/auth/login:
|
||||
post:
|
||||
summary: Login with username/password
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/x-www-form-urlencoded:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
username:
|
||||
type: string
|
||||
password:
|
||||
type: string
|
||||
responses:
|
||||
'200':
|
||||
description: Successful login
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/Token'
|
||||
'401':
|
||||
description: Invalid credentials
|
||||
|
||||
/api/auth/login/adfs:
|
||||
get:
|
||||
summary: Initiate ADFS login flow
|
||||
responses:
|
||||
'302':
|
||||
description: Redirect to ADFS provider
|
||||
|
||||
/api/auth/callback/adfs:
|
||||
get:
|
||||
summary: ADFS callback handler
|
||||
parameters:
|
||||
- in: query
|
||||
name: code
|
||||
schema:
|
||||
type: string
|
||||
required: true
|
||||
responses:
|
||||
'200':
|
||||
description: Successful login via ADFS
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/Token'
|
||||
|
||||
/api/auth/me:
|
||||
get:
|
||||
summary: Get current user profile
|
||||
security:
|
||||
- bearerAuth: []
|
||||
responses:
|
||||
'200':
|
||||
description: User profile
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/User'
|
||||
|
||||
/api/admin/users:
|
||||
get:
|
||||
summary: List all users
|
||||
security:
|
||||
- bearerAuth: []
|
||||
responses:
|
||||
'200':
|
||||
description: List of users
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: array
|
||||
items:
|
||||
$ref: '#/components/schemas/User'
|
||||
post:
|
||||
summary: Create a new user
|
||||
security:
|
||||
- bearerAuth: []
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/UserCreate'
|
||||
responses:
|
||||
'201':
|
||||
description: User created
|
||||
|
||||
components:
|
||||
securitySchemes:
|
||||
bearerAuth:
|
||||
type: http
|
||||
scheme: bearer
|
||||
bearerFormat: JWT
|
||||
schemas:
|
||||
Token:
|
||||
type: object
|
||||
properties:
|
||||
access_token:
|
||||
type: string
|
||||
token_type:
|
||||
type: string
|
||||
User:
|
||||
type: object
|
||||
properties:
|
||||
id:
|
||||
type: string
|
||||
username:
|
||||
type: string
|
||||
email:
|
||||
type: string
|
||||
roles:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
UserCreate:
|
||||
type: object
|
||||
properties:
|
||||
username:
|
||||
type: string
|
||||
password:
|
||||
type: string
|
||||
roles:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
86
specs/016-multi-user-auth/data-model.md
Normal file
86
specs/016-multi-user-auth/data-model.md
Normal file
@@ -0,0 +1,86 @@
|
||||
# Data Model: Multi-User Authentication
|
||||
|
||||
## Entities
|
||||
|
||||
### User
|
||||
Represents an identity that can authenticate to the system.
|
||||
|
||||
| Field | Type | Description | Constraints |
|
||||
|-------|------|-------------|-------------|
|
||||
| `id` | UUID | Unique identifier | Primary Key |
|
||||
| `username` | String | Unique login name | Unique, Not Null |
|
||||
| `email` | String | User email address | Unique, Optional |
|
||||
| `password_hash` | String | Bcrypt hash of password | Nullable (if ADFS) |
|
||||
| `auth_source` | Enum | Source of identity | `LOCAL` or `ADFS` |
|
||||
| `is_active` | Boolean | Account status | Default `True` |
|
||||
| `created_at` | DateTime | Timestamp of creation | Auto-generated |
|
||||
| `last_login` | DateTime | Timestamp of last login | Nullable |
|
||||
|
||||
### Role
|
||||
Represents a collection of permissions.
|
||||
|
||||
| Field | Type | Description | Constraints |
|
||||
|-------|------|-------------|-------------|
|
||||
| `id` | UUID | Unique identifier | Primary Key |
|
||||
| `name` | String | Human-readable role name | Unique, Not Null |
|
||||
| `description` | String | Description of role purpose | Optional |
|
||||
|
||||
### Permission
|
||||
Represents a specific capability within the system.
|
||||
|
||||
| Field | Type | Description | Constraints |
|
||||
|-------|------|-------------|-------------|
|
||||
| `id` | UUID | Unique identifier | Primary Key |
|
||||
| `resource` | String | Target resource (e.g. `plugin:backup`) | Not Null |
|
||||
| `action` | Enum | Type of access | `READ`, `EXECUTE`, `WRITE` |
|
||||
|
||||
### ADGroupMapping
|
||||
Maps an Active Directory group to a local System Role.
|
||||
|
||||
| Field | Type | Description | Constraints |
|
||||
|-------|------|-------------|-------------|
|
||||
| `id` | UUID | Unique identifier | Primary Key |
|
||||
| `ad_group_name` | String | Name of the group in AD | Unique, Not Null |
|
||||
| `role_id` | UUID | ID of the local role to assign | Foreign Key -> Role.id |
|
||||
|
||||
## Relationships
|
||||
|
||||
- **User <-> Role**: Many-to-Many (via `user_roles` table)
|
||||
- A User can have multiple Roles.
|
||||
- A Role can be assigned to multiple Users.
|
||||
- **Role <-> Permission**: Many-to-Many (via `role_permissions` table)
|
||||
- A Role is defined by a set of Permissions.
|
||||
- A Permission can belong to multiple Roles.
|
||||
|
||||
## Storage Schema (SQLAlchemy)
|
||||
|
||||
```python
|
||||
# Conceptual Schema Definition
|
||||
|
||||
class User(Base):
|
||||
__tablename__ = "users"
|
||||
id = Column(String, primary_key=True, default=generate_uuid)
|
||||
username = Column(String, unique=True, index=True, nullable=False)
|
||||
password_hash = Column(String, nullable=True)
|
||||
auth_source = Column(String, default="local")
|
||||
is_active = Column(Boolean, default=True)
|
||||
roles = relationship("Role", secondary="user_roles", back_populates="users")
|
||||
|
||||
class Role(Base):
|
||||
__tablename__ = "roles"
|
||||
id = Column(String, primary_key=True, default=generate_uuid)
|
||||
name = Column(String, unique=True, nullable=False)
|
||||
permissions = relationship("Permission", secondary="role_permissions")
|
||||
users = relationship("User", secondary="user_roles", back_populates="roles")
|
||||
|
||||
class Permission(Base):
|
||||
__tablename__ = "permissions"
|
||||
id = Column(String, primary_key=True, default=generate_uuid)
|
||||
resource = Column(String, nullable=False) # e.g., "plugin:backup"
|
||||
action = Column(String, nullable=False) # e.g., "execute"
|
||||
|
||||
class ADGroupMapping(Base):
|
||||
__tablename__ = "ad_group_mappings"
|
||||
id = Column(String, primary_key=True, default=generate_uuid)
|
||||
ad_group_name = Column(String, unique=True, nullable=False)
|
||||
role_id = Column(String, ForeignKey("roles.id"), nullable=False)
|
||||
98
specs/016-multi-user-auth/plan.md
Normal file
98
specs/016-multi-user-auth/plan.md
Normal file
@@ -0,0 +1,98 @@
|
||||
# Implementation Plan: Multi-User Authentication and Authorization
|
||||
|
||||
**Branch**: `016-multi-user-auth` | **Date**: 2026-01-26 | **Spec**: [`specs/016-multi-user-auth/spec.md`](spec.md)
|
||||
**Input**: Feature specification from `specs/016-multi-user-auth/spec.md`
|
||||
|
||||
**Note**: This template is filled in by the `/speckit.plan` command. See `.specify/templates/commands/plan.md` for the execution workflow.
|
||||
|
||||
## Summary
|
||||
|
||||
Implement a robust authentication system supporting local users (username/password) and corporate SSO (ADFS via OIDC/OAuth2) simultaneously. The system will enforce Role-Based Access Control (RBAC) to restrict plugin access. Data will be persisted in a dedicated SQLite database (`auth.db`), and sessions will be managed via stateless JWTs. A CLI tool will be provided for initial admin provisioning. The login interface will provide dual options (Form + SSO Button) to ensure administrator access even during ADFS outages.
|
||||
|
||||
## Technical Context
|
||||
|
||||
**Language/Version**: Python 3.9+ (Backend), Node.js 18+ (Frontend)
|
||||
**Primary Dependencies**:
|
||||
- Backend: FastAPI, Authlib (ADFS/OIDC), Passlib[bcrypt] (Password hashing), PyJWT (Token management), SQLAlchemy (ORM for auth.db)
|
||||
- Frontend: SvelteKit (UI), standard fetch API (JWT handling)
|
||||
**Storage**: SQLite (`auth.db`) for Users, Roles, Permissions, and Mappings.
|
||||
**Testing**: pytest (Backend), vitest/playwright (Frontend)
|
||||
**Target Platform**: Linux server (Dockerized environment)
|
||||
**Project Type**: Web Application (FastAPI Backend + SvelteKit Frontend)
|
||||
**Performance Goals**: <100ms auth verification overhead per request.
|
||||
**Constraints**: Must run in existing environment without external DB dependencies (hence SQLite).
|
||||
**Scale/Scope**: ~10-100 concurrent users, ~5-10 distinct roles.
|
||||
|
||||
## Constitution Check
|
||||
|
||||
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
|
||||
|
||||
- [x] **I. Semantic Protocol Compliance**: All new modules will use `[DEF]` anchors and `@RELATION` tags.
|
||||
- [x] **II. Causal Validity**: Contracts (OpenAPI/Pydantic models) will be defined before implementation.
|
||||
- [x] **III. Immutability of Architecture**: No changes to existing core architecture invariants; adding a new `AuthModule` layer.
|
||||
- [x] **IV. Design by Contract**: All auth functions will define `@PRE`/`@POST` conditions.
|
||||
- [x] **V. Belief State Logging**: Auth events will be logged using the standard belief scope logger.
|
||||
- [x] **VI. Fractal Complexity Limit**: Auth logic will be modularized (Service, Repository, API layers).
|
||||
- [x] **VII. Everything is a Plugin**: While core auth is middleware, the *management* of users/roles will be exposed via a System Plugin or dedicated Admin API, respecting the modular design.
|
||||
- [x] **VIII. Unified Frontend Experience**: Login and Admin UI will use standard Svelte components and i18n.
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Documentation (this feature)
|
||||
|
||||
```text
|
||||
specs/[###-feature]/
|
||||
├── plan.md # This file (/speckit.plan command output)
|
||||
├── research.md # Phase 0 output (/speckit.plan command)
|
||||
├── data-model.md # Phase 1 output (/speckit.plan command)
|
||||
├── quickstart.md # Phase 1 output (/speckit.plan command)
|
||||
├── contracts/ # Phase 1 output (/speckit.plan command)
|
||||
└── tasks.md # Phase 2 output (/speckit.tasks command - NOT created by /speckit.plan)
|
||||
```
|
||||
|
||||
### Source Code (repository root)
|
||||
|
||||
```text
|
||||
backend/
|
||||
├── src/
|
||||
│ ├── api/
|
||||
│ │ ├── auth/ # New: Auth endpoints (login, logout, refresh)
|
||||
│ │ ├── admin/ # New: Admin endpoints (users, roles)
|
||||
│ │ └── dependencies.py # Update: Add get_current_user, get_current_active_user
|
||||
│ ├── core/
|
||||
│ │ ├── auth/ # New: Core auth logic
|
||||
│ │ │ ├── jwt.py # Token handling
|
||||
│ │ │ ├── security.py # Password hashing
|
||||
│ │ │ └── config.py # Auth settings
|
||||
│ │ └── database.py # Update: Support for multiple DBs (auth.db)
|
||||
│ ├── models/
|
||||
│ │ └── auth.py # New: SQLAlchemy models (User, Role, Permission)
|
||||
│ ├── schemas/ # New: Pydantic schemas for Auth
|
||||
│ │ └── auth.py
|
||||
│ └── services/
|
||||
│ └── auth_service.py # New: Auth business logic
|
||||
└── tests/
|
||||
└── auth/ # New: Auth tests
|
||||
|
||||
frontend/
|
||||
├── src/
|
||||
│ ├── lib/
|
||||
│ │ ├── auth/ # New: Frontend auth stores/logic
|
||||
│ │ └── api/ # Update: Add auth headers to requests
|
||||
│ ├── routes/
|
||||
│ │ ├── login/ # New: Login page
|
||||
│ │ └── admin/ # New: Admin dashboard (Users/Roles)
|
||||
│ └── components/
|
||||
│ └── auth/ # New: Auth components (ProtectedRoute, Login form)
|
||||
```
|
||||
|
||||
**Structure Decision**: Web application structure with separated backend (FastAPI) and frontend (SvelteKit). Auth logic is centralized in `backend/src/core/auth` and `backend/src/services`, with a new persistent store `auth.db`. Frontend will implement a reactive auth store.
|
||||
|
||||
## Complexity Tracking
|
||||
|
||||
> **Fill ONLY if Constitution Check has violations that must be justified**
|
||||
|
||||
| Violation | Why Needed | Simpler Alternative Rejected Because |
|
||||
|-----------|------------|-------------------------------------|
|
||||
| [e.g., 4th project] | [current need] | [why 3 projects insufficient] |
|
||||
| [e.g., Repository pattern] | [specific problem] | [why direct DB access insufficient] |
|
||||
54
specs/016-multi-user-auth/quickstart.md
Normal file
54
specs/016-multi-user-auth/quickstart.md
Normal file
@@ -0,0 +1,54 @@
|
||||
# Quickstart: Multi-User Auth
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.9+
|
||||
- Node.js 18+
|
||||
- Existing project environment
|
||||
|
||||
## Setup
|
||||
|
||||
1. **Install Dependencies**:
|
||||
```bash
|
||||
pip install "passlib[bcrypt]" "python-jose[cryptography]" "Authlib" "sqlalchemy"
|
||||
```
|
||||
|
||||
2. **Initialize Database**:
|
||||
Run the migration script to create `auth.db` and tables.
|
||||
```bash
|
||||
python backend/src/scripts/init_auth_db.py
|
||||
```
|
||||
|
||||
3. **Create Admin User**:
|
||||
Use the CLI tool to create the initial superuser.
|
||||
```bash
|
||||
python backend/src/scripts/create_admin.py --username admin --password securepassword
|
||||
```
|
||||
|
||||
## Running the Application
|
||||
|
||||
1. **Start Backend**:
|
||||
```bash
|
||||
cd backend
|
||||
uvicorn src.app:app --reload
|
||||
```
|
||||
|
||||
2. **Start Frontend**:
|
||||
```bash
|
||||
cd frontend
|
||||
npm run dev
|
||||
```
|
||||
|
||||
3. **Login**:
|
||||
Navigate to `http://localhost:5173/login` and use the admin credentials created above.
|
||||
|
||||
## Configuring ADFS
|
||||
|
||||
1. Set environment variables in `.env`:
|
||||
```ini
|
||||
ADFS_CLIENT_ID=your-client-id
|
||||
ADFS_CLIENT_SECRET=your-client-secret
|
||||
ADFS_METADATA_URL=https://fs.your-company.com/adfs/.well-known/openid-configuration
|
||||
```
|
||||
|
||||
2. Configure Group Mappings via the Admin UI or API.
|
||||
76
specs/016-multi-user-auth/research.md
Normal file
76
specs/016-multi-user-auth/research.md
Normal file
@@ -0,0 +1,76 @@
|
||||
# Research: Multi-User Authentication and Authorization
|
||||
|
||||
## 1. Authentication Strategy
|
||||
|
||||
### Decision: Hybrid Local + ADFS (OIDC)
|
||||
We will implement a dual authentication strategy:
|
||||
1. **Local Auth**: Username/Password stored in `auth.db` with bcrypt hashing.
|
||||
2. **ADFS**: OpenID Connect (OIDC) integration for enterprise SSO.
|
||||
|
||||
**Rationale**:
|
||||
- **Local Auth**: Ensures the system is usable without external dependencies (ADFS) and provides a fallback for admins.
|
||||
- **ADFS**: Requirement for corporate environment integration. OIDC is the modern standard supported by ADFS 2016+.
|
||||
- **Just-In-Time (JIT)**: ADFS users will be provisioned locally upon first successful login if they belong to a mapped AD group.
|
||||
|
||||
**Alternatives Considered**:
|
||||
- *SAML 2.0*: Older protocol, more complex to implement (XML-based) than OIDC. Rejected in favor of OIDC/OAuth2 support in `Authlib`.
|
||||
- *LDAP Direct Bind*: Requires handling credentials directly, less secure than token-based SSO.
|
||||
|
||||
## 2. Session Management
|
||||
|
||||
### Decision: Stateless JWT (JSON Web Tokens)
|
||||
Sessions will be managed using signed JWTs containing `sub` (user_id), `exp` (expiration), and `scopes` (roles).
|
||||
|
||||
**Rationale**:
|
||||
- **Stateless**: No need to query the DB for every request to validate session validity (signature check is fast).
|
||||
- **Scalable**: Works well with load balancers (though not a primary concern for this scale).
|
||||
- **Frontend Friendly**: Easy to parse in JS to get user info without an extra API call.
|
||||
|
||||
**Security Measures**:
|
||||
- Short-lived Access Tokens (e.g., 15-30 min).
|
||||
- HttpOnly Cookies for storage to prevent XSS theft.
|
||||
- Refresh Token rotation (stored in DB) for long-lived sessions.
|
||||
|
||||
## 3. Authorization Model
|
||||
|
||||
### Decision: RBAC (Role-Based Access Control)
|
||||
Permissions are assigned to Roles. Users are assigned one or more Roles.
|
||||
|
||||
**Structure**:
|
||||
- **Permissions**: Granular capabilities (e.g., `plugin:backup:execute`, `plugin:migration:read`).
|
||||
- **Roles**: Collections of permissions (e.g., `Admin`, `Operator`, `Viewer`).
|
||||
- **Users**: Assigned to Roles.
|
||||
|
||||
**Rationale**:
|
||||
- Standard industry practice.
|
||||
- Simplifies management: Admin assigns a role to a user rather than 50 individual permissions.
|
||||
- AD Group Mapping fits naturally: `AD_Group_X` -> `Role_Y`.
|
||||
|
||||
## 4. Persistence
|
||||
|
||||
### Decision: Dedicated SQLite Database (`auth.db`)
|
||||
A separate SQLite database file for authentication data.
|
||||
|
||||
**Rationale**:
|
||||
- **Separation of Concerns**: Keeps auth data distinct from task history or other app data.
|
||||
- **Relational Integrity**: Enforces foreign keys between Users, Roles, and Permissions better than JSON.
|
||||
- **Concurrency**: SQLite WAL mode handles concurrent reads/writes better than a single JSON config file.
|
||||
|
||||
**Schema Draft**:
|
||||
- `users` (id, username, password_hash, is_active, auth_source)
|
||||
- `roles` (id, name, description)
|
||||
- `permissions` (id, resource, action)
|
||||
- `role_permissions` (role_id, permission_id)
|
||||
- `user_roles` (user_id, role_id)
|
||||
- `ad_group_mappings` (ad_group_name, role_id)
|
||||
|
||||
## 5. Frontend Integration
|
||||
|
||||
### Decision: SvelteKit Stores + HttpOnly Cookies
|
||||
Authentication state will be synchronized between the server (cookies) and client (Svelte store).
|
||||
|
||||
**Mechanism**:
|
||||
- Login endpoint sets `access_token` cookie (HttpOnly).
|
||||
- Client makes API calls; browser automatically sends cookie.
|
||||
- `hooks.server.ts` (or similar middleware) validates token on server-side rendering.
|
||||
- Client-side store (`$auth`) holds user profile (decoded from token or fetched via `/me` endpoint) for UI logic (show/hide buttons).
|
||||
113
specs/016-multi-user-auth/spec.md
Normal file
113
specs/016-multi-user-auth/spec.md
Normal file
@@ -0,0 +1,113 @@
|
||||
# Feature Specification: Multi-User Authentication and Authorization
|
||||
|
||||
**Feature Branch**: `016-multi-user-auth`
|
||||
**Created**: 2026-01-26
|
||||
**Status**: Draft
|
||||
**Input**: User description: "Нужна поддержка многопользовательского логина. Нужно, чтобы пользователи могли логинится по связке логин/пароль, поддержка adfs, разделение прав доступа по плагинам"
|
||||
|
||||
## Clarifications
|
||||
|
||||
### Session 2026-01-26
|
||||
- Q: Permission Model Structure? → A: RBAC (Role-Based Access Control) - Permissions assigned to Roles, Users assigned to Roles.
|
||||
- Q: Initial Admin Provisioning? → A: CLI Command/Script - Explicit script to create the first admin user.
|
||||
- Q: ADFS User Role Assignment? → A: AD Group Mapping - Login requires valid AD group membership; AD groups map to local Roles (e.g., 'superset_admin' -> 'Admin').
|
||||
- Q: Token Management? → A: JWT (JSON Web Tokens) - Stateless, scalable, standard for SPAs.
|
||||
- Q: Persistence Layer? → A: Dedicated SQLite DB (`auth.db`) - Relational storage for Users, Roles, Permissions.
|
||||
- Q: Switching Auth Providers? → A: Dual Support - Both Local and ADFS login options are available simultaneously on the login page.
|
||||
|
||||
## User Scenarios & Testing *(mandatory)*
|
||||
|
||||
### User Story 1 - Local User Authentication (Priority: P1)
|
||||
|
||||
As a user, I want to log in using a username and password so that I can securely access the application.
|
||||
|
||||
**Why this priority**: Basic authentication is the foundation for multi-user support and is required before implementing more complex auth methods or permissions.
|
||||
|
||||
**Independent Test**: Can be fully tested by creating a local user account and successfully logging in/out without any external dependencies.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** a registered user, **When** they enter valid credentials on the login page, **Then** they are redirected to the dashboard and receive a session token.
|
||||
2. **Given** a registered user, **When** they enter invalid credentials, **Then** they see an error message "Invalid username or password".
|
||||
3. **Given** an authenticated user, **When** they click logout, **Then** their session is terminated and they are redirected to the login page.
|
||||
|
||||
---
|
||||
|
||||
### User Story 2 - Plugin-Based Access Control (Priority: P1)
|
||||
|
||||
As an administrator, I want to assign specific plugin access rights to users so that I can control who can use sensitive tools (e.g., Backup, Migration).
|
||||
|
||||
**Why this priority**: Security is a core requirement. Without granular permissions, all authenticated users would have full administrative access, which defeats the purpose of multi-user support.
|
||||
|
||||
**Independent Test**: Create two users with different permissions (e.g., User A has access to "Backup", User B does not). Verify User A can access the Backup tool while User B receives a 403 Forbidden error.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** a user with "Backup" plugin permission, **When** they navigate to the Backup tool, **Then** the page loads successfully.
|
||||
2. **Given** a user WITHOUT "Backup" plugin permission, **When** they navigate to the Backup tool, **Then** they are denied access (UI hides the link, API returns 403).
|
||||
3. **Given** an administrator, **When** they edit a user's permissions, **Then** the changes take effect immediately or upon next login.
|
||||
|
||||
---
|
||||
|
||||
### User Story 3 - ADFS Integration (Priority: P2)
|
||||
|
||||
As a corporate user, I want to log in using my organization's ADFS credentials so that I don't have to manage a separate password.
|
||||
|
||||
**Why this priority**: Essential for enterprise environments but dependent on the core authentication infrastructure being in place (Story 1).
|
||||
|
||||
**Independent Test**: Configure the application with a test ADFS provider (or mock). Verify a user can initiate the SSO flow and be logged in automatically.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** a configured ADFS provider, **When** a user clicks "Login with ADFS", **Then** they are redirected to the identity provider.
|
||||
2. **Given** a successful ADFS authentication, **When** the user returns to the app, **Then** a local user session is created/matched and they are logged in.
|
||||
3. **Given** a new ADFS user, **When** they log in for the first time, **Then** a local user record is automatically created (JIT provisioning) with default permissions.
|
||||
|
||||
---
|
||||
|
||||
### Edge Cases
|
||||
|
||||
- What happens when an ADFS user's account is disabled in the local system? (Should block login even if ADFS succeeds)
|
||||
- How does the system handle concurrent sessions? (Allow or restrict?)
|
||||
- What happens if a plugin is removed but users still have permission for it? (Graceful handling/cleanup)
|
||||
- What happens if the ADFS server is unreachable? (Fallback to local login if applicable, or clear error message)
|
||||
|
||||
## Requirements *(mandatory)*
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
- **FR-001**: System MUST support local user authentication via username and password.
|
||||
- **FR-002**: System MUST support authentication via ADFS (Active Directory Federation Services) using standard federation protocols.
|
||||
- **FR-003**: System MUST provide a mechanism to manage users (Create, Read, Update, Delete) - restricted to administrators.
|
||||
- **FR-004**: System MUST implement Role-Based Access Control (RBAC) where permissions are assigned to Roles, and Roles are assigned to Users.
|
||||
- **FR-005**: System MUST enforce permissions at the server level for all plugin execution requests.
|
||||
- **FR-006**: System MUST enforce permissions at the user interface level (hide navigation items/buttons for unauthorized plugins).
|
||||
- **FR-007**: System MUST securely store local user credentials.
|
||||
- **FR-008**: System MUST support Just-In-Time (JIT) provisioning for ADFS users ONLY if they belong to a mapped AD group.
|
||||
- **FR-009**: System MUST provide a CLI utility to create an initial administrator account to prevent lockout during first deployment.
|
||||
- **FR-010**: System MUST allow configuring mappings between Active Directory Groups and local System Roles.
|
||||
- **FR-011**: System MUST use JWT (JSON Web Tokens) for API session management.
|
||||
- **FR-012**: System MUST persist authentication and authorization data in a dedicated SQLite database (`auth.db`).
|
||||
- **FR-013**: System MUST provide a unified login interface supporting both Local (Username/Password) and ADFS (SSO Button) authentication methods simultaneously.
|
||||
|
||||
### Key Entities
|
||||
|
||||
- **User**: Represents a system user. Attributes: ID, Username, Email, PasswordHash, AuthSource (Local/ADFS), IsActive, Roles (List[RoleID]).
|
||||
- **Role**: Named collection of permissions. Attributes: ID, Name, Description, Permissions (List[Permission]).
|
||||
- **Permission**: Represents access capability. Attributes: ResourceID (e.g., Plugin ID), Action (Execute, Read).
|
||||
- **ADGroupMapping**: Configuration mapping AD Group names to Role IDs.
|
||||
|
||||
## Success Criteria *(mandatory)*
|
||||
|
||||
### Measurable Outcomes
|
||||
|
||||
- **SC-001**: Administrators can successfully create a new local user and assign specific plugin permissions in under 2 minutes.
|
||||
- **SC-002**: Users without permission for a specific plugin are denied access 100% of the time when attempting to use its functions.
|
||||
- **SC-003**: ADFS login flow completes successfully for valid credentials and maps to the correct local user identity.
|
||||
- **SC-004**: User interface dynamically updates to show only permitted tools for the logged-in user.
|
||||
|
||||
## Assumptions
|
||||
|
||||
- The application currently has a simple or placeholder authentication mechanism.
|
||||
- "Plugin access" refers to the ability to use the plugin's functionality and view its interface.
|
||||
- A default administrator account will be available upon initial system setup to prevent lockout.
|
||||
91
specs/016-multi-user-auth/tasks.md
Normal file
91
specs/016-multi-user-auth/tasks.md
Normal file
@@ -0,0 +1,91 @@
|
||||
# Tasks: Multi-User Authentication and Authorization
|
||||
|
||||
**Feature Branch**: `016-multi-user-auth`
|
||||
**Feature Spec**: [`specs/016-multi-user-auth/spec.md`](spec.md)
|
||||
**Implementation Plan**: [`specs/016-multi-user-auth/plan.md`](plan.md)
|
||||
|
||||
## Phase 1: Setup & Infrastructure (Blocking)
|
||||
|
||||
*Goal: Initialize the auth database, core dependencies, and backend infrastructure.*
|
||||
|
||||
- [ ] T001 Install backend dependencies (Authlib, Passlib, PyJWT, SQLAlchemy) in `backend/requirements.txt`
|
||||
- [ ] T002 Implement core configuration for Auth and Database in `backend/src/core/auth/config.py`
|
||||
- [ ] T003 Implement database connection logic for `auth.db` in `backend/src/core/database.py`
|
||||
- [ ] T004 Create SQLAlchemy models for User, Role, Permission in `backend/src/models/auth.py`
|
||||
- [ ] T005 Create migration/init script to generate `auth.db` schema in `backend/src/scripts/init_auth_db.py`
|
||||
- [ ] T006 Implement password hashing utility using Passlib in `backend/src/core/auth/security.py`
|
||||
- [ ] T007 Implement JWT token generation and validation logic in `backend/src/core/auth/jwt.py`
|
||||
- [ ] T008 [P] Implement CLI tool for creating the initial admin user in `backend/src/scripts/create_admin.py`
|
||||
|
||||
## Phase 2: User Story 1 - Local User Authentication (Priority: P1)
|
||||
|
||||
*Goal: Enable users to log in with username/password and receive a JWT session.*
|
||||
|
||||
- [ ] T009 [US1] Create Pydantic schemas for User, UserCreate, Token in `backend/src/schemas/auth.py`
|
||||
- [ ] T010 [US1] Implement `AuthRepository` for DB operations in `backend/src/core/auth/repository.py`
|
||||
- [ ] T011 [US1] Implement `AuthService` for login logic (verify password, create token) in `backend/src/services/auth_service.py`
|
||||
- [ ] T012 [US1] Create API endpoint `POST /api/auth/login` in `backend/src/api/auth.py`
|
||||
- [ ] T013 [US1] Implement `get_current_user` dependency for JWT verification in `backend/src/dependencies.py`
|
||||
- [ ] T014 [US1] Create API endpoint `GET /api/auth/me` to retrieve current user profile in `backend/src/api/auth.py`
|
||||
- [ ] T043 [US1] Implement session revocation (Logout) endpoint in `backend/src/api/auth.py`
|
||||
- [ ] T044 [US1] Implement account status check (`is_active`) in authentication flow in `backend/src/services/auth_service.py`
|
||||
- [ ] T015 [US1] Implement frontend auth store (Svelte store) in `frontend/src/lib/auth/store.ts`
|
||||
- [ ] T016 [US1] Implement Login Page UI using `src/lib/ui` and `src/lib/i18n` in `frontend/src/routes/login/+page.svelte`
|
||||
- [ ] T017 [US1] Integrate Login Page with Backend API in `frontend/src/routes/login/+page.svelte`
|
||||
- [ ] T018 [US1] Implement `ProtectedRoute` component to redirect unauthenticated users in `frontend/src/components/auth/ProtectedRoute.svelte`
|
||||
- [ ] T037 [US1] Implement password complexity validation logic in `backend/src/core/auth/security.py`
|
||||
|
||||
## Phase 3: User Story 2 - Plugin-Based Access Control (Priority: P1)
|
||||
|
||||
*Goal: Restrict access to plugins based on user roles and permissions.*
|
||||
|
||||
- [ ] T019 [US2] Update `PluginBase` to include required permission strings in `backend/src/core/plugin_base.py`
|
||||
- [ ] T020 [US2] Implement `has_permission` dependency for route protection in `backend/src/dependencies.py`
|
||||
- [ ] T021 [US2] Protect existing plugin API routes using `has_permission` in `backend/src/api/routes/*.py`
|
||||
- [ ] T022 [US2] Implement `SystemAdminPlugin` inheriting from `PluginBase` for User/Role management in `backend/src/plugins/system_admin.py`
|
||||
- [ ] T023 [US2] Implement Admin API endpoints within `SystemAdminPlugin` (with pagination) in `backend/src/api/routes/admin.py`
|
||||
- [ ] T024 [US2] Create Admin Dashboard UI using `src/lib/ui` and `src/lib/i18n` in `frontend/src/routes/admin/users/+page.svelte`
|
||||
- [ ] T025 [US2] Update Navigation Bar to hide links and show user profile/logout using `src/lib/ui` in `frontend/src/components/Navbar.svelte`
|
||||
- [ ] T042 [US2] Implement `PermissionGuard` frontend component for granular UI element protection in `frontend/src/components/auth/PermissionGuard.svelte`
|
||||
- [ ] T045 [US2] Implement multi-role permission resolution logic (union of permissions) in `backend/src/services/auth_service.py`
|
||||
|
||||
## Phase 4: User Story 3 - ADFS Integration (Priority: P2)
|
||||
|
||||
*Goal: Enable corporate SSO login via ADFS and JIT provisioning.*
|
||||
|
||||
- [ ] T026 [US3] Configure Authlib for ADFS OIDC in `backend/src/core/auth/oauth.py`
|
||||
- [ ] T027 [US3] Create `ADGroupMapping` model in `backend/src/models/auth.py` and update DB init script
|
||||
- [ ] T028 [US3] Implement JIT provisioning logic (create user if maps to group) in `backend/src/services/auth_service.py`
|
||||
- [ ] T029 [US3] Create API endpoints `GET /api/auth/login/adfs` and `GET /api/auth/callback/adfs` in `backend/src/api/auth.py`
|
||||
- [ ] T030 [US3] Update Login Page to include "Login with ADFS" button using `src/lib/ui` in `frontend/src/routes/login/+page.svelte`
|
||||
- [ ] T031 [US3] Implement Admin UI for configuring AD Group Mappings in `frontend/src/routes/admin/settings/+page.svelte`
|
||||
- [ ] T041 [US3] Create ADFS mock provider for local testing and CI in `backend/tests/auth/mock_adfs.py`
|
||||
- [ ] T046 [US3] Implement token refresh logic for ADFS OIDC tokens in `backend/src/core/auth/jwt.py`
|
||||
|
||||
## Phase 5: Polish & Security Hardening
|
||||
|
||||
*Goal: Ensure security best practices and smooth UX.*
|
||||
|
||||
- [ ] T032 Ensure all cookies are set with `HttpOnly` and `Secure` flags in `backend/src/api/auth.py`
|
||||
- [ ] T033 Implement rate limiting and account lockout policy in `backend/src/api/auth.py`
|
||||
- [ ] T034 Verify error messages are generic (no username enumeration) across all auth endpoints
|
||||
- [ ] T035 Add "Session Expired" handling in frontend interceptor in `frontend/src/lib/api/client.ts`
|
||||
- [ ] T036 Final manual test of switching between Local and ADFS login flows
|
||||
- [ ] T040 Add confirmation dialogs for destructive admin actions using `src/lib/ui` in `frontend/src/routes/admin/users/+page.svelte`
|
||||
- [ ] T047 Implement audit logging for security events (login, logout, permission changes) in `backend/src/core/auth/logger.py`
|
||||
- [ ] T048 Perform UI accessibility audit (keyboard nav, ARIA alerts) for all auth components
|
||||
- [ ] T049 Implement unit and integration tests for Local Auth and RBAC in `backend/tests/auth/`
|
||||
- [ ] T050 Implement E2E tests for ADFS flow using mock provider in `tests/e2e/auth.spec.ts`
|
||||
|
||||
## Dependencies
|
||||
|
||||
1. **Phase 1** must be completed before any User Stories.
|
||||
2. **Phase 2 (Local Auth)** is the foundation for authentication and session management.
|
||||
3. **Phase 3 (RBAC)** depends on Phase 2 (needs authenticated users to check permissions).
|
||||
4. **Phase 4 (ADFS)** depends on Phase 2 (uses same session mechanism) and Phase 3 (needs roles for JIT).
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
- **MVP**: Complete Phases 1 and 2. This gives a working auth system with local users.
|
||||
- **Increment 1**: Complete Phase 3. This adds the critical security controls (RBAC).
|
||||
- **Increment 2**: Complete Phase 4. This adds corporate SSO convenience.
|
||||
1664
specs/project_map.md
1664
specs/project_map.md
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user