feat(027): Final Phase T038-T043 implementation

- T038: SessionEvent logger and persistence logic
  - Added SessionEventLogger service with explicit audit event persistence
  - Added SessionEvent model with events relationship on DatasetReviewSession
  - Integrated event logging into orchestrator flows and API mutation endpoints

- T039: Semantic source version propagation
  - Added source_version column to SemanticFieldEntry
  - Added propagate_source_version_update() to SemanticResolver
  - Preserves locked/manual field invariants during propagation

- T040: Batch approval API and UI actions
  - Added batch semantic approval endpoint (/fields/semantic/approve-batch)
  - Added batch mapping approval endpoint (/mappings/approve-batch)
  - Added batch approval actions to SemanticLayerReview and ExecutionMappingReview components
  - Aligned batch semantics with single-item approval contracts

- T041: Superset compatibility matrix tests
  - Added test_superset_matrix.py with preview and SQL Lab fallback coverage
  - Tests verify client method preference and matrix fallback behavior

- T042: RBAC audit sweep on session-mutation endpoints
  - Added _require_owner_mutation_scope() helper
  - Applied owner guards to update_session, delete_session, and all mutation endpoints
  - Ensured no bypass of existing permission checks

- T043: i18n coverage for dataset-review UI
  - Added workspace state labels (empty/importing/review) to en.json and ru.json
  - Added batch action labels for semantics and mappings
  - Fixed workspace state comparison to lowercase strings
  - Removed hardcoded workspace state display strings

Signed-off-by: Implementation Specialist <impl@ss-tools>
This commit is contained in:
2026-03-17 14:29:33 +03:00
parent 38bda6a714
commit ed3d5f3039
33 changed files with 99234 additions and 93415 deletions

25
.gitignore vendored
View File

@@ -59,21 +59,22 @@ keyring passwords.py
*github* *github*
*tech_spec* *tech_spec*
/dashboards /dashboards
dashboards_example/**/dashboards/ dashboards_example/**/dashboards/
backend/mappings.db backend/mappings.db
backend/tasks.db backend/tasks.db
backend/logs backend/logs
backend/auth.db backend/auth.db
semantics/reports semantics/reports
backend/tasks.db backend/tasks.db
backend/**/*.db backend/**/*.db
backend/**/*.sqlite backend/**/*.sqlite
# Universal / tooling # Universal / tooling
node_modules/ node_modules/
.venv/ .venv/
coverage/ coverage/
*.tmp *.tmp
logs/app.log.1

View File

@@ -1 +1 @@
{"mcpServers":{"axiom-core":{"command":"/home/busya/dev/ast-mcp-core-server/.venv/bin/python","args":["-c","from src.server import main; main()"],"env":{"PYTHONPATH":"/home/busya/dev/ast-mcp-core-server"},"alwaysAllow":["read_grace_outline_tool","ast_search_tool","get_semantic_context_tool","build_task_context_tool","audit_contracts_tool","diff_contract_semantics_tool","simulate_patch_tool","patch_contract_tool","rename_contract_id_tool","move_contract_tool","extract_contract_tool","infer_missing_relations_tool","map_runtime_trace_to_contracts_tool","scaffold_contract_tests_tool","search_contracts_tool","reindex_workspace_tool","prune_contract_metadata_tool","workspace_semantic_health_tool","trace_tests_for_contract_tool"]}}} {"mcpServers":{"axiom-core":{"command":"/home/busya/dev/ast-mcp-core-server/.venv/bin/python","args":["-c","from src.server import main; main()"],"env":{"PYTHONPATH":"/home/busya/dev/ast-mcp-core-server"},"alwaysAllow":["read_grace_outline_tool","ast_search_tool","get_semantic_context_tool","build_task_context_tool","audit_contracts_tool","diff_contract_semantics_tool","simulate_patch_tool","patch_contract_tool","rename_contract_id_tool","move_contract_tool","extract_contract_tool","infer_missing_relations_tool","map_runtime_trace_to_contracts_tool","scaffold_contract_tests_tool","search_contracts_tool","reindex_workspace_tool","prune_contract_metadata_tool","workspace_semantic_health_tool","trace_tests_for_contract_tool","guarded_patch_contract_tool","impact_analysis_tool"]}}}

File diff suppressed because it is too large Load Diff

View File

@@ -1,7 +1,8 @@
# [DEF:test_clean_release_v2_api:Module] # [DEF:CleanReleaseV2ApiTests:Module]
# @COMPLEXITY: 3 # @COMPLEXITY: 3
# @PURPOSE: API contract tests for redesigned clean release endpoints. # @PURPOSE: API contract tests for redesigned clean release endpoints.
# @LAYER: Domain # @LAYER: Domain
# @RELATION: DEPENDS_ON -> backend.src.api.routes.clean_release_v2
from datetime import datetime, timezone from datetime import datetime, timezone
from types import SimpleNamespace from types import SimpleNamespace
@@ -90,4 +91,4 @@ def test_manifest_build_contract():
assert "manifest_digest" in data assert "manifest_digest" in data
assert data["candidate_id"] == candidate_id assert data["candidate_id"] == candidate_id
# [/DEF:test_clean_release_v2_api:Module] # [/DEF:CleanReleaseV2ApiTests:Module]

View File

@@ -1,8 +1,8 @@
# [DEF:test_clean_release_v2_release_api:Module] # [DEF:CleanReleaseV2ReleaseApiTests:Module]
# @COMPLEXITY: 3 # @COMPLEXITY: 3
# @PURPOSE: API contract test scaffolding for clean release approval and publication endpoints. # @PURPOSE: API contract test scaffolding for clean release approval and publication endpoints.
# @LAYER: Domain # @LAYER: Domain
# @RELATION: IMPLEMENTS -> clean_release_v2_release_api_contracts # @RELATION: DEPENDS_ON -> backend.src.api.routes.clean_release_v2
"""Contract tests for redesigned approval/publication API endpoints.""" """Contract tests for redesigned approval/publication API endpoints."""
@@ -104,4 +104,4 @@ def test_release_reject_contract() -> None:
assert payload["decision"] == "REJECTED" assert payload["decision"] == "REJECTED"
# [/DEF:test_clean_release_v2_release_api:Module] # [/DEF:CleanReleaseV2ReleaseApiTests:Module]

View File

@@ -1,8 +1,8 @@
# [DEF:backend.src.api.routes.__tests__.test_connections_routes:Module] # [DEF:ConnectionsRoutesTests:Module]
# @COMPLEXITY: 3 # @COMPLEXITY: 3
# @PURPOSE: Verifies connection routes bootstrap their table before CRUD access. # @PURPOSE: Verifies connection routes bootstrap their table before CRUD access.
# @LAYER: API # @LAYER: API
# @RELATION: VERIFIES -> backend.src.api.routes.connections # @RELATION: DEPENDS_ON -> ConnectionsRouter
import os import os
import sys import sys
@@ -69,4 +69,4 @@ def test_create_connection_bootstraps_missing_table(db_session):
assert created.host == "warehouse.internal" assert created.host == "warehouse.internal"
assert "connection_configs" in inspector.get_table_names() assert "connection_configs" in inspector.get_table_names()
# [/DEF:backend.src.api.routes.__tests__.test_connections_routes:Module] # [/DEF:ConnectionsRoutesTests:Module]

View File

@@ -1,8 +1,8 @@
# [DEF:backend.src.api.routes.__tests__.test_dashboards:Module] # [DEF:DashboardsApiTests:Module]
# @COMPLEXITY: 3 # @COMPLEXITY: 3
# @PURPOSE: Unit tests for Dashboards API endpoints # @PURPOSE: Unit tests for dashboards API endpoints.
# @LAYER: API # @LAYER: API
# @RELATION: TESTS -> backend.src.api.routes.dashboards # @RELATION: DEPENDS_ON -> backend.src.api.routes.dashboards
import pytest import pytest
from unittest.mock import MagicMock, patch, AsyncMock from unittest.mock import MagicMock, patch, AsyncMock
@@ -57,6 +57,7 @@ client = TestClient(app)
# [DEF:test_get_dashboards_success:Function] # [DEF:test_get_dashboards_success:Function]
# @PURPOSE: Validate dashboards listing returns a populated response that satisfies the schema contract.
# @TEST: GET /api/dashboards returns 200 and valid schema # @TEST: GET /api/dashboards returns 200 and valid schema
# @PRE: env_id exists # @PRE: env_id exists
# @POST: Response matches DashboardsResponse schema # @POST: Response matches DashboardsResponse schema
@@ -95,6 +96,7 @@ def test_get_dashboards_success(mock_deps):
# [DEF:test_get_dashboards_with_search:Function] # [DEF:test_get_dashboards_with_search:Function]
# @PURPOSE: Validate dashboards listing applies the search filter and returns only matching rows.
# @TEST: GET /api/dashboards filters by search term # @TEST: GET /api/dashboards filters by search term
# @PRE: search parameter provided # @PRE: search parameter provided
# @POST: Only matching dashboards returned # @POST: Only matching dashboards returned
@@ -126,6 +128,7 @@ def test_get_dashboards_with_search(mock_deps):
# [DEF:test_get_dashboards_empty:Function] # [DEF:test_get_dashboards_empty:Function]
# @PURPOSE: Validate dashboards listing returns an empty payload for an environment without dashboards.
# @TEST_EDGE: empty_dashboards -> {env_id: 'empty_env', expected_total: 0} # @TEST_EDGE: empty_dashboards -> {env_id: 'empty_env', expected_total: 0}
def test_get_dashboards_empty(mock_deps): def test_get_dashboards_empty(mock_deps):
"""@TEST_EDGE: empty_dashboards -> {env_id: 'empty_env', expected_total: 0}""" """@TEST_EDGE: empty_dashboards -> {env_id: 'empty_env', expected_total: 0}"""
@@ -146,6 +149,7 @@ def test_get_dashboards_empty(mock_deps):
# [DEF:test_get_dashboards_superset_failure:Function] # [DEF:test_get_dashboards_superset_failure:Function]
# @PURPOSE: Validate dashboards listing surfaces a 503 contract when Superset access fails.
# @TEST_EDGE: external_superset_failure -> {env_id: 'bad_conn', status: 503} # @TEST_EDGE: external_superset_failure -> {env_id: 'bad_conn', status: 503}
def test_get_dashboards_superset_failure(mock_deps): def test_get_dashboards_superset_failure(mock_deps):
"""@TEST_EDGE: external_superset_failure -> {env_id: 'bad_conn', status: 503}""" """@TEST_EDGE: external_superset_failure -> {env_id: 'bad_conn', status: 503}"""
@@ -164,6 +168,7 @@ def test_get_dashboards_superset_failure(mock_deps):
# [DEF:test_get_dashboards_env_not_found:Function] # [DEF:test_get_dashboards_env_not_found:Function]
# @PURPOSE: Validate dashboards listing returns 404 when the requested environment does not exist.
# @TEST: GET /api/dashboards returns 404 if env_id missing # @TEST: GET /api/dashboards returns 404 if env_id missing
# @PRE: env_id does not exist # @PRE: env_id does not exist
# @POST: Returns 404 error # @POST: Returns 404 error
@@ -179,6 +184,7 @@ def test_get_dashboards_env_not_found(mock_deps):
# [DEF:test_get_dashboards_invalid_pagination:Function] # [DEF:test_get_dashboards_invalid_pagination:Function]
# @PURPOSE: Validate dashboards listing rejects invalid pagination parameters with 400 responses.
# @TEST: GET /api/dashboards returns 400 for invalid page/page_size # @TEST: GET /api/dashboards returns 400 for invalid page/page_size
# @PRE: page < 1 or page_size > 100 # @PRE: page < 1 or page_size > 100
# @POST: Returns 400 error # @POST: Returns 400 error
@@ -199,6 +205,7 @@ def test_get_dashboards_invalid_pagination(mock_deps):
# [DEF:test_get_dashboard_detail_success:Function] # [DEF:test_get_dashboard_detail_success:Function]
# @PURPOSE: Validate dashboard detail returns charts and datasets for an existing dashboard.
# @TEST: GET /api/dashboards/{id} returns dashboard detail with charts and datasets # @TEST: GET /api/dashboards/{id} returns dashboard detail with charts and datasets
def test_get_dashboard_detail_success(mock_deps): def test_get_dashboard_detail_success(mock_deps):
with patch("src.api.routes.dashboards.SupersetClient") as mock_client_cls: with patch("src.api.routes.dashboards.SupersetClient") as mock_client_cls:
@@ -251,6 +258,7 @@ def test_get_dashboard_detail_success(mock_deps):
# [DEF:test_get_dashboard_detail_env_not_found:Function] # [DEF:test_get_dashboard_detail_env_not_found:Function]
# @PURPOSE: Validate dashboard detail returns 404 when the requested environment is missing.
# @TEST: GET /api/dashboards/{id} returns 404 for missing environment # @TEST: GET /api/dashboards/{id} returns 404 for missing environment
def test_get_dashboard_detail_env_not_found(mock_deps): def test_get_dashboard_detail_env_not_found(mock_deps):
mock_deps["config"].get_environments.return_value = [] mock_deps["config"].get_environments.return_value = []
@@ -265,6 +273,7 @@ def test_get_dashboard_detail_env_not_found(mock_deps):
# [DEF:test_migrate_dashboards_success:Function] # [DEF:test_migrate_dashboards_success:Function]
# @TEST: POST /api/dashboards/migrate creates migration task # @TEST: POST /api/dashboards/migrate creates migration task
# @PRE: Valid source_env_id, target_env_id, dashboard_ids # @PRE: Valid source_env_id, target_env_id, dashboard_ids
# @PURPOSE: Validate dashboard migration request creates an async task and returns its identifier.
# @POST: Returns task_id and create_task was called # @POST: Returns task_id and create_task was called
def test_migrate_dashboards_success(mock_deps): def test_migrate_dashboards_success(mock_deps):
mock_source = MagicMock() mock_source = MagicMock()
@@ -300,6 +309,7 @@ def test_migrate_dashboards_success(mock_deps):
# [DEF:test_migrate_dashboards_no_ids:Function] # [DEF:test_migrate_dashboards_no_ids:Function]
# @TEST: POST /api/dashboards/migrate returns 400 for empty dashboard_ids # @TEST: POST /api/dashboards/migrate returns 400 for empty dashboard_ids
# @PRE: dashboard_ids is empty # @PRE: dashboard_ids is empty
# @PURPOSE: Validate dashboard migration rejects empty dashboard identifier lists.
# @POST: Returns 400 error # @POST: Returns 400 error
def test_migrate_dashboards_no_ids(mock_deps): def test_migrate_dashboards_no_ids(mock_deps):
response = client.post( response = client.post(
@@ -319,6 +329,7 @@ def test_migrate_dashboards_no_ids(mock_deps):
# [DEF:test_migrate_dashboards_env_not_found:Function] # [DEF:test_migrate_dashboards_env_not_found:Function]
# @PURPOSE: Validate migration creation returns 404 when the source environment cannot be resolved.
# @PRE: source_env_id and target_env_id are valid environment IDs # @PRE: source_env_id and target_env_id are valid environment IDs
def test_migrate_dashboards_env_not_found(mock_deps): def test_migrate_dashboards_env_not_found(mock_deps):
"""@PRE: source_env_id and target_env_id are valid environment IDs.""" """@PRE: source_env_id and target_env_id are valid environment IDs."""
@@ -339,6 +350,7 @@ def test_migrate_dashboards_env_not_found(mock_deps):
# [DEF:test_backup_dashboards_success:Function] # [DEF:test_backup_dashboards_success:Function]
# @TEST: POST /api/dashboards/backup creates backup task # @TEST: POST /api/dashboards/backup creates backup task
# @PRE: Valid env_id, dashboard_ids # @PRE: Valid env_id, dashboard_ids
# @PURPOSE: Validate dashboard backup request creates an async backup task and returns its identifier.
# @POST: Returns task_id and create_task was called # @POST: Returns task_id and create_task was called
def test_backup_dashboards_success(mock_deps): def test_backup_dashboards_success(mock_deps):
mock_env = MagicMock() mock_env = MagicMock()
@@ -369,6 +381,7 @@ def test_backup_dashboards_success(mock_deps):
# [DEF:test_backup_dashboards_env_not_found:Function] # [DEF:test_backup_dashboards_env_not_found:Function]
# @PURPOSE: Validate backup task creation returns 404 when the target environment is missing.
# @PRE: env_id is a valid environment ID # @PRE: env_id is a valid environment ID
def test_backup_dashboards_env_not_found(mock_deps): def test_backup_dashboards_env_not_found(mock_deps):
"""@PRE: env_id is a valid environment ID.""" """@PRE: env_id is a valid environment ID."""
@@ -388,6 +401,7 @@ def test_backup_dashboards_env_not_found(mock_deps):
# [DEF:test_get_database_mappings_success:Function] # [DEF:test_get_database_mappings_success:Function]
# @TEST: GET /api/dashboards/db-mappings returns mapping suggestions # @TEST: GET /api/dashboards/db-mappings returns mapping suggestions
# @PRE: Valid source_env_id, target_env_id # @PRE: Valid source_env_id, target_env_id
# @PURPOSE: Validate database mapping suggestions are returned for valid source and target environments.
# @POST: Returns list of database mappings # @POST: Returns list of database mappings
def test_get_database_mappings_success(mock_deps): def test_get_database_mappings_success(mock_deps):
mock_source = MagicMock() mock_source = MagicMock()
@@ -419,6 +433,7 @@ def test_get_database_mappings_success(mock_deps):
# [DEF:test_get_database_mappings_env_not_found:Function] # [DEF:test_get_database_mappings_env_not_found:Function]
# @PURPOSE: Validate database mapping suggestions return 404 when either environment is missing.
# @PRE: source_env_id and target_env_id are valid environment IDs # @PRE: source_env_id and target_env_id are valid environment IDs
def test_get_database_mappings_env_not_found(mock_deps): def test_get_database_mappings_env_not_found(mock_deps):
"""@PRE: source_env_id must be a valid environment.""" """@PRE: source_env_id must be a valid environment."""
@@ -429,6 +444,7 @@ def test_get_database_mappings_env_not_found(mock_deps):
# [DEF:test_get_dashboard_tasks_history_filters_success:Function] # [DEF:test_get_dashboard_tasks_history_filters_success:Function]
# @PURPOSE: Validate dashboard task history returns only related backup and LLM tasks.
# @TEST: GET /api/dashboards/{id}/tasks returns backup and llm tasks for dashboard # @TEST: GET /api/dashboards/{id}/tasks returns backup and llm tasks for dashboard
def test_get_dashboard_tasks_history_filters_success(mock_deps): def test_get_dashboard_tasks_history_filters_success(mock_deps):
now = datetime.now(timezone.utc) now = datetime.now(timezone.utc)
@@ -473,6 +489,7 @@ def test_get_dashboard_tasks_history_filters_success(mock_deps):
# [DEF:test_get_dashboard_thumbnail_success:Function] # [DEF:test_get_dashboard_thumbnail_success:Function]
# @PURPOSE: Validate dashboard thumbnail endpoint proxies image bytes and content type from Superset.
# @TEST: GET /api/dashboards/{id}/thumbnail proxies image bytes from Superset # @TEST: GET /api/dashboards/{id}/thumbnail proxies image bytes from Superset
def test_get_dashboard_thumbnail_success(mock_deps): def test_get_dashboard_thumbnail_success(mock_deps):
with patch("src.api.routes.dashboards.SupersetClient") as mock_client_cls: with patch("src.api.routes.dashboards.SupersetClient") as mock_client_cls:
@@ -540,6 +557,7 @@ def _matches_actor_case_insensitive(bound_username, owners, modified_by):
# [DEF:test_get_dashboards_profile_filter_contract_owners_or_modified_by:Function] # [DEF:test_get_dashboards_profile_filter_contract_owners_or_modified_by:Function]
# @TEST: GET /api/dashboards applies profile-default filter with owners OR modified_by trim+case-insensitive semantics. # @TEST: GET /api/dashboards applies profile-default filter with owners OR modified_by trim+case-insensitive semantics.
# @PURPOSE: Validate profile-default filtering matches owner and modifier aliases using normalized Superset actor values.
# @PRE: Current user has enabled profile-default preference and bound username. # @PRE: Current user has enabled profile-default preference and bound username.
# @POST: Response includes only matching dashboards and effective_profile_filter metadata. # @POST: Response includes only matching dashboards and effective_profile_filter metadata.
def test_get_dashboards_profile_filter_contract_owners_or_modified_by(mock_deps): def test_get_dashboards_profile_filter_contract_owners_or_modified_by(mock_deps):
@@ -599,6 +617,7 @@ def test_get_dashboards_profile_filter_contract_owners_or_modified_by(mock_deps)
# [DEF:test_get_dashboards_override_show_all_contract:Function] # [DEF:test_get_dashboards_override_show_all_contract:Function]
# @TEST: GET /api/dashboards honors override_show_all and disables profile-default filter for current page. # @TEST: GET /api/dashboards honors override_show_all and disables profile-default filter for current page.
# @PURPOSE: Validate override_show_all bypasses profile-default filtering without changing dashboard list semantics.
# @PRE: Profile-default preference exists but override_show_all=true query is provided. # @PRE: Profile-default preference exists but override_show_all=true query is provided.
# @POST: Response remains unfiltered and effective_profile_filter.applied is false. # @POST: Response remains unfiltered and effective_profile_filter.applied is false.
def test_get_dashboards_override_show_all_contract(mock_deps): def test_get_dashboards_override_show_all_contract(mock_deps):
@@ -640,6 +659,7 @@ def test_get_dashboards_override_show_all_contract(mock_deps):
# [DEF:test_get_dashboards_profile_filter_no_match_results_contract:Function] # [DEF:test_get_dashboards_profile_filter_no_match_results_contract:Function]
# @TEST: GET /api/dashboards returns empty result set when profile-default filter is active and no dashboard actors match. # @TEST: GET /api/dashboards returns empty result set when profile-default filter is active and no dashboard actors match.
# @PURPOSE: Validate profile-default filtering returns an empty dashboard page when no actor aliases match the bound user.
# @PRE: Profile-default preference is enabled with bound username and all dashboards are non-matching. # @PRE: Profile-default preference is enabled with bound username and all dashboards are non-matching.
# @POST: Response total is 0 with deterministic pagination and active effective_profile_filter metadata. # @POST: Response total is 0 with deterministic pagination and active effective_profile_filter metadata.
def test_get_dashboards_profile_filter_no_match_results_contract(mock_deps): def test_get_dashboards_profile_filter_no_match_results_contract(mock_deps):
@@ -695,6 +715,7 @@ def test_get_dashboards_profile_filter_no_match_results_contract(mock_deps):
# [DEF:test_get_dashboards_page_context_other_disables_profile_default:Function] # [DEF:test_get_dashboards_page_context_other_disables_profile_default:Function]
# @TEST: GET /api/dashboards does not auto-apply profile-default filter outside dashboards_main page context. # @TEST: GET /api/dashboards does not auto-apply profile-default filter outside dashboards_main page context.
# @PURPOSE: Validate non-dashboard page contexts suppress profile-default filtering and preserve unfiltered results.
# @PRE: Profile-default preference exists but page_context=other query is provided. # @PRE: Profile-default preference exists but page_context=other query is provided.
# @POST: Response remains unfiltered and metadata reflects source_page=other. # @POST: Response remains unfiltered and metadata reflects source_page=other.
def test_get_dashboards_page_context_other_disables_profile_default(mock_deps): def test_get_dashboards_page_context_other_disables_profile_default(mock_deps):
@@ -736,6 +757,7 @@ def test_get_dashboards_page_context_other_disables_profile_default(mock_deps):
# [DEF:test_get_dashboards_profile_filter_matches_display_alias_without_detail_fanout:Function] # [DEF:test_get_dashboards_profile_filter_matches_display_alias_without_detail_fanout:Function]
# @TEST: GET /api/dashboards resolves Superset display-name alias once and filters without per-dashboard detail calls. # @TEST: GET /api/dashboards resolves Superset display-name alias once and filters without per-dashboard detail calls.
# @PURPOSE: Validate profile-default filtering reuses resolved Superset display aliases without triggering per-dashboard detail fanout.
# @PRE: Profile-default filter is active, bound username is `admin`, dashboard actors contain display labels. # @PRE: Profile-default filter is active, bound username is `admin`, dashboard actors contain display labels.
# @POST: Route matches by alias (`Superset Admin`) and does not call `SupersetClient.get_dashboard` in list filter path. # @POST: Route matches by alias (`Superset Admin`) and does not call `SupersetClient.get_dashboard` in list filter path.
def test_get_dashboards_profile_filter_matches_display_alias_without_detail_fanout(mock_deps): def test_get_dashboards_profile_filter_matches_display_alias_without_detail_fanout(mock_deps):
@@ -809,6 +831,7 @@ def test_get_dashboards_profile_filter_matches_display_alias_without_detail_fano
# [DEF:test_get_dashboards_profile_filter_matches_owner_object_payload_contract:Function] # [DEF:test_get_dashboards_profile_filter_matches_owner_object_payload_contract:Function]
# @TEST: GET /api/dashboards profile-default filter matches Superset owner object payloads. # @TEST: GET /api/dashboards profile-default filter matches Superset owner object payloads.
# @PURPOSE: Validate profile-default filtering accepts owner object payloads once aliases resolve to the bound Superset username.
# @PRE: Profile-default preference is enabled and owners list contains dict payloads. # @PRE: Profile-default preference is enabled and owners list contains dict payloads.
# @POST: Response keeps dashboards where owner object resolves to bound username alias. # @POST: Response keeps dashboards where owner object resolves to bound username alias.
def test_get_dashboards_profile_filter_matches_owner_object_payload_contract(mock_deps): def test_get_dashboards_profile_filter_matches_owner_object_payload_contract(mock_deps):
@@ -853,11 +876,16 @@ def test_get_dashboards_profile_filter_matches_owner_object_payload_contract(moc
"src.api.routes.dashboards._resolve_profile_actor_aliases", "src.api.routes.dashboards._resolve_profile_actor_aliases",
return_value=["user_1"], return_value=["user_1"],
): ):
profile_service = DomainProfileService(db=MagicMock(), config_manager=MagicMock()) profile_service = MagicMock(spec=DomainProfileService)
profile_service.get_my_preference = MagicMock( profile_service.get_my_preference.return_value = _build_profile_preference_stub(
return_value=_build_profile_preference_stub( username="user_1",
username="user_1", enabled=True,
enabled=True, )
profile_service.matches_dashboard_actor.side_effect = (
lambda bound_username, owners, modified_by: any(
str(owner.get("email", "")).split("@", 1)[0].strip().lower() == str(bound_username).strip().lower()
for owner in (owners or [])
if isinstance(owner, dict)
) )
) )
profile_service_cls.return_value = profile_service profile_service_cls.return_value = profile_service
@@ -874,4 +902,4 @@ def test_get_dashboards_profile_filter_matches_owner_object_payload_contract(moc
# [/DEF:test_get_dashboards_profile_filter_matches_owner_object_payload_contract:Function] # [/DEF:test_get_dashboards_profile_filter_matches_owner_object_payload_contract:Function]
# [/DEF:backend.src.api.routes.__tests__.test_dashboards:Module] # [/DEF:DashboardsApiTests:Module]

View File

@@ -15,24 +15,56 @@ import pytest
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
from src.app import app from src.app import app
from src.api.routes.dataset_review import _get_orchestrator, _get_repository from src.api.routes.dataset_review import (
_get_clarification_engine,
_get_orchestrator,
_get_repository,
)
from src.core.config_models import Environment, GlobalSettings, AppConfig from src.core.config_models import Environment, GlobalSettings, AppConfig
from src.core.utils.superset_context_extractor import SupersetContextExtractor from src.core.utils.superset_context_extractor import SupersetContextExtractor
from src.dependencies import get_config_manager, get_current_user, get_task_manager from src.dependencies import get_config_manager, get_current_user, get_task_manager
from src.models.dataset_review import ( from src.models.dataset_review import (
AnswerKind,
ApprovalState,
BusinessSummarySource, BusinessSummarySource,
CandidateMatchType,
CandidateStatus,
ClarificationOption,
ClarificationQuestion,
ClarificationSession,
ClarificationStatus,
CompiledPreview,
ConfidenceState, ConfidenceState,
DatasetReviewSession, DatasetReviewSession,
LaunchStatus,
ExecutionMapping,
FieldKind,
FieldProvenance,
FindingArea, FindingArea,
FindingSeverity, FindingSeverity,
MappingMethod,
PreviewStatus,
QuestionState,
ReadinessState, ReadinessState,
RecommendedAction, RecommendedAction,
ResolutionState, ResolutionState,
SemanticCandidate,
SemanticFieldEntry,
SemanticSource,
SessionPhase, SessionPhase,
SessionStatus, SessionStatus,
SemanticSourceStatus,
SemanticSourceType,
TrustLevel,
)
from src.services.dataset_review.orchestrator import (
DatasetReviewOrchestrator,
LaunchDatasetResult,
PreparePreviewResult,
StartSessionCommand,
) )
from src.services.dataset_review.orchestrator import DatasetReviewOrchestrator, StartSessionCommand
from src.services.dataset_review.semantic_resolver import SemanticSourceResolver from src.services.dataset_review.semantic_resolver import SemanticSourceResolver
from src.services.dataset_review.event_logger import SessionEventLogger
client = TestClient(app) client = TestClient(app)
@@ -85,6 +117,183 @@ def _make_session():
# [/DEF:_make_session:Function] # [/DEF:_make_session:Function]
# [DEF:_make_us2_session:Function]
def _make_us2_session():
now = datetime.now(timezone.utc)
session = _make_session()
session.readiness_state = ReadinessState.CLARIFICATION_NEEDED
session.recommended_action = RecommendedAction.START_CLARIFICATION
session.current_phase = SessionPhase.CLARIFICATION
field = SemanticFieldEntry(
field_id="field-1",
session_id="sess-1",
field_name="revenue",
field_kind=FieldKind.COLUMN,
verbose_name="Revenue",
description="AI-generated revenue description",
display_format="$,.2f",
provenance=FieldProvenance.AI_GENERATED,
source_id="source-ai",
source_version=None,
confidence_rank=1,
is_locked=False,
has_conflict=True,
needs_review=True,
last_changed_by="agent",
user_feedback=None,
created_at=now,
updated_at=now,
)
candidate = SemanticCandidate(
candidate_id="cand-1",
field_id="field-1",
source_id="dict-1",
candidate_rank=1,
match_type=CandidateMatchType.EXACT,
confidence_score=1.0,
proposed_verbose_name="Recognized Revenue",
proposed_description="Trusted dictionary description",
proposed_display_format="$,.2f",
status=CandidateStatus.PROPOSED,
created_at=now,
)
field.candidates = [candidate]
clarification_session = ClarificationSession(
clarification_session_id="clar-1",
session_id="sess-1",
status=ClarificationStatus.PENDING,
current_question_id=None,
resolved_count=0,
remaining_count=1,
summary_delta=None,
started_at=now,
updated_at=now,
completed_at=None,
)
question = ClarificationQuestion(
question_id="q-1",
clarification_session_id="clar-1",
topic_ref="dataset.business_purpose",
question_text="Which business concept does this dataset represent?",
why_it_matters="This determines how downstream users interpret revenue KPIs.",
current_guess="Revenue reporting",
priority=100,
state=QuestionState.OPEN,
created_at=now,
updated_at=now,
)
question.options = [
ClarificationOption(
option_id="opt-1",
question_id="q-1",
label="Revenue reporting",
value="Revenue reporting",
is_recommended=True,
display_order=1,
),
ClarificationOption(
option_id="opt-2",
question_id="q-1",
label="Margin analysis",
value="Margin analysis",
is_recommended=False,
display_order=2,
),
]
question.answer = None
clarification_session.questions = [question]
session.findings = []
session.collaborators = []
session.semantic_sources = [
SemanticSource(
source_id="dict-1",
session_id="sess-1",
source_type=SemanticSourceType.CONNECTED_DICTIONARY,
source_ref="dict://finance",
source_version="2026.03",
display_name="Finance Dictionary",
trust_level=TrustLevel.TRUSTED,
schema_overlap_score=1.0,
status=SemanticSourceStatus.AVAILABLE,
created_at=now,
)
]
session.semantic_fields = [field]
session.imported_filters = []
session.template_variables = []
session.execution_mappings = []
session.clarification_sessions = [clarification_session]
session.previews = []
session.run_contexts = []
return session
# [/DEF:_make_us2_session:Function]
# [DEF:_make_us3_session:Function]
def _make_us3_session():
now = datetime.now(timezone.utc)
session = _make_session()
session.readiness_state = ReadinessState.MAPPING_REVIEW_NEEDED
session.recommended_action = RecommendedAction.APPROVE_MAPPING
session.current_phase = SessionPhase.MAPPING_REVIEW
imported_filter = MagicMock()
imported_filter.filter_id = "filter-1"
imported_filter.session_id = "sess-1"
imported_filter.filter_name = "country"
imported_filter.display_name = "Country"
imported_filter.raw_value = "DE"
imported_filter.normalized_value = "DE"
imported_filter.source = "superset_url"
imported_filter.confidence_state = "imported"
imported_filter.requires_confirmation = False
imported_filter.recovery_status = "recovered"
imported_filter.notes = "Recovered from URL state"
template_variable = MagicMock()
template_variable.variable_id = "var-1"
template_variable.session_id = "sess-1"
template_variable.variable_name = "country"
template_variable.expression_source = "{{ filter_values('country') }}"
template_variable.variable_kind = "native_filter"
template_variable.is_required = True
template_variable.default_value = None
template_variable.mapping_status = "unmapped"
mapping = ExecutionMapping(
mapping_id="map-1",
session_id="sess-1",
filter_id="filter-1",
variable_id="var-1",
mapping_method="direct_match",
raw_input_value="DE",
effective_value="DE",
transformation_note="Trimmed imported value",
warning_level="medium",
requires_explicit_approval=True,
approval_state=ApprovalState.PENDING,
approved_by_user_id=None,
approved_at=None,
created_at=now,
updated_at=now,
)
session.findings = []
session.collaborators = []
session.semantic_sources = []
session.semantic_fields = []
session.imported_filters = [imported_filter]
session.template_variables = [template_variable]
session.execution_mappings = [mapping]
session.clarification_sessions = []
session.previews = []
session.run_contexts = []
return session
# [/DEF:_make_us3_session:Function]
# [DEF:dataset_review_api_dependencies:Function] # [DEF:dataset_review_api_dependencies:Function]
@pytest.fixture(autouse=True) @pytest.fixture(autouse=True)
def dataset_review_api_dependencies(): def dataset_review_api_dependencies():
@@ -319,6 +528,8 @@ def test_get_session_detail_export_and_lifecycle_endpoints(dataset_review_api_de
repository.load_session_detail.return_value = session repository.load_session_detail.return_value = session
repository.list_sessions_for_user.return_value = [session] repository.list_sessions_for_user.return_value = [session]
repository.db = MagicMock() repository.db = MagicMock()
repository.event_logger = MagicMock(spec=SessionEventLogger)
repository.event_logger.log_for_session.return_value = SimpleNamespace(session_event_id="evt-0")
app.dependency_overrides[_get_repository] = lambda: repository app.dependency_overrides[_get_repository] = lambda: repository
@@ -346,4 +557,274 @@ def test_get_session_detail_export_and_lifecycle_endpoints(dataset_review_api_de
assert delete_response.status_code == 204 assert delete_response.status_code == 204
# [/DEF:test_get_session_detail_export_and_lifecycle_endpoints:Function] # [/DEF:test_get_session_detail_export_and_lifecycle_endpoints:Function]
# [DEF:test_us2_clarification_endpoints_persist_answer_and_feedback:Function]
# @PURPOSE: Clarification endpoints should expose one current question, persist the answer before advancement, and store feedback on the answer audit record.
def test_us2_clarification_endpoints_persist_answer_and_feedback(dataset_review_api_dependencies):
session = _make_us2_session()
repository = MagicMock()
repository.load_session_detail.return_value = session
repository.db = MagicMock()
repository.db.commit.side_effect = lambda: None
repository.db.refresh.side_effect = lambda obj: None
def _add_side_effect(obj):
if obj.__class__.__name__ == "ClarificationAnswer":
session.clarification_sessions[0].questions[0].answer = obj
repository.db.add.side_effect = _add_side_effect
repository.db.flush.side_effect = lambda: None
app.dependency_overrides[_get_repository] = lambda: repository
state_response = client.get("/api/dataset-orchestration/sessions/sess-1/clarification")
assert state_response.status_code == 200
state_payload = state_response.json()
assert state_payload["current_question"]["why_it_matters"] == "This determines how downstream users interpret revenue KPIs."
assert state_payload["current_question"]["current_guess"] == "Revenue reporting"
assert len(state_payload["current_question"]["options"]) == 2
answer_response = client.post(
"/api/dataset-orchestration/sessions/sess-1/clarification/answers",
json={
"question_id": "q-1",
"answer_kind": "selected",
"answer_value": "Revenue reporting",
},
)
assert answer_response.status_code == 200
answer_payload = answer_response.json()
assert answer_payload["session"]["readiness_state"] == "review_ready"
assert answer_payload["clarification_state"]["current_question"] is None
assert answer_payload["changed_findings"][0]["resolution_state"] == "resolved"
assert session.clarification_sessions[0].questions[0].answer.answer_value == "Revenue reporting"
feedback_response = client.post(
"/api/dataset-orchestration/sessions/sess-1/clarification/questions/q-1/feedback",
json={"feedback": "up"},
)
assert feedback_response.status_code == 200
assert feedback_response.json() == {"target_id": "q-1", "feedback": "up"}
assert session.clarification_sessions[0].questions[0].answer.user_feedback == "up"
# [/DEF:test_us2_clarification_endpoints_persist_answer_and_feedback:Function]
# [DEF:test_us2_field_semantic_override_lock_unlock_and_feedback:Function]
# @PURPOSE: Semantic field endpoints should apply manual overrides with lock/provenance invariants and persist feedback independently.
def test_us2_field_semantic_override_lock_unlock_and_feedback(dataset_review_api_dependencies):
session = _make_us2_session()
repository = MagicMock()
repository.load_session_detail.return_value = session
repository.db = MagicMock()
repository.db.commit.side_effect = lambda: None
repository.db.refresh.side_effect = lambda obj: None
repository.db.add.side_effect = lambda obj: None
repository.db.flush.side_effect = lambda: None
repository.event_logger = MagicMock(spec=SessionEventLogger)
repository.event_logger.log_for_session.return_value = SimpleNamespace(session_event_id="evt-1")
app.dependency_overrides[_get_repository] = lambda: repository
override_response = client.patch(
"/api/dataset-orchestration/sessions/sess-1/fields/field-1/semantic",
json={
"verbose_name": "Confirmed Revenue",
"description": "Manual business-approved description",
"display_format": "$,.0f",
},
)
assert override_response.status_code == 200
override_payload = override_response.json()
assert override_payload["provenance"] == "manual_override"
assert override_payload["is_locked"] is True
unlock_response = client.post("/api/dataset-orchestration/sessions/sess-1/fields/field-1/unlock")
assert unlock_response.status_code == 200
assert unlock_response.json()["is_locked"] is False
candidate_response = client.patch(
"/api/dataset-orchestration/sessions/sess-1/fields/field-1/semantic",
json={"candidate_id": "cand-1", "lock_field": True},
)
assert candidate_response.status_code == 200
candidate_payload = candidate_response.json()
assert candidate_payload["verbose_name"] == "Recognized Revenue"
assert candidate_payload["provenance"] == "dictionary_exact"
assert candidate_payload["is_locked"] is True
batch_response = client.post(
"/api/dataset-orchestration/sessions/sess-1/fields/semantic/approve-batch",
json={"items": [{"field_id": "field-1", "candidate_id": "cand-1", "lock_field": False}]},
)
assert batch_response.status_code == 200
assert batch_response.json()[0]["field_id"] == "field-1"
feedback_response = client.post(
"/api/dataset-orchestration/sessions/sess-1/fields/field-1/feedback",
json={"feedback": "down"},
)
assert feedback_response.status_code == 200
assert feedback_response.json() == {"target_id": "field-1", "feedback": "down"}
assert session.semantic_fields[0].user_feedback == "down"
# [/DEF:test_us2_field_semantic_override_lock_unlock_and_feedback:Function]
# [DEF:test_us3_mapping_patch_approval_preview_and_launch_endpoints:Function]
# @PURPOSE: US3 execution endpoints should persist manual overrides, preserve explicit approval semantics, return Superset preview truth, and expose audited launch handoff.
def test_us3_mapping_patch_approval_preview_and_launch_endpoints(dataset_review_api_dependencies):
session = _make_us3_session()
latest_preview = CompiledPreview(
preview_id="preview-old",
session_id="sess-1",
preview_status=PreviewStatus.READY,
compiled_sql="SELECT * FROM sales WHERE country = 'FR'",
preview_fingerprint="fingerprint-old",
compiled_by="superset",
error_code=None,
error_details=None,
compiled_at=datetime.now(timezone.utc),
created_at=datetime.now(timezone.utc),
)
session.previews = [latest_preview]
repository = MagicMock()
repository.load_session_detail.return_value = session
repository.db = MagicMock()
repository.db.commit.side_effect = lambda: None
repository.db.refresh.side_effect = lambda obj: None
repository.event_logger = MagicMock(spec=SessionEventLogger)
repository.event_logger.log_for_session.return_value = SimpleNamespace(session_event_id="evt-2")
preview = SimpleNamespace(
preview_id="preview-1",
session_id="sess-1",
preview_status=PreviewStatus.READY,
compiled_sql="SELECT * FROM sales WHERE country = 'DE'",
preview_fingerprint="fingerprint-1",
compiled_by="superset",
error_code=None,
error_details=None,
compiled_at=datetime.now(timezone.utc),
created_at=datetime.now(timezone.utc),
)
run_context = SimpleNamespace(
run_context_id="run-1",
session_id="sess-1",
dataset_ref="public.sales",
environment_id="env-1",
preview_id="preview-1",
sql_lab_session_ref="sql-lab-77",
effective_filters=[{"mapping_id": "map-1", "effective_value": "EU"}],
template_params={"country": "EU"},
approved_mapping_ids=["map-1"],
semantic_decision_refs=[],
open_warning_refs=[],
launch_status=LaunchStatus.STARTED,
launch_error=None,
created_at=datetime.now(timezone.utc),
)
orchestrator = MagicMock()
orchestrator.prepare_launch_preview.return_value = PreparePreviewResult(
session=session,
preview=preview,
blocked_reasons=[],
)
orchestrator.launch_dataset.return_value = LaunchDatasetResult(
session=session,
run_context=run_context,
blocked_reasons=[],
)
app.dependency_overrides[_get_repository] = lambda: repository
app.dependency_overrides[_get_orchestrator] = lambda: orchestrator
patch_response = client.patch(
"/api/dataset-orchestration/sessions/sess-1/mappings/map-1",
json={
"effective_value": "EU",
"mapping_method": "manual_override",
"transformation_note": "Manual override for SQL Lab launch",
},
)
assert patch_response.status_code == 200
patch_payload = patch_response.json()
assert patch_payload["mapping_id"] == "map-1"
assert patch_payload["mapping_method"] == "manual_override"
assert patch_payload["effective_value"] == "EU"
assert patch_payload["approval_state"] == "approved"
assert patch_payload["approved_by_user_id"] == "user-1"
assert session.execution_mappings[0].mapping_method == MappingMethod.MANUAL_OVERRIDE
assert session.execution_mappings[0].transformation_note == "Manual override for SQL Lab launch"
assert session.execution_mappings[0].effective_value == "EU"
assert session.recommended_action == RecommendedAction.GENERATE_SQL_PREVIEW
assert latest_preview.preview_status == PreviewStatus.STALE
approve_response = client.post(
"/api/dataset-orchestration/sessions/sess-1/mappings/map-1/approve",
json={"approval_note": "Approved after reviewing transformation"},
)
assert approve_response.status_code == 200
approve_payload = approve_response.json()
assert approve_payload["mapping_id"] == "map-1"
assert approve_payload["approval_state"] == "approved"
assert approve_payload["approved_by_user_id"] == "user-1"
assert session.execution_mappings[0].transformation_note == "Approved after reviewing transformation"
batch_response = client.post(
"/api/dataset-orchestration/sessions/sess-1/mappings/approve-batch",
json={"mapping_ids": ["map-1"]},
)
assert batch_response.status_code == 200
assert batch_response.json()[0]["mapping_id"] == "map-1"
preview_response = client.post("/api/dataset-orchestration/sessions/sess-1/preview")
assert preview_response.status_code == 200
preview_payload = preview_response.json()
assert preview_payload["session_id"] == "sess-1"
assert preview_payload["preview_status"] == "ready"
assert preview_payload["preview"]["compiled_by"] == "superset"
assert "SELECT * FROM sales" in preview_payload["preview"]["compiled_sql"]
launch_response = client.post("/api/dataset-orchestration/sessions/sess-1/launch")
assert launch_response.status_code == 200
launch_payload = launch_response.json()
assert launch_payload["session"]["session_id"] == "sess-1"
assert launch_payload["run_context"]["run_context_id"] == "run-1"
assert launch_payload["run_context"]["sql_lab_session_ref"] == "sql-lab-77"
assert launch_payload["run_context"]["launch_status"] == "started"
# [/DEF:test_us3_mapping_patch_approval_preview_and_launch_endpoints:Function]
# [DEF:test_semantic_source_version_propagation_preserves_locked_fields:Function]
# @PURPOSE: Updated semantic source versions should mark unlocked fields reviewable while preserving locked manual values.
def test_semantic_source_version_propagation_preserves_locked_fields():
resolver = SemanticSourceResolver()
source = SimpleNamespace(source_id="src-1", source_version="2026.04")
unlocked_field = SimpleNamespace(
source_id="src-1",
source_version="2026.03",
is_locked=False,
provenance=FieldProvenance.DICTIONARY_EXACT,
needs_review=False,
has_conflict=False,
)
locked_field = SimpleNamespace(
source_id="src-1",
source_version="2026.03",
is_locked=True,
provenance=FieldProvenance.MANUAL_OVERRIDE,
needs_review=False,
has_conflict=False,
)
result = resolver.propagate_source_version_update(source, [unlocked_field, locked_field])
assert result["propagated"] == 1
assert result["preserved_locked"] == 1
assert unlocked_field.source_version == "2026.04"
assert unlocked_field.needs_review is True
assert locked_field.source_version == "2026.03"
assert locked_field.needs_review is False
# [/DEF:test_semantic_source_version_propagation_preserves_locked_fields:Function]
# [/DEF:DatasetReviewApiTests:Module] # [/DEF:DatasetReviewApiTests:Module]

View File

@@ -1,9 +1,9 @@
# [DEF:backend.src.api.routes.__tests__.test_datasets:Module] # [DEF:DatasetsApiTests:Module]
# @COMPLEXITY: 3 # @COMPLEXITY: 3
# @SEMANTICS: datasets, api, tests, pagination, mapping, docs # @SEMANTICS: datasets, api, tests, pagination, mapping, docs
# @PURPOSE: Unit tests for Datasets API endpoints # @PURPOSE: Unit tests for datasets API endpoints.
# @LAYER: API # @LAYER: API
# @RELATION: TESTS -> backend.src.api.routes.datasets # @RELATION: DEPENDS_ON -> backend.src.api.routes.datasets
# @INVARIANT: Endpoint contracts remain stable for success and validation failure paths. # @INVARIANT: Endpoint contracts remain stable for success and validation failure paths.
import pytest import pytest
@@ -89,6 +89,7 @@ def test_get_datasets_success(mock_deps):
# [DEF:test_get_datasets_env_not_found:Function] # [DEF:test_get_datasets_env_not_found:Function]
# @PURPOSE: Validate datasets listing returns 404 when the requested environment does not exist.
# @TEST: GET /api/datasets returns 404 if env_id missing # @TEST: GET /api/datasets returns 404 if env_id missing
# @PRE: env_id does not exist # @PRE: env_id does not exist
# @POST: Returns 404 error # @POST: Returns 404 error
@@ -105,6 +106,7 @@ def test_get_datasets_env_not_found(mock_deps):
# [DEF:test_get_datasets_invalid_pagination:Function] # [DEF:test_get_datasets_invalid_pagination:Function]
# @PURPOSE: Validate datasets listing rejects invalid pagination parameters with 400 responses.
# @TEST: GET /api/datasets returns 400 for invalid page/page_size # @TEST: GET /api/datasets returns 400 for invalid page/page_size
# @PRE: page < 1 or page_size > 100 # @PRE: page < 1 or page_size > 100
# @POST: Returns 400 error # @POST: Returns 400 error
@@ -133,6 +135,7 @@ def test_get_datasets_invalid_pagination(mock_deps):
# [DEF:test_map_columns_success:Function] # [DEF:test_map_columns_success:Function]
# @PURPOSE: Validate map-columns request creates an async mapping task and returns its identifier.
# @TEST: POST /api/datasets/map-columns creates mapping task # @TEST: POST /api/datasets/map-columns creates mapping task
# @PRE: Valid env_id, dataset_ids, source_type # @PRE: Valid env_id, dataset_ids, source_type
# @POST: Returns task_id # @POST: Returns task_id
@@ -167,6 +170,7 @@ def test_map_columns_success(mock_deps):
# [DEF:test_map_columns_invalid_source_type:Function] # [DEF:test_map_columns_invalid_source_type:Function]
# @PURPOSE: Validate map-columns rejects unsupported source types with a 400 contract response.
# @TEST: POST /api/datasets/map-columns returns 400 for invalid source_type # @TEST: POST /api/datasets/map-columns returns 400 for invalid source_type
# @PRE: source_type is not 'postgresql' or 'xlsx' # @PRE: source_type is not 'postgresql' or 'xlsx'
# @POST: Returns 400 error # @POST: Returns 400 error
@@ -190,6 +194,7 @@ def test_map_columns_invalid_source_type(mock_deps):
# [DEF:test_generate_docs_success:Function] # [DEF:test_generate_docs_success:Function]
# @TEST: POST /api/datasets/generate-docs creates doc generation task # @TEST: POST /api/datasets/generate-docs creates doc generation task
# @PRE: Valid env_id, dataset_ids, llm_provider # @PRE: Valid env_id, dataset_ids, llm_provider
# @PURPOSE: Validate generate-docs request creates an async documentation task and returns its identifier.
# @POST: Returns task_id # @POST: Returns task_id
def test_generate_docs_success(mock_deps): def test_generate_docs_success(mock_deps):
# Mock environment # Mock environment
@@ -222,6 +227,7 @@ def test_generate_docs_success(mock_deps):
# [DEF:test_map_columns_empty_ids:Function] # [DEF:test_map_columns_empty_ids:Function]
# @PURPOSE: Validate map-columns rejects empty dataset identifier lists.
# @TEST: POST /api/datasets/map-columns returns 400 for empty dataset_ids # @TEST: POST /api/datasets/map-columns returns 400 for empty dataset_ids
# @PRE: dataset_ids is empty # @PRE: dataset_ids is empty
# @POST: Returns 400 error # @POST: Returns 400 error
@@ -241,6 +247,7 @@ def test_map_columns_empty_ids(mock_deps):
# [DEF:test_generate_docs_empty_ids:Function] # [DEF:test_generate_docs_empty_ids:Function]
# @PURPOSE: Validate generate-docs rejects empty dataset identifier lists.
# @TEST: POST /api/datasets/generate-docs returns 400 for empty dataset_ids # @TEST: POST /api/datasets/generate-docs returns 400 for empty dataset_ids
# @PRE: dataset_ids is empty # @PRE: dataset_ids is empty
# @POST: Returns 400 error # @POST: Returns 400 error
@@ -262,6 +269,7 @@ def test_generate_docs_empty_ids(mock_deps):
# [DEF:test_generate_docs_env_not_found:Function] # [DEF:test_generate_docs_env_not_found:Function]
# @TEST: POST /api/datasets/generate-docs returns 404 for missing env # @TEST: POST /api/datasets/generate-docs returns 404 for missing env
# @PRE: env_id does not exist # @PRE: env_id does not exist
# @PURPOSE: Validate generate-docs returns 404 when the requested environment cannot be resolved.
# @POST: Returns 404 error # @POST: Returns 404 error
def test_generate_docs_env_not_found(mock_deps): def test_generate_docs_env_not_found(mock_deps):
"""@PRE: env_id must be a valid environment.""" """@PRE: env_id must be a valid environment."""
@@ -280,6 +288,7 @@ def test_generate_docs_env_not_found(mock_deps):
# [DEF:test_get_datasets_superset_failure:Function] # [DEF:test_get_datasets_superset_failure:Function]
# @PURPOSE: Validate datasets listing surfaces a 503 contract when Superset access fails.
# @TEST_EDGE: external_superset_failure -> {status: 503} # @TEST_EDGE: external_superset_failure -> {status: 503}
def test_get_datasets_superset_failure(mock_deps): def test_get_datasets_superset_failure(mock_deps):
"""@TEST_EDGE: external_superset_failure -> {status: 503}""" """@TEST_EDGE: external_superset_failure -> {status: 503}"""
@@ -297,4 +306,4 @@ def test_get_datasets_superset_failure(mock_deps):
# [/DEF:test_get_datasets_superset_failure:Function] # [/DEF:test_get_datasets_superset_failure:Function]
# [/DEF:backend.src.api.routes.__tests__.test_datasets:Module] # [/DEF:DatasetsApiTests:Module]

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,406 @@
# [DEF:SupersetCompilationAdapter:Module]
# @COMPLEXITY: 4
# @SEMANTICS: dataset_review, superset, compilation_preview, sql_lab_launch, execution_truth
# @PURPOSE: Interact with Superset preview compilation and SQL Lab execution endpoints using the current approved execution context.
# @LAYER: Infra
# @RELATION: [CALLS] ->[backend.src.core.superset_client.SupersetClient]
# @RELATION: [DEPENDS_ON] ->[CompiledPreview]
# @RELATION: [DEPENDS_ON] ->[DatasetRunContext]
# @PRE: effective template params and dataset execution reference are available.
# @POST: preview and launch calls return Superset-originated artifacts or explicit errors.
# @SIDE_EFFECT: performs upstream Superset preview and SQL Lab calls.
# @INVARIANT: The adapter never fabricates compiled SQL locally; preview truth is delegated to Superset only.
from __future__ import annotations
# [DEF:SupersetCompilationAdapter.imports:Block]
from dataclasses import dataclass
from datetime import datetime
from typing import Any, Dict, List, Optional
from src.core.config_models import Environment
from src.core.logger import belief_scope, logger
from src.core.superset_client import SupersetClient
from src.models.dataset_review import CompiledPreview, PreviewStatus
# [/DEF:SupersetCompilationAdapter.imports:Block]
# [DEF:PreviewCompilationPayload:Class]
# @COMPLEXITY: 2
# @PURPOSE: Typed preview payload for Superset-side compilation.
@dataclass(frozen=True)
class PreviewCompilationPayload:
session_id: str
dataset_id: int
preview_fingerprint: str
template_params: Dict[str, Any]
effective_filters: List[Dict[str, Any]]
# [/DEF:PreviewCompilationPayload:Class]
# [DEF:SqlLabLaunchPayload:Class]
# @COMPLEXITY: 2
# @PURPOSE: Typed SQL Lab payload for audited launch handoff.
@dataclass(frozen=True)
class SqlLabLaunchPayload:
session_id: str
dataset_id: int
preview_id: str
compiled_sql: str
template_params: Dict[str, Any]
# [/DEF:SqlLabLaunchPayload:Class]
# [DEF:SupersetCompilationAdapter:Class]
# @COMPLEXITY: 4
# @PURPOSE: Delegate preview compilation and SQL Lab launch to Superset without local SQL fabrication.
# @RELATION: [CALLS] ->[backend.src.core.superset_client.SupersetClient]
# @PRE: environment is configured and Superset is reachable for the target session.
# @POST: adapter can return explicit ready/failed preview artifacts and canonical SQL Lab references.
# @SIDE_EFFECT: issues network requests to Superset API surfaces.
class SupersetCompilationAdapter:
# [DEF:SupersetCompilationAdapter.__init__:Function]
# @COMPLEXITY: 2
# @PURPOSE: Bind adapter to one Superset environment and client instance.
def __init__(self, environment: Environment, client: Optional[SupersetClient] = None) -> None:
self.environment = environment
self.client = client or SupersetClient(environment)
# [/DEF:SupersetCompilationAdapter.__init__:Function]
# [DEF:SupersetCompilationAdapter.compile_preview:Function]
# @COMPLEXITY: 4
# @PURPOSE: Request Superset-side compiled SQL preview for the current effective inputs.
# @RELATION: [CALLS] ->[SupersetCompilationAdapter._request_superset_preview]
# @PRE: dataset_id and effective inputs are available for the current session.
# @POST: returns a ready or failed preview artifact backed only by Superset-originated SQL or diagnostics.
# @SIDE_EFFECT: performs upstream preview requests.
# @DATA_CONTRACT: Input[PreviewCompilationPayload] -> Output[CompiledPreview]
def compile_preview(self, payload: PreviewCompilationPayload) -> CompiledPreview:
with belief_scope("SupersetCompilationAdapter.compile_preview"):
if payload.dataset_id <= 0:
logger.explore(
"Preview compilation rejected because dataset identifier is invalid",
extra={"dataset_id": payload.dataset_id, "session_id": payload.session_id},
)
raise ValueError("dataset_id must be a positive integer")
logger.reason(
"Requesting Superset-generated SQL preview",
extra={
"session_id": payload.session_id,
"dataset_id": payload.dataset_id,
"template_param_count": len(payload.template_params),
"filter_count": len(payload.effective_filters),
},
)
try:
preview_result = self._request_superset_preview(payload)
except Exception as exc:
logger.explore(
"Superset preview compilation failed with explicit upstream error",
extra={
"session_id": payload.session_id,
"dataset_id": payload.dataset_id,
"error": str(exc),
},
)
return CompiledPreview(
session_id=payload.session_id,
preview_status=PreviewStatus.FAILED,
compiled_sql=None,
preview_fingerprint=payload.preview_fingerprint,
compiled_by="superset",
error_code="superset_preview_failed",
error_details=str(exc),
compiled_at=None,
)
compiled_sql = str(preview_result.get("compiled_sql") or "").strip()
if not compiled_sql:
logger.explore(
"Superset preview response did not include compiled SQL",
extra={
"session_id": payload.session_id,
"dataset_id": payload.dataset_id,
"response_keys": sorted(preview_result.keys()),
},
)
return CompiledPreview(
session_id=payload.session_id,
preview_status=PreviewStatus.FAILED,
compiled_sql=None,
preview_fingerprint=payload.preview_fingerprint,
compiled_by="superset",
error_code="superset_preview_empty",
error_details="Superset preview response did not include compiled SQL",
compiled_at=None,
)
preview = CompiledPreview(
session_id=payload.session_id,
preview_status=PreviewStatus.READY,
compiled_sql=compiled_sql,
preview_fingerprint=payload.preview_fingerprint,
compiled_by="superset",
error_code=None,
error_details=None,
compiled_at=datetime.utcnow(),
)
logger.reflect(
"Superset-generated SQL preview captured successfully",
extra={
"session_id": payload.session_id,
"dataset_id": payload.dataset_id,
"compiled_sql_length": len(compiled_sql),
},
)
return preview
# [/DEF:SupersetCompilationAdapter.compile_preview:Function]
# [DEF:SupersetCompilationAdapter.mark_preview_stale:Function]
# @COMPLEXITY: 2
# @PURPOSE: Invalidate previous preview after mapping or value changes.
# @PRE: preview is a persisted preview artifact or current in-memory snapshot.
# @POST: preview status becomes stale without fabricating a replacement artifact.
def mark_preview_stale(self, preview: CompiledPreview) -> CompiledPreview:
preview.preview_status = PreviewStatus.STALE
return preview
# [/DEF:SupersetCompilationAdapter.mark_preview_stale:Function]
# [DEF:SupersetCompilationAdapter.create_sql_lab_session:Function]
# @COMPLEXITY: 4
# @PURPOSE: Create the canonical audited execution session after all launch gates pass.
# @RELATION: [CALLS] ->[SupersetCompilationAdapter._request_sql_lab_session]
# @PRE: compiled_sql is Superset-originated and launch gates are already satisfied.
# @POST: returns one canonical SQL Lab session reference from Superset.
# @SIDE_EFFECT: performs upstream SQL Lab execution/session creation.
# @DATA_CONTRACT: Input[SqlLabLaunchPayload] -> Output[str]
def create_sql_lab_session(self, payload: SqlLabLaunchPayload) -> str:
with belief_scope("SupersetCompilationAdapter.create_sql_lab_session"):
compiled_sql = str(payload.compiled_sql or "").strip()
if not compiled_sql:
logger.explore(
"SQL Lab launch rejected because compiled SQL is empty",
extra={"session_id": payload.session_id, "preview_id": payload.preview_id},
)
raise ValueError("compiled_sql must be non-empty")
logger.reason(
"Creating SQL Lab execution session from Superset-originated preview",
extra={
"session_id": payload.session_id,
"dataset_id": payload.dataset_id,
"preview_id": payload.preview_id,
},
)
result = self._request_sql_lab_session(payload)
sql_lab_session_ref = str(
result.get("sql_lab_session_ref")
or result.get("query_id")
or result.get("id")
or result.get("result", {}).get("id")
or ""
).strip()
if not sql_lab_session_ref:
logger.explore(
"Superset SQL Lab launch response did not include a stable session reference",
extra={"session_id": payload.session_id, "preview_id": payload.preview_id},
)
raise RuntimeError("Superset SQL Lab launch response did not include a session reference")
logger.reflect(
"Canonical SQL Lab session created successfully",
extra={
"session_id": payload.session_id,
"preview_id": payload.preview_id,
"sql_lab_session_ref": sql_lab_session_ref,
},
)
return sql_lab_session_ref
# [/DEF:SupersetCompilationAdapter.create_sql_lab_session:Function]
# [DEF:SupersetCompilationAdapter._request_superset_preview:Function]
# @COMPLEXITY: 4
# @PURPOSE: Probe supported Superset preview surfaces and return the first explicit compilation response.
# @RELATION: [CALLS] ->[backend.src.core.superset_client.SupersetClient]
# @PRE: payload contains a valid dataset identifier and deterministic execution inputs for one preview attempt.
# @POST: returns the first upstream response that exposes compiled SQL without fabricating local SQL.
# @SIDE_EFFECT: issues one or more Superset preview requests until a supported surface responds.
# @DATA_CONTRACT: Input[PreviewCompilationPayload] -> Output[Dict[str,Any]]
def _request_superset_preview(self, payload: PreviewCompilationPayload) -> Dict[str, Any]:
request_payload = {
"dataset_id": payload.dataset_id,
"template_params": payload.template_params,
"effective_filters": payload.effective_filters,
"session_id": payload.session_id,
}
candidate_calls = self._build_preview_call_candidates(payload.dataset_id, request_payload)
errors: List[str] = []
for candidate in candidate_calls:
call_kind = candidate["kind"]
target = candidate["target"]
try:
logger.reason(
"Attempting Superset preview compilation candidate",
extra={"kind": call_kind, "target": target},
)
if call_kind == "client_method":
method = getattr(self.client, target)
response = method(request_payload)
else:
response = self.client.network.request(
method=candidate["http_method"],
endpoint=target,
data=candidate["data"],
headers={"Content-Type": "application/json"},
)
normalized = self._normalize_preview_response(response)
if normalized is not None:
return normalized
except Exception as exc:
errors.append(f"{call_kind}:{target}:{exc}")
logger.explore(
"Superset preview compilation candidate failed",
extra={"kind": call_kind, "target": target, "error": str(exc)},
)
raise RuntimeError("; ".join(errors) or "No Superset preview surface accepted the request")
# [/DEF:SupersetCompilationAdapter._request_superset_preview:Function]
# [DEF:SupersetCompilationAdapter._request_sql_lab_session:Function]
# @COMPLEXITY: 4
# @PURPOSE: Probe supported SQL Lab execution surfaces and return the first successful response.
# @RELATION: [CALLS] ->[backend.src.core.superset_client.SupersetClient]
# @PRE: payload carries non-empty Superset-originated SQL and a preview identifier for the current launch.
# @POST: returns the first successful SQL Lab execution response from Superset.
# @SIDE_EFFECT: issues Superset dataset lookup and SQL Lab execution requests.
# @DATA_CONTRACT: Input[SqlLabLaunchPayload] -> Output[Dict[str,Any]]
def _request_sql_lab_session(self, payload: SqlLabLaunchPayload) -> Dict[str, Any]:
dataset_raw = self.client.get_dataset(payload.dataset_id)
dataset_record = dataset_raw.get("result", dataset_raw) if isinstance(dataset_raw, dict) else {}
database_id = dataset_record.get("database", {}).get("id") if isinstance(dataset_record.get("database"), dict) else dataset_record.get("database_id")
if database_id is None:
raise RuntimeError("Superset dataset does not expose a database identifier for SQL Lab launch")
request_payload = {
"database_id": database_id,
"sql": payload.compiled_sql,
"templateParams": payload.template_params,
"schema": dataset_record.get("schema"),
"client_id": payload.preview_id,
}
candidate_calls = [
{"kind": "network", "target": "/sqllab/execute/", "http_method": "POST"},
{"kind": "network", "target": "/sql_lab/execute/", "http_method": "POST"},
]
errors: List[str] = []
for candidate in candidate_calls:
try:
response = self.client.network.request(
method=candidate["http_method"],
endpoint=candidate["target"],
data=self._dump_json(request_payload),
headers={"Content-Type": "application/json"},
)
if isinstance(response, dict) and response:
return response
except Exception as exc:
errors.append(f"{candidate['target']}:{exc}")
logger.explore(
"Superset SQL Lab candidate failed",
extra={"target": candidate["target"], "error": str(exc)},
)
raise RuntimeError("; ".join(errors) or "No Superset SQL Lab surface accepted the request")
# [/DEF:SupersetCompilationAdapter._request_sql_lab_session:Function]
# [DEF:SupersetCompilationAdapter._build_preview_call_candidates:Function]
# @COMPLEXITY: 2
# @PURPOSE: Assemble preview candidate call shapes in priority order.
def _build_preview_call_candidates(
self,
dataset_id: int,
request_payload: Dict[str, Any],
) -> List[Dict[str, Any]]:
candidates: List[Dict[str, Any]] = []
for method_name in (
"compile_sql_preview",
"compile_preview",
"get_compiled_sql_preview",
):
if hasattr(self.client, method_name):
candidates.append({"kind": "client_method", "target": method_name})
encoded_payload = self._dump_json(request_payload)
candidates.extend(
[
{
"kind": "network",
"target": f"/dataset/{dataset_id}/preview",
"http_method": "POST",
"data": encoded_payload,
},
{
"kind": "network",
"target": f"/dataset/{dataset_id}/sql",
"http_method": "POST",
"data": encoded_payload,
},
{
"kind": "network",
"target": "/sqllab/format_sql/",
"http_method": "POST",
"data": encoded_payload,
},
]
)
return candidates
# [/DEF:SupersetCompilationAdapter._build_preview_call_candidates:Function]
# [DEF:SupersetCompilationAdapter._normalize_preview_response:Function]
# @COMPLEXITY: 3
# @PURPOSE: Normalize candidate Superset preview responses into one compiled-sql structure.
# @RELATION: [DEPENDS_ON] ->[CompiledPreview]
def _normalize_preview_response(self, response: Any) -> Optional[Dict[str, Any]]:
if not isinstance(response, dict):
return None
compiled_sql_candidates = [
response.get("compiled_sql"),
response.get("sql"),
response.get("query"),
]
result_payload = response.get("result")
if isinstance(result_payload, dict):
compiled_sql_candidates.extend(
[
result_payload.get("compiled_sql"),
result_payload.get("sql"),
result_payload.get("query"),
]
)
for candidate in compiled_sql_candidates:
compiled_sql = str(candidate or "").strip()
if compiled_sql:
return {
"compiled_sql": compiled_sql,
"raw_response": response,
}
return None
# [/DEF:SupersetCompilationAdapter._normalize_preview_response:Function]
# [DEF:SupersetCompilationAdapter._dump_json:Function]
# @COMPLEXITY: 1
# @PURPOSE: Serialize Superset request payload deterministically for network transport.
def _dump_json(self, payload: Dict[str, Any]) -> str:
import json
return json.dumps(payload, sort_keys=True, default=str)
# [/DEF:SupersetCompilationAdapter._dump_json:Function]
# [/DEF:SupersetCompilationAdapter:Class]
# [/DEF:SupersetCompilationAdapter:Module]

View File

@@ -15,8 +15,9 @@ from __future__ import annotations
# [DEF:SupersetContextExtractor.imports:Block] # [DEF:SupersetContextExtractor.imports:Block]
import json import json
import re
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional from typing import Any, Dict, List, Optional, Set
from urllib.parse import parse_qs, unquote, urlparse from urllib.parse import parse_qs, unquote, urlparse
from src.core.config_models import Environment from src.core.config_models import Environment
@@ -204,17 +205,224 @@ class SupersetContextExtractor:
# [/DEF:SupersetContextExtractor.parse_superset_link:Function] # [/DEF:SupersetContextExtractor.parse_superset_link:Function]
# [DEF:SupersetContextExtractor.recover_imported_filters:Function] # [DEF:SupersetContextExtractor.recover_imported_filters:Function]
# @COMPLEXITY: 2 # @COMPLEXITY: 4
# @PURPOSE: Build imported filter entries from URL state and Superset-side saved context. # @PURPOSE: Build imported filter entries from URL state and Superset-side saved context.
# @RELATION: [CALLS] ->[backend.src.core.superset_client.SupersetClient]
# @PRE: parsed_context comes from a successful Superset link parse for one environment.
# @POST: returns explicit recovered and partial filter entries with preserved provenance and confirmation requirements.
# @SIDE_EFFECT: may issue Superset reads for dashboard metadata enrichment.
# @DATA_CONTRACT: Input[SupersetParsedContext] -> Output[List[Dict[str,Any]]]
def recover_imported_filters(self, parsed_context: SupersetParsedContext) -> List[Dict[str, Any]]: def recover_imported_filters(self, parsed_context: SupersetParsedContext) -> List[Dict[str, Any]]:
return list(parsed_context.imported_filters) with belief_scope("SupersetContextExtractor.recover_imported_filters"):
recovered_filters: List[Dict[str, Any]] = []
seen_filter_keys: Set[str] = set()
for item in parsed_context.imported_filters:
normalized = self._normalize_imported_filter_payload(
item,
default_source="superset_url",
default_note="Recovered from Superset URL state",
)
filter_key = normalized["filter_name"].strip().lower()
if filter_key in seen_filter_keys:
continue
seen_filter_keys.add(filter_key)
recovered_filters.append(normalized)
if parsed_context.dashboard_id is None:
logger.reflect(
"Imported filter recovery completed without dashboard enrichment",
extra={
"dashboard_id": None,
"filter_count": len(recovered_filters),
"partial_recovery": parsed_context.partial_recovery,
},
)
return recovered_filters
try:
dashboard_payload = self.client.get_dashboard(parsed_context.dashboard_id)
dashboard_record = (
dashboard_payload.get("result", dashboard_payload)
if isinstance(dashboard_payload, dict)
else {}
)
json_metadata = dashboard_record.get("json_metadata")
if isinstance(json_metadata, str) and json_metadata.strip():
json_metadata = json.loads(json_metadata)
if not isinstance(json_metadata, dict):
json_metadata = {}
native_filter_configuration = json_metadata.get("native_filter_configuration") or []
default_filters = json_metadata.get("default_filters") or {}
if isinstance(default_filters, str) and default_filters.strip():
try:
default_filters = json.loads(default_filters)
except Exception:
logger.explore(
"Superset default_filters payload was not valid JSON",
extra={"dashboard_id": parsed_context.dashboard_id},
)
default_filters = {}
for item in native_filter_configuration:
if not isinstance(item, dict):
continue
filter_name = str(
item.get("name")
or item.get("filter_name")
or item.get("column")
or ""
).strip()
if not filter_name:
continue
filter_key = filter_name.lower()
if filter_key in seen_filter_keys:
continue
default_value = None
if isinstance(default_filters, dict):
default_value = default_filters.get(filter_name)
saved_filter = self._normalize_imported_filter_payload(
{
"filter_name": filter_name,
"display_name": item.get("label") or item.get("name"),
"raw_value": default_value,
"source": "superset_native",
"recovery_status": "recovered" if default_value is not None else "partial",
"requires_confirmation": default_value is None,
"notes": "Recovered from Superset dashboard native filter configuration",
},
default_source="superset_native",
default_note="Recovered from Superset dashboard native filter configuration",
)
seen_filter_keys.add(filter_key)
recovered_filters.append(saved_filter)
logger.reflect(
"Imported filter recovery completed with dashboard enrichment",
extra={
"dashboard_id": parsed_context.dashboard_id,
"filter_count": len(recovered_filters),
"partial_entries": len(
[
item
for item in recovered_filters
if item["recovery_status"] == "partial"
]
),
},
)
return recovered_filters
except Exception as exc:
logger.explore(
"Dashboard native filter enrichment failed; preserving partial imported filters",
extra={
"dashboard_id": parsed_context.dashboard_id,
"error": str(exc),
"filter_count": len(recovered_filters),
},
)
if not recovered_filters:
recovered_filters.append(
self._normalize_imported_filter_payload(
{
"filter_name": f"dashboard_{parsed_context.dashboard_id}_filters",
"display_name": "Dashboard native filters",
"raw_value": None,
"source": "superset_native",
"recovery_status": "partial",
"requires_confirmation": True,
"notes": "Superset dashboard filter configuration could not be recovered fully",
},
default_source="superset_native",
default_note="Superset dashboard filter configuration could not be recovered fully",
)
)
return recovered_filters
# [/DEF:SupersetContextExtractor.recover_imported_filters:Function] # [/DEF:SupersetContextExtractor.recover_imported_filters:Function]
# [DEF:SupersetContextExtractor.discover_template_variables:Function] # [DEF:SupersetContextExtractor.discover_template_variables:Function]
# @COMPLEXITY: 2 # @COMPLEXITY: 4
# @PURPOSE: Detect runtime variables and Jinja references from dataset query-bearing fields. # @PURPOSE: Detect runtime variables and Jinja references from dataset query-bearing fields.
# @RELATION: [DEPENDS_ON] ->[TemplateVariable]
# @PRE: dataset_payload is a Superset dataset-detail style payload with query-bearing fields when available.
# @POST: returns deduplicated explicit variable records without executing Jinja or fabricating runtime values.
# @SIDE_EFFECT: none.
# @DATA_CONTRACT: Input[dataset_payload:Dict[str,Any]] -> Output[List[Dict[str,Any]]]
def discover_template_variables(self, dataset_payload: Dict[str, Any]) -> List[Dict[str, Any]]: def discover_template_variables(self, dataset_payload: Dict[str, Any]) -> List[Dict[str, Any]]:
return [] with belief_scope("SupersetContextExtractor.discover_template_variables"):
discovered: List[Dict[str, Any]] = []
seen_variable_names: Set[str] = set()
for expression_source in self._collect_query_bearing_expressions(dataset_payload):
for filter_match in re.finditer(
r"filter_values\(\s*['\"]([^'\"]+)['\"]\s*\)",
expression_source,
flags=re.IGNORECASE,
):
variable_name = str(filter_match.group(1) or "").strip()
if not variable_name:
continue
self._append_template_variable(
discovered=discovered,
seen_variable_names=seen_variable_names,
variable_name=variable_name,
expression_source=expression_source,
variable_kind="native_filter",
is_required=True,
default_value=None,
)
for url_param_match in re.finditer(
r"url_param\(\s*['\"]([^'\"]+)['\"]\s*(?:,\s*([^)]+))?\)",
expression_source,
flags=re.IGNORECASE,
):
variable_name = str(url_param_match.group(1) or "").strip()
if not variable_name:
continue
default_literal = url_param_match.group(2)
self._append_template_variable(
discovered=discovered,
seen_variable_names=seen_variable_names,
variable_name=variable_name,
expression_source=expression_source,
variable_kind="parameter",
is_required=default_literal is None,
default_value=self._normalize_default_literal(default_literal),
)
for jinja_match in re.finditer(r"\{\{\s*(.*?)\s*\}\}", expression_source, flags=re.DOTALL):
expression = str(jinja_match.group(1) or "").strip()
if not expression:
continue
if any(token in expression for token in ("filter_values(", "url_param(", "get_filters(")):
continue
variable_name = self._extract_primary_jinja_identifier(expression)
if not variable_name:
continue
self._append_template_variable(
discovered=discovered,
seen_variable_names=seen_variable_names,
variable_name=variable_name,
expression_source=expression_source,
variable_kind="derived" if "." in expression or "|" in expression else "parameter",
is_required=True,
default_value=None,
)
logger.reflect(
"Template variable discovery completed deterministically",
extra={
"dataset_id": dataset_payload.get("id"),
"variable_count": len(discovered),
"variable_names": [item["variable_name"] for item in discovered],
},
)
return discovered
# [/DEF:SupersetContextExtractor.discover_template_variables:Function] # [/DEF:SupersetContextExtractor.discover_template_variables:Function]
# [DEF:SupersetContextExtractor.build_recovery_summary:Function] # [DEF:SupersetContextExtractor.build_recovery_summary:Function]
@@ -329,6 +537,151 @@ class SupersetContextExtractor:
return imported_filters return imported_filters
# [/DEF:SupersetContextExtractor._extract_imported_filters:Function] # [/DEF:SupersetContextExtractor._extract_imported_filters:Function]
# [DEF:SupersetContextExtractor._normalize_imported_filter_payload:Function]
# @COMPLEXITY: 2
# @PURPOSE: Normalize one imported-filter payload with explicit provenance and confirmation state.
def _normalize_imported_filter_payload(
self,
payload: Dict[str, Any],
default_source: str,
default_note: str,
) -> Dict[str, Any]:
raw_value = payload.get("raw_value")
if "raw_value" not in payload and "value" in payload:
raw_value = payload.get("value")
recovery_status = str(
payload.get("recovery_status")
or ("recovered" if raw_value is not None else "partial")
).strip().lower()
requires_confirmation = bool(
payload.get("requires_confirmation", raw_value is None or recovery_status != "recovered")
)
return {
"filter_name": str(payload.get("filter_name") or "unresolved_filter").strip(),
"display_name": payload.get("display_name"),
"raw_value": raw_value,
"normalized_value": payload.get("normalized_value"),
"source": str(payload.get("source") or default_source),
"confidence_state": "imported" if raw_value is not None else "unresolved",
"requires_confirmation": requires_confirmation,
"recovery_status": recovery_status,
"notes": str(payload.get("notes") or default_note),
}
# [/DEF:SupersetContextExtractor._normalize_imported_filter_payload:Function]
# [DEF:SupersetContextExtractor._collect_query_bearing_expressions:Function]
# @COMPLEXITY: 3
# @PURPOSE: Collect SQL and expression-bearing dataset fields for deterministic template-variable discovery.
# @RELATION: [DEPENDS_ON] ->[SupersetContextExtractor.discover_template_variables]
def _collect_query_bearing_expressions(self, dataset_payload: Dict[str, Any]) -> List[str]:
expressions: List[str] = []
def append_expression(candidate: Any) -> None:
if not isinstance(candidate, str):
return
normalized = candidate.strip()
if normalized:
expressions.append(normalized)
append_expression(dataset_payload.get("sql"))
append_expression(dataset_payload.get("query"))
append_expression(dataset_payload.get("template_sql"))
metrics_payload = dataset_payload.get("metrics") or []
if isinstance(metrics_payload, list):
for metric in metrics_payload:
if isinstance(metric, str):
append_expression(metric)
continue
if not isinstance(metric, dict):
continue
append_expression(metric.get("expression"))
append_expression(metric.get("sqlExpression"))
append_expression(metric.get("metric_name"))
columns_payload = dataset_payload.get("columns") or []
if isinstance(columns_payload, list):
for column in columns_payload:
if not isinstance(column, dict):
continue
append_expression(column.get("sqlExpression"))
append_expression(column.get("expression"))
return expressions
# [/DEF:SupersetContextExtractor._collect_query_bearing_expressions:Function]
# [DEF:SupersetContextExtractor._append_template_variable:Function]
# @COMPLEXITY: 2
# @PURPOSE: Append one deduplicated template-variable descriptor.
def _append_template_variable(
self,
discovered: List[Dict[str, Any]],
seen_variable_names: Set[str],
variable_name: str,
expression_source: str,
variable_kind: str,
is_required: bool,
default_value: Any,
) -> None:
normalized_name = str(variable_name or "").strip()
if not normalized_name:
return
seen_key = normalized_name.lower()
if seen_key in seen_variable_names:
return
seen_variable_names.add(seen_key)
discovered.append(
{
"variable_name": normalized_name,
"expression_source": expression_source,
"variable_kind": variable_kind,
"is_required": is_required,
"default_value": default_value,
"mapping_status": "unmapped",
}
)
# [/DEF:SupersetContextExtractor._append_template_variable:Function]
# [DEF:SupersetContextExtractor._extract_primary_jinja_identifier:Function]
# @COMPLEXITY: 2
# @PURPOSE: Extract a deterministic primary identifier from a Jinja expression without executing it.
def _extract_primary_jinja_identifier(self, expression: str) -> Optional[str]:
matched = re.match(r"([A-Za-z_][A-Za-z0-9_]*)", expression.strip())
if matched is None:
return None
candidate = matched.group(1)
if candidate in {"if", "else", "for", "set", "True", "False", "none", "None"}:
return None
return candidate
# [/DEF:SupersetContextExtractor._extract_primary_jinja_identifier:Function]
# [DEF:SupersetContextExtractor._normalize_default_literal:Function]
# @COMPLEXITY: 2
# @PURPOSE: Normalize literal default fragments from template helper calls into JSON-safe values.
def _normalize_default_literal(self, literal: Optional[str]) -> Any:
normalized_literal = str(literal or "").strip()
if not normalized_literal:
return None
if (
(normalized_literal.startswith("'") and normalized_literal.endswith("'"))
or (normalized_literal.startswith('"') and normalized_literal.endswith('"'))
):
return normalized_literal[1:-1]
lowered = normalized_literal.lower()
if lowered in {"true", "false"}:
return lowered == "true"
if lowered in {"null", "none"}:
return None
try:
return int(normalized_literal)
except ValueError:
try:
return float(normalized_literal)
except ValueError:
return normalized_literal
# [/DEF:SupersetContextExtractor._normalize_default_literal:Function]
# [/DEF:SupersetContextExtractor:Class] # [/DEF:SupersetContextExtractor:Class]
# [/DEF:SupersetContextExtractor:Module] # [/DEF:SupersetContextExtractor:Module]

View File

@@ -132,6 +132,7 @@ class DatasetReviewSession(Base):
previews = relationship("CompiledPreview", back_populates="session", cascade="all, delete-orphan") previews = relationship("CompiledPreview", back_populates="session", cascade="all, delete-orphan")
run_contexts = relationship("DatasetRunContext", back_populates="session", cascade="all, delete-orphan") run_contexts = relationship("DatasetRunContext", back_populates="session", cascade="all, delete-orphan")
export_artifacts = relationship("ExportArtifact", back_populates="session", cascade="all, delete-orphan") export_artifacts = relationship("ExportArtifact", back_populates="session", cascade="all, delete-orphan")
events = relationship("SessionEvent", back_populates="session", cascade="all, delete-orphan")
# [/DEF:DatasetReviewSession:Class] # [/DEF:DatasetReviewSession:Class]
# [DEF:BusinessSummarySource:Class] # [DEF:BusinessSummarySource:Class]
@@ -303,6 +304,7 @@ class SemanticFieldEntry(Base):
display_format = Column(String, nullable=True) display_format = Column(String, nullable=True)
provenance = Column(SQLEnum(FieldProvenance), nullable=False, default=FieldProvenance.UNRESOLVED) provenance = Column(SQLEnum(FieldProvenance), nullable=False, default=FieldProvenance.UNRESOLVED)
source_id = Column(String, nullable=True) source_id = Column(String, nullable=True)
source_version = Column(String, nullable=True)
confidence_rank = Column(Integer, nullable=True) confidence_rank = Column(Integer, nullable=True)
is_locked = Column(Boolean, nullable=False, default=False) is_locked = Column(Boolean, nullable=False, default=False)
has_conflict = Column(Boolean, nullable=False, default=False) has_conflict = Column(Boolean, nullable=False, default=False)
@@ -567,6 +569,7 @@ class ClarificationAnswer(Base):
answer_value = Column(Text, nullable=True) answer_value = Column(Text, nullable=True)
answered_by_user_id = Column(String, nullable=False) answered_by_user_id = Column(String, nullable=False)
impact_summary = Column(Text, nullable=True) impact_summary = Column(Text, nullable=True)
user_feedback = Column(String, nullable=True) # up, down, null
created_at = Column(DateTime, default=datetime.utcnow, nullable=False) created_at = Column(DateTime, default=datetime.utcnow, nullable=False)
question = relationship("ClarificationQuestion", back_populates="answer") question = relationship("ClarificationQuestion", back_populates="answer")
@@ -627,6 +630,24 @@ class DatasetRunContext(Base):
session = relationship("DatasetReviewSession", back_populates="run_contexts") session = relationship("DatasetReviewSession", back_populates="run_contexts")
# [/DEF:DatasetRunContext:Class] # [/DEF:DatasetRunContext:Class]
# [DEF:SessionEvent:Class]
class SessionEvent(Base):
__tablename__ = "session_events"
session_event_id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
session_id = Column(String, ForeignKey("dataset_review_sessions.session_id"), nullable=False)
actor_user_id = Column(String, ForeignKey("users.id"), nullable=False)
event_type = Column(String, nullable=False)
event_summary = Column(Text, nullable=False)
current_phase = Column(String, nullable=True)
readiness_state = Column(String, nullable=True)
event_details = Column(JSON, nullable=False, default=dict)
created_at = Column(DateTime, default=datetime.utcnow, nullable=False)
session = relationship("DatasetReviewSession", back_populates="events")
actor = relationship("User")
# [/DEF:SessionEvent:Class]
# [DEF:ArtifactType:Class] # [DEF:ArtifactType:Class]
class ArtifactType(str, enum.Enum): class ArtifactType(str, enum.Enum):
DOCUMENTATION = "documentation" DOCUMENTATION = "documentation"

View File

@@ -145,6 +145,7 @@ class SemanticFieldEntryDto(BaseModel):
display_format: Optional[str] = None display_format: Optional[str] = None
provenance: FieldProvenance provenance: FieldProvenance
source_id: Optional[str] = None source_id: Optional[str] = None
source_version: Optional[str] = None
confidence_rank: Optional[int] = None confidence_rank: Optional[int] = None
is_locked: bool is_locked: bool
has_conflict: bool has_conflict: bool
@@ -239,6 +240,7 @@ class ClarificationAnswerDto(BaseModel):
answer_value: Optional[str] = None answer_value: Optional[str] = None
answered_by_user_id: str answered_by_user_id: str
impact_summary: Optional[str] = None impact_summary: Optional[str] = None
user_feedback: Optional[str] = None
created_at: datetime created_at: datetime
class Config: class Config:

View File

@@ -0,0 +1,552 @@
# [DEF:ClarificationEngine:Module]
# @COMPLEXITY: 4
# @SEMANTICS: dataset_review, clarification, question_payload, answer_persistence, readiness, findings
# @PURPOSE: Manage one-question-at-a-time clarification state, deterministic answer persistence, and readiness/finding updates.
# @LAYER: Domain
# @RELATION: [DEPENDS_ON] ->[DatasetReviewSessionRepository]
# @RELATION: [DEPENDS_ON] ->[ClarificationSession]
# @RELATION: [DEPENDS_ON] ->[ClarificationQuestion]
# @RELATION: [DEPENDS_ON] ->[ClarificationAnswer]
# @RELATION: [DEPENDS_ON] ->[ValidationFinding]
# @PRE: Target session contains a persisted clarification aggregate in the current ownership scope.
# @POST: Active clarification payload exposes one highest-priority unresolved question, and each recorded answer is persisted before pointer/readiness mutation.
# @SIDE_EFFECT: Persists clarification answers, question/session states, and related readiness/finding changes.
# @DATA_CONTRACT: Input[DatasetReviewSession|ClarificationAnswerCommand] -> Output[ClarificationStateResult]
# @INVARIANT: Only one active clarification question may exist at a time; skipped and expert-review items remain unresolved and visible.
from __future__ import annotations
# [DEF:ClarificationEngine.imports:Block]
import uuid
from dataclasses import dataclass, field
from datetime import datetime
from typing import List, Optional
from src.core.logger import belief_scope, logger
from src.models.auth import User
from src.models.dataset_review import (
AnswerKind,
ClarificationAnswer,
ClarificationQuestion,
ClarificationSession,
ClarificationStatus,
DatasetReviewSession,
FindingArea,
FindingSeverity,
QuestionState,
ReadinessState,
RecommendedAction,
ResolutionState,
SessionPhase,
ValidationFinding,
)
from src.services.dataset_review.repositories.session_repository import (
DatasetReviewSessionRepository,
)
# [/DEF:ClarificationEngine.imports:Block]
# [DEF:ClarificationQuestionPayload:Class]
# @COMPLEXITY: 2
# @PURPOSE: Typed active-question payload returned to the API layer.
@dataclass
class ClarificationQuestionPayload:
question_id: str
clarification_session_id: str
topic_ref: str
question_text: str
why_it_matters: str
current_guess: Optional[str]
priority: int
state: QuestionState
options: list[dict[str, object]] = field(default_factory=list)
# [/DEF:ClarificationQuestionPayload:Class]
# [DEF:ClarificationStateResult:Class]
# @COMPLEXITY: 2
# @PURPOSE: Clarification state result carrying the current session, active payload, and changed findings.
@dataclass
class ClarificationStateResult:
clarification_session: ClarificationSession
current_question: Optional[ClarificationQuestionPayload]
session: DatasetReviewSession
changed_findings: List[ValidationFinding] = field(default_factory=list)
# [/DEF:ClarificationStateResult:Class]
# [DEF:ClarificationAnswerCommand:Class]
# @COMPLEXITY: 2
# @PURPOSE: Typed answer command for clarification state mutation.
@dataclass
class ClarificationAnswerCommand:
session: DatasetReviewSession
question_id: str
answer_kind: AnswerKind
answer_value: Optional[str]
user: User
# [/DEF:ClarificationAnswerCommand:Class]
# [DEF:ClarificationEngine:Class]
# @COMPLEXITY: 4
# @PURPOSE: Provide deterministic one-question-at-a-time clarification selection and answer persistence.
# @RELATION: [DEPENDS_ON] ->[DatasetReviewSessionRepository]
# @RELATION: [DEPENDS_ON] ->[ClarificationSession]
# @RELATION: [DEPENDS_ON] ->[ValidationFinding]
# @PRE: Repository is bound to the current request transaction scope.
# @POST: Returned clarification state is persistence-backed and aligned with session readiness/recommended action.
# @SIDE_EFFECT: Mutates clarification answers, session flags, and related clarification findings.
class ClarificationEngine:
# [DEF:ClarificationEngine.__init__:Function]
# @COMPLEXITY: 2
# @PURPOSE: Bind repository dependency for clarification persistence operations.
def __init__(self, repository: DatasetReviewSessionRepository) -> None:
self.repository = repository
# [/DEF:ClarificationEngine.__init__:Function]
# [DEF:ClarificationEngine.build_question_payload:Function]
# @COMPLEXITY: 4
# @PURPOSE: Return the one active highest-priority clarification question payload with why-it-matters, current guess, and options.
# @RELATION: [DEPENDS_ON] ->[ClarificationQuestion]
# @RELATION: [DEPENDS_ON] ->[ClarificationOption]
# @PRE: Session contains unresolved clarification state or a resumable clarification session.
# @POST: Returns exactly one active/open question payload or None when no unresolved question remains.
# @SIDE_EFFECT: Normalizes the active-question pointer and clarification status in persistence.
# @DATA_CONTRACT: Input[DatasetReviewSession] -> Output[ClarificationQuestionPayload|None]
def build_question_payload(
self,
session: DatasetReviewSession,
) -> Optional[ClarificationQuestionPayload]:
with belief_scope("ClarificationEngine.build_question_payload"):
clarification_session = self._get_latest_clarification_session(session)
if clarification_session is None:
logger.reason(
"Clarification payload requested without clarification session",
extra={"session_id": session.session_id},
)
return None
active_questions = [
question for question in clarification_session.questions
if question.state == QuestionState.OPEN
]
active_questions.sort(key=lambda item: (-int(item.priority), item.created_at, item.question_id))
if not active_questions:
clarification_session.current_question_id = None
clarification_session.status = ClarificationStatus.COMPLETED
session.readiness_state = self._derive_readiness_state(session)
session.recommended_action = self._derive_recommended_action(session)
if session.current_phase == SessionPhase.CLARIFICATION:
session.current_phase = SessionPhase.REVIEW
self.repository.db.commit()
logger.reflect(
"No unresolved clarification question remains",
extra={"session_id": session.session_id},
)
return None
selected_question = active_questions[0]
clarification_session.current_question_id = selected_question.question_id
clarification_session.status = ClarificationStatus.ACTIVE
session.readiness_state = ReadinessState.CLARIFICATION_ACTIVE
session.recommended_action = RecommendedAction.ANSWER_NEXT_QUESTION
session.current_phase = SessionPhase.CLARIFICATION
logger.reason(
"Selected active clarification question",
extra={
"session_id": session.session_id,
"clarification_session_id": clarification_session.clarification_session_id,
"question_id": selected_question.question_id,
"priority": selected_question.priority,
},
)
self.repository.db.commit()
payload = ClarificationQuestionPayload(
question_id=selected_question.question_id,
clarification_session_id=selected_question.clarification_session_id,
topic_ref=selected_question.topic_ref,
question_text=selected_question.question_text,
why_it_matters=selected_question.why_it_matters,
current_guess=selected_question.current_guess,
priority=selected_question.priority,
state=selected_question.state,
options=[
{
"option_id": option.option_id,
"question_id": option.question_id,
"label": option.label,
"value": option.value,
"is_recommended": option.is_recommended,
"display_order": option.display_order,
}
for option in sorted(
selected_question.options,
key=lambda item: (item.display_order, item.label, item.option_id),
)
],
)
logger.reflect(
"Clarification payload built",
extra={
"session_id": session.session_id,
"question_id": payload.question_id,
"option_count": len(payload.options),
},
)
return payload
# [/DEF:ClarificationEngine.build_question_payload:Function]
# [DEF:ClarificationEngine.record_answer:Function]
# @COMPLEXITY: 4
# @PURPOSE: Persist one clarification answer before any pointer/readiness mutation and compute deterministic state impact.
# @RELATION: [DEPENDS_ON] ->[ClarificationAnswer]
# @RELATION: [DEPENDS_ON] ->[ValidationFinding]
# @PRE: Target question belongs to the session's active clarification session and is still open.
# @POST: Answer row is persisted before current-question pointer advances; skipped/expert-review items remain unresolved and visible.
# @SIDE_EFFECT: Inserts answer row, mutates question/session states, updates clarification findings, and commits.
# @DATA_CONTRACT: Input[ClarificationAnswerCommand] -> Output[ClarificationStateResult]
def record_answer(self, command: ClarificationAnswerCommand) -> ClarificationStateResult:
with belief_scope("ClarificationEngine.record_answer"):
session = command.session
clarification_session = self._get_latest_clarification_session(session)
if clarification_session is None:
logger.explore(
"Cannot record clarification answer because no clarification session exists",
extra={"session_id": session.session_id},
)
raise ValueError("Clarification session not found")
question = self._find_question(clarification_session, command.question_id)
if question is None:
logger.explore(
"Cannot record clarification answer for foreign or missing question",
extra={"session_id": session.session_id, "question_id": command.question_id},
)
raise ValueError("Clarification question not found")
if question.answer is not None:
logger.explore(
"Rejected duplicate clarification answer submission",
extra={"session_id": session.session_id, "question_id": command.question_id},
)
raise ValueError("Clarification question already answered")
if clarification_session.current_question_id and clarification_session.current_question_id != question.question_id:
logger.explore(
"Rejected answer for non-active clarification question",
extra={
"session_id": session.session_id,
"question_id": question.question_id,
"current_question_id": clarification_session.current_question_id,
},
)
raise ValueError("Only the active clarification question can be answered")
normalized_answer_value = self._normalize_answer_value(command.answer_kind, command.answer_value, question)
logger.reason(
"Persisting clarification answer before state advancement",
extra={
"session_id": session.session_id,
"question_id": question.question_id,
"answer_kind": command.answer_kind.value,
},
)
persisted_answer = ClarificationAnswer(
question_id=question.question_id,
answer_kind=command.answer_kind,
answer_value=normalized_answer_value,
answered_by_user_id=command.user.id,
impact_summary=self._build_impact_summary(question, command.answer_kind, normalized_answer_value),
)
self.repository.db.add(persisted_answer)
self.repository.db.flush()
changed_finding = self._upsert_clarification_finding(
session=session,
question=question,
answer_kind=command.answer_kind,
answer_value=normalized_answer_value,
)
if command.answer_kind == AnswerKind.SELECTED:
question.state = QuestionState.ANSWERED
elif command.answer_kind == AnswerKind.CUSTOM:
question.state = QuestionState.ANSWERED
elif command.answer_kind == AnswerKind.SKIPPED:
question.state = QuestionState.SKIPPED
elif command.answer_kind == AnswerKind.EXPERT_REVIEW:
question.state = QuestionState.EXPERT_REVIEW
question.updated_at = datetime.utcnow()
self.repository.db.flush()
clarification_session.resolved_count = self._count_resolved_questions(clarification_session)
clarification_session.remaining_count = self._count_remaining_questions(clarification_session)
clarification_session.summary_delta = self.summarize_progress(clarification_session)
clarification_session.updated_at = datetime.utcnow()
next_question = self._select_next_open_question(clarification_session)
clarification_session.current_question_id = next_question.question_id if next_question else None
clarification_session.status = (
ClarificationStatus.ACTIVE if next_question else ClarificationStatus.COMPLETED
)
if clarification_session.status == ClarificationStatus.COMPLETED:
clarification_session.completed_at = datetime.utcnow()
session.readiness_state = self._derive_readiness_state(session)
session.recommended_action = self._derive_recommended_action(session)
session.current_phase = (
SessionPhase.CLARIFICATION
if clarification_session.current_question_id
else SessionPhase.REVIEW
)
session.last_activity_at = datetime.utcnow()
self.repository.db.commit()
self.repository.db.refresh(session)
logger.reflect(
"Clarification answer recorded and session advanced",
extra={
"session_id": session.session_id,
"question_id": question.question_id,
"next_question_id": clarification_session.current_question_id,
"readiness_state": session.readiness_state.value,
"remaining_count": clarification_session.remaining_count,
},
)
return ClarificationStateResult(
clarification_session=clarification_session,
current_question=self.build_question_payload(session),
session=session,
changed_findings=[changed_finding] if changed_finding else [],
)
# [/DEF:ClarificationEngine.record_answer:Function]
# [DEF:ClarificationEngine.summarize_progress:Function]
# @COMPLEXITY: 3
# @PURPOSE: Produce a compact progress summary for pause/resume and completion UX.
# @RELATION: [DEPENDS_ON] ->[ClarificationSession]
def summarize_progress(self, clarification_session: ClarificationSession) -> str:
resolved = self._count_resolved_questions(clarification_session)
remaining = self._count_remaining_questions(clarification_session)
return f"{resolved} resolved, {remaining} unresolved"
# [/DEF:ClarificationEngine.summarize_progress:Function]
# [DEF:ClarificationEngine._get_latest_clarification_session:Function]
# @COMPLEXITY: 2
# @PURPOSE: Select the latest clarification session for the current dataset review aggregate.
def _get_latest_clarification_session(
self,
session: DatasetReviewSession,
) -> Optional[ClarificationSession]:
if not session.clarification_sessions:
return None
ordered_sessions = sorted(
session.clarification_sessions,
key=lambda item: (item.started_at, item.clarification_session_id),
reverse=True,
)
return ordered_sessions[0]
# [/DEF:ClarificationEngine._get_latest_clarification_session:Function]
# [DEF:ClarificationEngine._find_question:Function]
# @COMPLEXITY: 1
# @PURPOSE: Resolve a clarification question from the active clarification aggregate.
def _find_question(
self,
clarification_session: ClarificationSession,
question_id: str,
) -> Optional[ClarificationQuestion]:
for question in clarification_session.questions:
if question.question_id == question_id:
return question
return None
# [/DEF:ClarificationEngine._find_question:Function]
# [DEF:ClarificationEngine._select_next_open_question:Function]
# @COMPLEXITY: 2
# @PURPOSE: Select the next unresolved question in deterministic priority order.
def _select_next_open_question(
self,
clarification_session: ClarificationSession,
) -> Optional[ClarificationQuestion]:
open_questions = [
question for question in clarification_session.questions
if question.state == QuestionState.OPEN
]
if not open_questions:
return None
open_questions.sort(key=lambda item: (-int(item.priority), item.created_at, item.question_id))
return open_questions[0]
# [/DEF:ClarificationEngine._select_next_open_question:Function]
# [DEF:ClarificationEngine._count_resolved_questions:Function]
# @COMPLEXITY: 1
# @PURPOSE: Count questions whose answers fully resolved the ambiguity.
def _count_resolved_questions(self, clarification_session: ClarificationSession) -> int:
return sum(
1
for question in clarification_session.questions
if question.state == QuestionState.ANSWERED
)
# [/DEF:ClarificationEngine._count_resolved_questions:Function]
# [DEF:ClarificationEngine._count_remaining_questions:Function]
# @COMPLEXITY: 1
# @PURPOSE: Count questions still unresolved or deferred after clarification interaction.
def _count_remaining_questions(self, clarification_session: ClarificationSession) -> int:
return sum(
1
for question in clarification_session.questions
if question.state in {QuestionState.OPEN, QuestionState.SKIPPED, QuestionState.EXPERT_REVIEW}
)
# [/DEF:ClarificationEngine._count_remaining_questions:Function]
# [DEF:ClarificationEngine._normalize_answer_value:Function]
# @COMPLEXITY: 2
# @PURPOSE: Validate and normalize answer payload based on answer kind and active question options.
def _normalize_answer_value(
self,
answer_kind: AnswerKind,
answer_value: Optional[str],
question: ClarificationQuestion,
) -> Optional[str]:
normalized_answer_value = str(answer_value).strip() if answer_value is not None else None
if answer_kind in {AnswerKind.SELECTED, AnswerKind.CUSTOM} and not normalized_answer_value:
raise ValueError("answer_value is required for selected or custom clarification answers")
if answer_kind == AnswerKind.SELECTED:
allowed_values = {option.value for option in question.options}
if normalized_answer_value not in allowed_values:
raise ValueError("answer_value must match one of the current clarification options")
if answer_kind == AnswerKind.SKIPPED:
return normalized_answer_value or "skipped"
if answer_kind == AnswerKind.EXPERT_REVIEW:
return normalized_answer_value or "expert_review"
return normalized_answer_value
# [/DEF:ClarificationEngine._normalize_answer_value:Function]
# [DEF:ClarificationEngine._build_impact_summary:Function]
# @COMPLEXITY: 2
# @PURPOSE: Build a compact audit note describing how the clarification answer affects session state.
def _build_impact_summary(
self,
question: ClarificationQuestion,
answer_kind: AnswerKind,
answer_value: Optional[str],
) -> str:
if answer_kind == AnswerKind.SKIPPED:
return f"Clarification for {question.topic_ref} was skipped and remains unresolved."
if answer_kind == AnswerKind.EXPERT_REVIEW:
return f"Clarification for {question.topic_ref} was deferred for expert review."
return f"Clarification for {question.topic_ref} recorded as '{answer_value}'."
# [/DEF:ClarificationEngine._build_impact_summary:Function]
# [DEF:ClarificationEngine._upsert_clarification_finding:Function]
# @COMPLEXITY: 3
# @PURPOSE: Keep one finding per clarification topic aligned with answer outcome and unresolved visibility rules.
# @RELATION: [DEPENDS_ON] ->[ValidationFinding]
def _upsert_clarification_finding(
self,
session: DatasetReviewSession,
question: ClarificationQuestion,
answer_kind: AnswerKind,
answer_value: Optional[str],
) -> ValidationFinding:
caused_by_ref = f"clarification:{question.question_id}"
existing = next(
(
finding for finding in session.findings
if finding.area == FindingArea.CLARIFICATION and finding.caused_by_ref == caused_by_ref
),
None,
)
if answer_kind in {AnswerKind.SELECTED, AnswerKind.CUSTOM}:
resolution_state = ResolutionState.RESOLVED
resolved_at = datetime.utcnow()
message = f"Clarified '{question.topic_ref}' with answer '{answer_value}'."
elif answer_kind == AnswerKind.SKIPPED:
resolution_state = ResolutionState.SKIPPED
resolved_at = None
message = f"Clarification for '{question.topic_ref}' was skipped and still needs review."
else:
resolution_state = ResolutionState.EXPERT_REVIEW
resolved_at = None
message = f"Clarification for '{question.topic_ref}' requires expert review."
if existing is None:
existing = ValidationFinding(
finding_id=str(uuid.uuid4()),
session_id=session.session_id,
area=FindingArea.CLARIFICATION,
severity=FindingSeverity.WARNING,
code="CLARIFICATION_PENDING",
title="Clarification pending",
message=message,
resolution_state=resolution_state,
resolution_note=None,
caused_by_ref=caused_by_ref,
created_at=datetime.utcnow(),
resolved_at=resolved_at,
)
self.repository.db.add(existing)
session.findings.append(existing)
else:
existing.message = message
existing.resolution_state = resolution_state
existing.resolved_at = resolved_at
if answer_kind in {AnswerKind.SELECTED, AnswerKind.CUSTOM}:
existing.code = "CLARIFICATION_RESOLVED"
existing.title = "Clarification resolved"
elif answer_kind == AnswerKind.SKIPPED:
existing.code = "CLARIFICATION_SKIPPED"
existing.title = "Clarification skipped"
else:
existing.code = "CLARIFICATION_EXPERT_REVIEW"
existing.title = "Clarification requires expert review"
return existing
# [/DEF:ClarificationEngine._upsert_clarification_finding:Function]
# [DEF:ClarificationEngine._derive_readiness_state:Function]
# @COMPLEXITY: 3
# @PURPOSE: Recompute readiness after clarification mutation while preserving unresolved visibility semantics.
# @RELATION: [DEPENDS_ON] ->[ClarificationSession]
# @RELATION: [DEPENDS_ON] ->[DatasetReviewSession]
def _derive_readiness_state(self, session: DatasetReviewSession) -> ReadinessState:
clarification_session = self._get_latest_clarification_session(session)
if clarification_session is None:
return session.readiness_state
if clarification_session.current_question_id:
return ReadinessState.CLARIFICATION_ACTIVE
if clarification_session.remaining_count > 0:
return ReadinessState.CLARIFICATION_NEEDED
return ReadinessState.REVIEW_READY
# [/DEF:ClarificationEngine._derive_readiness_state:Function]
# [DEF:ClarificationEngine._derive_recommended_action:Function]
# @COMPLEXITY: 2
# @PURPOSE: Recompute next-action guidance after clarification mutations.
def _derive_recommended_action(self, session: DatasetReviewSession) -> RecommendedAction:
clarification_session = self._get_latest_clarification_session(session)
if clarification_session is None:
return session.recommended_action
if clarification_session.current_question_id:
return RecommendedAction.ANSWER_NEXT_QUESTION
if clarification_session.remaining_count > 0:
return RecommendedAction.START_CLARIFICATION
return RecommendedAction.REVIEW_DOCUMENTATION
# [/DEF:ClarificationEngine._derive_recommended_action:Function]
# [/DEF:ClarificationEngine:Class]
# [/DEF:ClarificationEngine:Module]

View File

@@ -0,0 +1,156 @@
# [DEF:SessionEventLogger:Module]
# @COMPLEXITY: 4
# @SEMANTICS: dataset_review, audit, session_events, persistence, observability
# @PURPOSE: Persist explicit session mutation events for dataset-review audit trails without weakening ownership or approval invariants.
# @LAYER: Domain
# @RELATION: [DEPENDS_ON] ->[SessionEvent]
# @RELATION: [DEPENDS_ON] ->[DatasetReviewSession]
# @PRE: Caller provides an owned session scope and an authenticated actor identifier for each persisted mutation event.
# @POST: Every logged event is committed as an explicit, queryable audit record with deterministic event metadata.
# @SIDE_EFFECT: Inserts persisted session event rows and emits runtime belief-state logs for audit-sensitive mutations.
from __future__ import annotations
# [DEF:SessionEventLogger.imports:Block]
from dataclasses import dataclass, field
from typing import Any, Dict, Optional
from sqlalchemy.orm import Session
from src.core.logger import belief_scope, logger
from src.models.dataset_review import DatasetReviewSession, SessionEvent
# [/DEF:SessionEventLogger.imports:Block]
# [DEF:SessionEventPayload:Class]
# @COMPLEXITY: 2
# @PURPOSE: Typed input contract for one persisted dataset-review session audit event.
@dataclass(frozen=True)
class SessionEventPayload:
session_id: str
actor_user_id: str
event_type: str
event_summary: str
current_phase: Optional[str] = None
readiness_state: Optional[str] = None
event_details: Dict[str, Any] = field(default_factory=dict)
# [/DEF:SessionEventPayload:Class]
# [DEF:SessionEventLogger:Class]
# @COMPLEXITY: 4
# @PURPOSE: Persist explicit dataset-review session audit events with meaningful runtime reasoning logs.
# @RELATION: [DEPENDS_ON] ->[SessionEvent]
# @RELATION: [DEPENDS_ON] ->[sqlalchemy.orm.Session]
# @PRE: The database session is live and payload identifiers are non-empty.
# @POST: Returns the committed session event row with a stable identifier and stored detail payload.
# @SIDE_EFFECT: Writes one audit row to persistence and emits logger.reason/logger.reflect traces.
class SessionEventLogger:
# [DEF:SessionEventLogger.__init__:Function]
# @COMPLEXITY: 2
# @PURPOSE: Bind a live SQLAlchemy session to the session-event logger.
def __init__(self, db: Session) -> None:
self.db = db
# [/DEF:SessionEventLogger.__init__:Function]
# [DEF:SessionEventLogger.log_event:Function]
# @COMPLEXITY: 4
# @PURPOSE: Persist one explicit session event row for an owned dataset-review mutation.
# @RELATION: [DEPENDS_ON] ->[SessionEvent]
# @PRE: session_id, actor_user_id, event_type, and event_summary are non-empty.
# @POST: Returns the committed SessionEvent record with normalized detail payload.
# @SIDE_EFFECT: Inserts and commits one session_events row.
# @DATA_CONTRACT: Input[SessionEventPayload] -> Output[SessionEvent]
def log_event(self, payload: SessionEventPayload) -> SessionEvent:
with belief_scope("SessionEventLogger.log_event"):
session_id = str(payload.session_id or "").strip()
actor_user_id = str(payload.actor_user_id or "").strip()
event_type = str(payload.event_type or "").strip()
event_summary = str(payload.event_summary or "").strip()
if not session_id:
logger.explore("Session event logging rejected because session_id is empty")
raise ValueError("session_id must be non-empty")
if not actor_user_id:
logger.explore(
"Session event logging rejected because actor_user_id is empty",
extra={"session_id": session_id},
)
raise ValueError("actor_user_id must be non-empty")
if not event_type:
logger.explore(
"Session event logging rejected because event_type is empty",
extra={"session_id": session_id, "actor_user_id": actor_user_id},
)
raise ValueError("event_type must be non-empty")
if not event_summary:
logger.explore(
"Session event logging rejected because event_summary is empty",
extra={"session_id": session_id, "event_type": event_type},
)
raise ValueError("event_summary must be non-empty")
normalized_details = dict(payload.event_details or {})
logger.reason(
"Persisting explicit dataset-review session audit event",
extra={
"session_id": session_id,
"actor_user_id": actor_user_id,
"event_type": event_type,
"current_phase": payload.current_phase,
"readiness_state": payload.readiness_state,
},
)
event = SessionEvent(
session_id=session_id,
actor_user_id=actor_user_id,
event_type=event_type,
event_summary=event_summary,
current_phase=payload.current_phase,
readiness_state=payload.readiness_state,
event_details=normalized_details,
)
self.db.add(event)
self.db.commit()
self.db.refresh(event)
logger.reflect(
"Dataset-review session audit event persisted",
extra={
"session_id": session_id,
"session_event_id": event.session_event_id,
"event_type": event.event_type,
},
)
return event
# [/DEF:SessionEventLogger.log_event:Function]
# [DEF:SessionEventLogger.log_for_session:Function]
# @COMPLEXITY: 3
# @PURPOSE: Convenience wrapper for logging an event directly from a session aggregate root.
# @RELATION: [CALLS] ->[SessionEventLogger.log_event]
def log_for_session(
self,
session: DatasetReviewSession,
*,
actor_user_id: str,
event_type: str,
event_summary: str,
event_details: Optional[Dict[str, Any]] = None,
) -> SessionEvent:
return self.log_event(
SessionEventPayload(
session_id=session.session_id,
actor_user_id=actor_user_id,
event_type=event_type,
event_summary=event_summary,
current_phase=session.current_phase.value if session.current_phase else None,
readiness_state=session.readiness_state.value if session.readiness_state else None,
event_details=dict(event_details or {}),
)
)
# [/DEF:SessionEventLogger.log_for_session:Function]
# [/DEF:SessionEventLogger:Class]
# [/DEF:SessionEventLogger:Module]

View File

@@ -19,23 +19,36 @@ from __future__ import annotations
# [DEF:DatasetReviewOrchestrator.imports:Block] # [DEF:DatasetReviewOrchestrator.imports:Block]
from dataclasses import dataclass, field from dataclasses import dataclass, field
from datetime import datetime
import hashlib
import json
from typing import Any, Dict, List, Optional from typing import Any, Dict, List, Optional
from src.core.config_manager import ConfigManager from src.core.config_manager import ConfigManager
from src.core.logger import belief_scope, logger from src.core.logger import belief_scope, logger
from src.core.task_manager import TaskManager from src.core.task_manager import TaskManager
from src.core.utils.superset_compilation_adapter import (
PreviewCompilationPayload,
SqlLabLaunchPayload,
SupersetCompilationAdapter,
)
from src.core.utils.superset_context_extractor import ( from src.core.utils.superset_context_extractor import (
SupersetContextExtractor, SupersetContextExtractor,
SupersetParsedContext, SupersetParsedContext,
) )
from src.models.auth import User from src.models.auth import User
from src.models.dataset_review import ( from src.models.dataset_review import (
ApprovalState,
BusinessSummarySource, BusinessSummarySource,
CompiledPreview,
ConfidenceState, ConfidenceState,
DatasetProfile, DatasetProfile,
DatasetReviewSession, DatasetReviewSession,
DatasetRunContext,
FindingArea, FindingArea,
FindingSeverity, FindingSeverity,
LaunchStatus,
PreviewStatus,
RecommendedAction, RecommendedAction,
ReadinessState, ReadinessState,
ResolutionState, ResolutionState,
@@ -47,6 +60,7 @@ from src.services.dataset_review.repositories.session_repository import (
DatasetReviewSessionRepository, DatasetReviewSessionRepository,
) )
from src.services.dataset_review.semantic_resolver import SemanticSourceResolver from src.services.dataset_review.semantic_resolver import SemanticSourceResolver
from src.services.dataset_review.event_logger import SessionEventPayload
# [/DEF:DatasetReviewOrchestrator.imports:Block] # [/DEF:DatasetReviewOrchestrator.imports:Block]
@@ -73,6 +87,48 @@ class StartSessionResult:
# [/DEF:StartSessionResult:Class] # [/DEF:StartSessionResult:Class]
# [DEF:PreparePreviewCommand:Class]
# @COMPLEXITY: 2
# @PURPOSE: Typed input contract for compiling one Superset-backed session preview.
@dataclass
class PreparePreviewCommand:
user: User
session_id: str
# [/DEF:PreparePreviewCommand:Class]
# [DEF:PreparePreviewResult:Class]
# @COMPLEXITY: 2
# @PURPOSE: Result contract for one persisted compiled preview attempt.
@dataclass
class PreparePreviewResult:
session: DatasetReviewSession
preview: CompiledPreview
blocked_reasons: List[str] = field(default_factory=list)
# [/DEF:PreparePreviewResult:Class]
# [DEF:LaunchDatasetCommand:Class]
# @COMPLEXITY: 2
# @PURPOSE: Typed input contract for launching one dataset-review session into SQL Lab.
@dataclass
class LaunchDatasetCommand:
user: User
session_id: str
# [/DEF:LaunchDatasetCommand:Class]
# [DEF:LaunchDatasetResult:Class]
# @COMPLEXITY: 2
# @PURPOSE: Launch result carrying immutable run context and any gate blockers surfaced before launch.
@dataclass
class LaunchDatasetResult:
session: DatasetReviewSession
run_context: DatasetRunContext
blocked_reasons: List[str] = field(default_factory=list)
# [/DEF:LaunchDatasetResult:Class]
# [DEF:DatasetReviewOrchestrator:Class] # [DEF:DatasetReviewOrchestrator:Class]
# @COMPLEXITY: 5 # @COMPLEXITY: 5
# @PURPOSE: Coordinate safe session startup while preserving cross-user isolation and explicit partial recovery. # @PURPOSE: Coordinate safe session startup while preserving cross-user isolation and explicit partial recovery.
@@ -197,6 +253,23 @@ class DatasetReviewOrchestrator:
parsed_context=parsed_context, parsed_context=parsed_context,
dataset_ref=dataset_ref, dataset_ref=dataset_ref,
) )
self.repository.event_logger.log_event(
SessionEventPayload(
session_id=persisted_session.session_id,
actor_user_id=command.user.id,
event_type="session_started",
event_summary="Dataset review session shell created",
current_phase=persisted_session.current_phase.value,
readiness_state=persisted_session.readiness_state.value,
event_details={
"source_kind": persisted_session.source_kind,
"dataset_ref": persisted_session.dataset_ref,
"dataset_id": persisted_session.dataset_id,
"dashboard_id": persisted_session.dashboard_id,
"partial_recovery": bool(parsed_context and parsed_context.partial_recovery),
},
)
)
persisted_session = self.repository.save_profile_and_findings( persisted_session = self.repository.save_profile_and_findings(
persisted_session.session_id, persisted_session.session_id,
command.user.id, command.user.id,
@@ -213,6 +286,17 @@ class DatasetReviewOrchestrator:
persisted_session.active_task_id = active_task_id persisted_session.active_task_id = active_task_id
self.repository.db.commit() self.repository.db.commit()
self.repository.db.refresh(persisted_session) self.repository.db.refresh(persisted_session)
self.repository.event_logger.log_event(
SessionEventPayload(
session_id=persisted_session.session_id,
actor_user_id=command.user.id,
event_type="recovery_task_linked",
event_summary="Recovery task linked to dataset review session",
current_phase=persisted_session.current_phase.value,
readiness_state=persisted_session.readiness_state.value,
event_details={"task_id": active_task_id},
)
)
logger.reason( logger.reason(
"Linked recovery task to started dataset review session", "Linked recovery task to started dataset review session",
extra={"session_id": persisted_session.session_id, "task_id": active_task_id}, extra={"session_id": persisted_session.session_id, "task_id": active_task_id},
@@ -237,6 +321,238 @@ class DatasetReviewOrchestrator:
) )
# [/DEF:DatasetReviewOrchestrator.start_session:Function] # [/DEF:DatasetReviewOrchestrator.start_session:Function]
# [DEF:DatasetReviewOrchestrator.prepare_launch_preview:Function]
# @COMPLEXITY: 4
# @PURPOSE: Assemble effective execution inputs and trigger Superset-side preview compilation.
# @RELATION: [CALLS] ->[SupersetCompilationAdapter.compile_preview]
# @PRE: all required variables have candidate values or explicitly accepted defaults.
# @POST: returns preview artifact in pending, ready, failed, or stale state.
# @SIDE_EFFECT: persists preview attempt and upstream compilation diagnostics.
# @DATA_CONTRACT: Input[PreparePreviewCommand] -> Output[PreparePreviewResult]
def prepare_launch_preview(self, command: PreparePreviewCommand) -> PreparePreviewResult:
with belief_scope("DatasetReviewOrchestrator.prepare_launch_preview"):
session = self.repository.load_session_detail(command.session_id, command.user.id)
if session is None or session.user_id != command.user.id:
logger.explore(
"Preview preparation rejected because owned session was not found",
extra={"session_id": command.session_id, "user_id": command.user.id},
)
raise ValueError("Session not found")
if session.dataset_id is None:
raise ValueError("Preview requires a resolved dataset_id")
environment = self.config_manager.get_environment(session.environment_id)
if environment is None:
raise ValueError("Environment not found")
execution_snapshot = self._build_execution_snapshot(session)
preview_blockers = execution_snapshot["preview_blockers"]
if preview_blockers:
logger.explore(
"Preview preparation blocked by incomplete execution context",
extra={
"session_id": session.session_id,
"blocked_reasons": preview_blockers,
},
)
raise ValueError("Preview blocked: " + "; ".join(preview_blockers))
adapter = SupersetCompilationAdapter(environment)
preview = adapter.compile_preview(
PreviewCompilationPayload(
session_id=session.session_id,
dataset_id=session.dataset_id,
preview_fingerprint=execution_snapshot["preview_fingerprint"],
template_params=execution_snapshot["template_params"],
effective_filters=execution_snapshot["effective_filters"],
)
)
persisted_preview = self.repository.save_preview(
session.session_id,
command.user.id,
preview,
)
session.current_phase = SessionPhase.PREVIEW
session.last_activity_at = datetime.utcnow()
if persisted_preview.preview_status == PreviewStatus.READY:
launch_blockers = self._build_launch_blockers(
session=session,
execution_snapshot=execution_snapshot,
preview=persisted_preview,
)
if launch_blockers:
session.readiness_state = ReadinessState.COMPILED_PREVIEW_READY
session.recommended_action = RecommendedAction.APPROVE_MAPPING
else:
session.readiness_state = ReadinessState.RUN_READY
session.recommended_action = RecommendedAction.LAUNCH_DATASET
else:
session.readiness_state = ReadinessState.PARTIALLY_READY
session.recommended_action = RecommendedAction.GENERATE_SQL_PREVIEW
self.repository.db.commit()
self.repository.db.refresh(session)
self.repository.event_logger.log_event(
SessionEventPayload(
session_id=session.session_id,
actor_user_id=command.user.id,
event_type="preview_generated",
event_summary="Superset preview generation persisted",
current_phase=session.current_phase.value,
readiness_state=session.readiness_state.value,
event_details={
"preview_id": persisted_preview.preview_id,
"preview_status": persisted_preview.preview_status.value,
"preview_fingerprint": persisted_preview.preview_fingerprint,
},
)
)
logger.reflect(
"Superset preview preparation completed",
extra={
"session_id": session.session_id,
"preview_id": persisted_preview.preview_id,
"preview_status": persisted_preview.preview_status.value,
"preview_fingerprint": persisted_preview.preview_fingerprint,
},
)
return PreparePreviewResult(
session=session,
preview=persisted_preview,
blocked_reasons=[],
)
# [/DEF:DatasetReviewOrchestrator.prepare_launch_preview:Function]
# [DEF:DatasetReviewOrchestrator.launch_dataset:Function]
# @COMPLEXITY: 5
# @PURPOSE: Start the approved dataset execution through SQL Lab and persist run context for audit/replay.
# @RELATION: [CALLS] ->[SupersetCompilationAdapter.create_sql_lab_session]
# @PRE: session is run-ready and compiled preview is current.
# @POST: returns persisted run context with SQL Lab session reference and launch outcome.
# @SIDE_EFFECT: creates SQL Lab execution session and audit snapshot.
# @DATA_CONTRACT: Input[LaunchDatasetCommand] -> Output[LaunchDatasetResult]
# @INVARIANT: launch remains blocked unless blocking findings are closed, approvals are satisfied, and the latest Superset preview fingerprint matches current execution inputs.
def launch_dataset(self, command: LaunchDatasetCommand) -> LaunchDatasetResult:
with belief_scope("DatasetReviewOrchestrator.launch_dataset"):
session = self.repository.load_session_detail(command.session_id, command.user.id)
if session is None or session.user_id != command.user.id:
logger.explore(
"Launch rejected because owned session was not found",
extra={"session_id": command.session_id, "user_id": command.user.id},
)
raise ValueError("Session not found")
if session.dataset_id is None:
raise ValueError("Launch requires a resolved dataset_id")
environment = self.config_manager.get_environment(session.environment_id)
if environment is None:
raise ValueError("Environment not found")
execution_snapshot = self._build_execution_snapshot(session)
current_preview = self._get_latest_preview(session)
launch_blockers = self._build_launch_blockers(
session=session,
execution_snapshot=execution_snapshot,
preview=current_preview,
)
if launch_blockers:
logger.explore(
"Launch gate blocked dataset execution",
extra={
"session_id": session.session_id,
"blocked_reasons": launch_blockers,
},
)
raise ValueError("Launch blocked: " + "; ".join(launch_blockers))
adapter = SupersetCompilationAdapter(environment)
try:
sql_lab_session_ref = adapter.create_sql_lab_session(
SqlLabLaunchPayload(
session_id=session.session_id,
dataset_id=session.dataset_id,
preview_id=current_preview.preview_id,
compiled_sql=str(current_preview.compiled_sql or ""),
template_params=execution_snapshot["template_params"],
)
)
launch_status = LaunchStatus.STARTED
launch_error = None
except Exception as exc:
logger.explore(
"SQL Lab launch failed after passing gates",
extra={"session_id": session.session_id, "error": str(exc)},
)
sql_lab_session_ref = "unavailable"
launch_status = LaunchStatus.FAILED
launch_error = str(exc)
run_context = DatasetRunContext(
session_id=session.session_id,
dataset_ref=session.dataset_ref,
environment_id=session.environment_id,
preview_id=current_preview.preview_id,
sql_lab_session_ref=sql_lab_session_ref,
effective_filters=execution_snapshot["effective_filters"],
template_params=execution_snapshot["template_params"],
approved_mapping_ids=execution_snapshot["approved_mapping_ids"],
semantic_decision_refs=execution_snapshot["semantic_decision_refs"],
open_warning_refs=execution_snapshot["open_warning_refs"],
launch_status=launch_status,
launch_error=launch_error,
)
persisted_run_context = self.repository.save_run_context(
session.session_id,
command.user.id,
run_context,
)
session.current_phase = SessionPhase.LAUNCH
session.last_activity_at = datetime.utcnow()
if launch_status == LaunchStatus.FAILED:
session.readiness_state = ReadinessState.COMPILED_PREVIEW_READY
session.recommended_action = RecommendedAction.LAUNCH_DATASET
else:
session.readiness_state = ReadinessState.RUN_IN_PROGRESS
session.recommended_action = RecommendedAction.EXPORT_OUTPUTS
self.repository.db.commit()
self.repository.db.refresh(session)
self.repository.event_logger.log_event(
SessionEventPayload(
session_id=session.session_id,
actor_user_id=command.user.id,
event_type="dataset_launch_requested",
event_summary="Dataset launch handoff persisted",
current_phase=session.current_phase.value,
readiness_state=session.readiness_state.value,
event_details={
"run_context_id": persisted_run_context.run_context_id,
"launch_status": persisted_run_context.launch_status.value,
"preview_id": persisted_run_context.preview_id,
"sql_lab_session_ref": persisted_run_context.sql_lab_session_ref,
},
)
)
logger.reflect(
"Dataset launch orchestration completed with audited run context",
extra={
"session_id": session.session_id,
"run_context_id": persisted_run_context.run_context_id,
"launch_status": persisted_run_context.launch_status.value,
"preview_id": persisted_run_context.preview_id,
},
)
return LaunchDatasetResult(
session=session,
run_context=persisted_run_context,
blocked_reasons=[],
)
# [/DEF:DatasetReviewOrchestrator.launch_dataset:Function]
# [DEF:DatasetReviewOrchestrator._parse_dataset_selection:Function] # [DEF:DatasetReviewOrchestrator._parse_dataset_selection:Function]
# @COMPLEXITY: 3 # @COMPLEXITY: 3
# @PURPOSE: Normalize dataset-selection payload into canonical session references. # @PURPOSE: Normalize dataset-selection payload into canonical session references.
@@ -328,6 +644,158 @@ class DatasetReviewOrchestrator:
return findings return findings
# [/DEF:DatasetReviewOrchestrator._build_partial_recovery_findings:Function] # [/DEF:DatasetReviewOrchestrator._build_partial_recovery_findings:Function]
# [DEF:DatasetReviewOrchestrator._build_execution_snapshot:Function]
# @COMPLEXITY: 4
# @PURPOSE: Build effective filters, template params, approvals, and fingerprint for preview and launch gating.
# @RELATION: [DEPENDS_ON] ->[DatasetReviewSession]
# @PRE: Session aggregate includes imported filters, template variables, and current execution mappings.
# @POST: returns deterministic execution snapshot for current session state without mutating persistence.
# @SIDE_EFFECT: none.
# @DATA_CONTRACT: Input[DatasetReviewSession] -> Output[Dict[str,Any]]
def _build_execution_snapshot(self, session: DatasetReviewSession) -> Dict[str, Any]:
filter_lookup = {item.filter_id: item for item in session.imported_filters}
variable_lookup = {item.variable_id: item for item in session.template_variables}
effective_filters: List[Dict[str, Any]] = []
template_params: Dict[str, Any] = {}
approved_mapping_ids: List[str] = []
open_warning_refs: List[str] = []
preview_blockers: List[str] = []
for mapping in session.execution_mappings:
imported_filter = filter_lookup.get(mapping.filter_id)
template_variable = variable_lookup.get(mapping.variable_id)
if imported_filter is None:
preview_blockers.append(f"mapping:{mapping.mapping_id}:missing_filter")
continue
if template_variable is None:
preview_blockers.append(f"mapping:{mapping.mapping_id}:missing_variable")
continue
effective_value = mapping.effective_value
if effective_value is None:
effective_value = imported_filter.normalized_value
if effective_value is None:
effective_value = imported_filter.raw_value
if effective_value is None:
effective_value = template_variable.default_value
if effective_value is None and template_variable.is_required:
preview_blockers.append(f"variable:{template_variable.variable_name}:missing_required_value")
continue
effective_filters.append(
{
"mapping_id": mapping.mapping_id,
"filter_id": imported_filter.filter_id,
"filter_name": imported_filter.filter_name,
"variable_id": template_variable.variable_id,
"variable_name": template_variable.variable_name,
"effective_value": effective_value,
"raw_input_value": mapping.raw_input_value,
}
)
template_params[template_variable.variable_name] = effective_value
if mapping.approval_state == ApprovalState.APPROVED:
approved_mapping_ids.append(mapping.mapping_id)
if mapping.requires_explicit_approval and mapping.approval_state != ApprovalState.APPROVED:
open_warning_refs.append(mapping.mapping_id)
mapped_variable_ids = {mapping.variable_id for mapping in session.execution_mappings}
for variable in session.template_variables:
if variable.variable_id in mapped_variable_ids:
continue
if variable.default_value is not None:
template_params[variable.variable_name] = variable.default_value
continue
if variable.is_required:
preview_blockers.append(f"variable:{variable.variable_name}:unmapped")
semantic_decision_refs = [
field.field_id
for field in session.semantic_fields
if field.is_locked or not field.needs_review or field.provenance.value != "unresolved"
]
preview_fingerprint = self._compute_preview_fingerprint(
{
"dataset_id": session.dataset_id,
"template_params": template_params,
"effective_filters": effective_filters,
}
)
return {
"effective_filters": effective_filters,
"template_params": template_params,
"approved_mapping_ids": sorted(approved_mapping_ids),
"semantic_decision_refs": sorted(semantic_decision_refs),
"open_warning_refs": sorted(open_warning_refs),
"preview_blockers": sorted(set(preview_blockers)),
"preview_fingerprint": preview_fingerprint,
}
# [/DEF:DatasetReviewOrchestrator._build_execution_snapshot:Function]
# [DEF:DatasetReviewOrchestrator._build_launch_blockers:Function]
# @COMPLEXITY: 4
# @PURPOSE: Enforce launch gates from findings, approvals, and current preview truth.
# @RELATION: [DEPENDS_ON] ->[CompiledPreview]
# @PRE: execution_snapshot was computed from current session state and preview is the latest persisted preview or None.
# @POST: returns explicit blocker codes for every unmet launch invariant.
# @SIDE_EFFECT: none.
# @DATA_CONTRACT: Input[DatasetReviewSession,Dict[str,Any],CompiledPreview|None] -> Output[List[str]]
def _build_launch_blockers(
self,
session: DatasetReviewSession,
execution_snapshot: Dict[str, Any],
preview: Optional[CompiledPreview],
) -> List[str]:
blockers = list(execution_snapshot["preview_blockers"])
for finding in session.findings:
if (
finding.severity == FindingSeverity.BLOCKING
and finding.resolution_state not in {ResolutionState.RESOLVED, ResolutionState.APPROVED}
):
blockers.append(f"finding:{finding.code}:blocking")
for mapping in session.execution_mappings:
if mapping.requires_explicit_approval and mapping.approval_state != ApprovalState.APPROVED:
blockers.append(f"mapping:{mapping.mapping_id}:approval_required")
if preview is None:
blockers.append("preview:missing")
else:
if preview.preview_status != PreviewStatus.READY:
blockers.append(f"preview:{preview.preview_status.value}")
if preview.preview_fingerprint != execution_snapshot["preview_fingerprint"]:
blockers.append("preview:fingerprint_mismatch")
return sorted(set(blockers))
# [/DEF:DatasetReviewOrchestrator._build_launch_blockers:Function]
# [DEF:DatasetReviewOrchestrator._get_latest_preview:Function]
# @COMPLEXITY: 2
# @PURPOSE: Resolve the current latest preview snapshot for one session aggregate.
def _get_latest_preview(self, session: DatasetReviewSession) -> Optional[CompiledPreview]:
if not session.previews:
return None
if session.last_preview_id:
for preview in session.previews:
if preview.preview_id == session.last_preview_id:
return preview
return sorted(
session.previews,
key=lambda item: (item.created_at or datetime.min, item.preview_id),
reverse=True,
)[0]
# [/DEF:DatasetReviewOrchestrator._get_latest_preview:Function]
# [DEF:DatasetReviewOrchestrator._compute_preview_fingerprint:Function]
# @COMPLEXITY: 2
# @PURPOSE: Produce deterministic execution fingerprint for preview truth and staleness checks.
def _compute_preview_fingerprint(self, payload: Dict[str, Any]) -> str:
serialized = json.dumps(payload, sort_keys=True, default=str)
return hashlib.sha256(serialized.encode("utf-8")).hexdigest()
# [/DEF:DatasetReviewOrchestrator._compute_preview_fingerprint:Function]
# [DEF:DatasetReviewOrchestrator._enqueue_recovery_task:Function] # [DEF:DatasetReviewOrchestrator._enqueue_recovery_task:Function]
# @COMPLEXITY: 4 # @COMPLEXITY: 4
# @PURPOSE: Link session start to observable async recovery when task infrastructure is available. # @PURPOSE: Link session start to observable async recovery when task infrastructure is available.

View File

@@ -16,14 +16,19 @@ from typing import Optional, List
from sqlalchemy import or_ from sqlalchemy import or_
from sqlalchemy.orm import Session, joinedload from sqlalchemy.orm import Session, joinedload
from src.models.dataset_review import ( from src.models.dataset_review import (
ClarificationQuestion,
ClarificationSession,
DatasetReviewSession, DatasetReviewSession,
DatasetProfile, DatasetProfile,
ValidationFinding, ValidationFinding,
CompiledPreview, CompiledPreview,
DatasetRunContext, DatasetRunContext,
SessionCollaborator SemanticFieldEntry,
SessionCollaborator,
SessionEvent,
) )
from src.core.logger import belief_scope from src.core.logger import belief_scope, logger
from src.services.dataset_review.event_logger import SessionEventLogger
# [DEF:SessionRepo:Class] # [DEF:SessionRepo:Class]
# @COMPLEXITY: 4 # @COMPLEXITY: 4
@@ -37,16 +42,46 @@ from src.core.logger import belief_scope
# @SIDE_EFFECT: mutates and queries the persistence layer through the injected database session. # @SIDE_EFFECT: mutates and queries the persistence layer through the injected database session.
# @DATA_CONTRACT: Input[OwnedSessionQuery|SessionMutation] -> Output[PersistedSessionAggregate|PersistedChildRecord] # @DATA_CONTRACT: Input[OwnedSessionQuery|SessionMutation] -> Output[PersistedSessionAggregate|PersistedChildRecord]
class DatasetReviewSessionRepository: class DatasetReviewSessionRepository:
"""
@PURPOSE: Persist and retrieve dataset review session aggregates.
@INVARIANT: ownership_scope -> All operations must respect the session owner's user_id.
"""
# [DEF:init_repo:Function] # [DEF:init_repo:Function]
# @COMPLEXITY: 2
# @PURPOSE: Bind one live SQLAlchemy session to the repository instance.
def __init__(self, db: Session): def __init__(self, db: Session):
self.db = db self.db = db
self.event_logger = SessionEventLogger(db)
# [/DEF:init_repo:Function] # [/DEF:init_repo:Function]
# [DEF:get_owned_session:Function]
# @COMPLEXITY: 4
# @PURPOSE: Resolve one owner-scoped dataset review session for mutation paths without leaking foreign-session state.
# @RELATION: [DEPENDS_ON] -> [DatasetReviewSession]
# @PRE: session_id and user_id are non-empty identifiers from the authenticated ownership scope.
# @POST: returns the owned session or raises a deterministic access error.
# @SIDE_EFFECT: reads one session row from the current database transaction.
# @DATA_CONTRACT: Input[OwnedSessionQuery] -> Output[DatasetReviewSession|ValueError]
def _get_owned_session(self, session_id: str, user_id: str) -> DatasetReviewSession:
with belief_scope("DatasetReviewSessionRepository.get_owned_session"):
logger.reason(
"Resolving owner-scoped dataset review session for mutation path",
extra={"session_id": session_id, "user_id": user_id},
)
session = self.db.query(DatasetReviewSession).filter(
DatasetReviewSession.session_id == session_id,
DatasetReviewSession.user_id == user_id,
).first()
if not session:
logger.explore(
"Owner-scoped dataset review session lookup failed",
extra={"session_id": session_id, "user_id": user_id},
)
raise ValueError("Session not found or access denied")
logger.reflect(
"Owner-scoped dataset review session resolved",
extra={"session_id": session.session_id, "user_id": session.user_id},
)
return session
# [/DEF:get_owned_session:Function]
# [DEF:create_sess:Function] # [DEF:create_sess:Function]
# @COMPLEXITY: 4 # @COMPLEXITY: 4
# @PURPOSE: Persist an initial dataset review session shell. # @PURPOSE: Persist an initial dataset review session shell.
@@ -57,9 +92,17 @@ class DatasetReviewSessionRepository:
# @DATA_CONTRACT: Input[DatasetReviewSession] -> Output[DatasetReviewSession] # @DATA_CONTRACT: Input[DatasetReviewSession] -> Output[DatasetReviewSession]
def create_session(self, session: DatasetReviewSession) -> DatasetReviewSession: def create_session(self, session: DatasetReviewSession) -> DatasetReviewSession:
with belief_scope("DatasetReviewSessionRepository.create_session"): with belief_scope("DatasetReviewSessionRepository.create_session"):
logger.reason(
"Persisting dataset review session shell",
extra={"user_id": session.user_id, "environment_id": session.environment_id},
)
self.db.add(session) self.db.add(session)
self.db.commit() self.db.commit()
self.db.refresh(session) self.db.refresh(session)
logger.reflect(
"Dataset review session shell persisted with stable identifier",
extra={"session_id": session.session_id, "user_id": session.user_id},
)
return session return session
# [/DEF:create_sess:Function] # [/DEF:create_sess:Function]
@@ -69,25 +112,27 @@ class DatasetReviewSessionRepository:
# @RELATION: [DEPENDS_ON] -> [DatasetReviewSession] # @RELATION: [DEPENDS_ON] -> [DatasetReviewSession]
# @RELATION: [DEPENDS_ON] -> [SessionCollaborator] # @RELATION: [DEPENDS_ON] -> [SessionCollaborator]
def load_session_detail(self, session_id: str, user_id: str) -> Optional[DatasetReviewSession]: def load_session_detail(self, session_id: str, user_id: str) -> Optional[DatasetReviewSession]:
"""
@PRE: user_id must match session owner or authorized collaborator.
"""
with belief_scope("DatasetReviewSessionRepository.load_session_detail"): with belief_scope("DatasetReviewSessionRepository.load_session_detail"):
# Check if user is owner or collaborator logger.reason(
return self.db.query(DatasetReviewSession)\ "Loading dataset review session detail for owner-or-collaborator scope",
extra={"session_id": session_id, "user_id": user_id},
)
session = self.db.query(DatasetReviewSession)\
.outerjoin(SessionCollaborator, DatasetReviewSession.session_id == SessionCollaborator.session_id)\ .outerjoin(SessionCollaborator, DatasetReviewSession.session_id == SessionCollaborator.session_id)\
.options( .options(
joinedload(DatasetReviewSession.profile), joinedload(DatasetReviewSession.profile),
joinedload(DatasetReviewSession.findings), joinedload(DatasetReviewSession.findings),
joinedload(DatasetReviewSession.collaborators), joinedload(DatasetReviewSession.collaborators),
joinedload(DatasetReviewSession.semantic_sources), joinedload(DatasetReviewSession.semantic_sources),
joinedload(DatasetReviewSession.semantic_fields), joinedload(DatasetReviewSession.semantic_fields).joinedload(SemanticFieldEntry.candidates),
joinedload(DatasetReviewSession.imported_filters), joinedload(DatasetReviewSession.imported_filters),
joinedload(DatasetReviewSession.template_variables), joinedload(DatasetReviewSession.template_variables),
joinedload(DatasetReviewSession.execution_mappings), joinedload(DatasetReviewSession.execution_mappings),
joinedload(DatasetReviewSession.clarification_sessions), joinedload(DatasetReviewSession.clarification_sessions).joinedload(ClarificationSession.questions).joinedload(ClarificationQuestion.options),
joinedload(DatasetReviewSession.clarification_sessions).joinedload(ClarificationSession.questions).joinedload(ClarificationQuestion.answer),
joinedload(DatasetReviewSession.previews), joinedload(DatasetReviewSession.previews),
joinedload(DatasetReviewSession.run_contexts) joinedload(DatasetReviewSession.run_contexts),
joinedload(DatasetReviewSession.events)
)\ )\
.filter(DatasetReviewSession.session_id == session_id)\ .filter(DatasetReviewSession.session_id == session_id)\
.filter( .filter(
@@ -97,6 +142,15 @@ class DatasetReviewSessionRepository:
) )
)\ )\
.first() .first()
logger.reflect(
"Dataset review session detail lookup completed",
extra={
"session_id": session_id,
"user_id": user_id,
"found": bool(session),
},
)
return session
# [/DEF:load_detail:Function] # [/DEF:load_detail:Function]
# [DEF:save_prof_find:Function] # [DEF:save_prof_find:Function]
@@ -111,32 +165,40 @@ class DatasetReviewSessionRepository:
# @DATA_CONTRACT: Input[ProfileAndFindingsMutation] -> Output[DatasetReviewSession] # @DATA_CONTRACT: Input[ProfileAndFindingsMutation] -> Output[DatasetReviewSession]
def save_profile_and_findings(self, session_id: str, user_id: str, profile: DatasetProfile, findings: List[ValidationFinding]) -> DatasetReviewSession: def save_profile_and_findings(self, session_id: str, user_id: str, profile: DatasetProfile, findings: List[ValidationFinding]) -> DatasetReviewSession:
with belief_scope("DatasetReviewSessionRepository.save_profile_and_findings"): with belief_scope("DatasetReviewSessionRepository.save_profile_and_findings"):
session = self.db.query(DatasetReviewSession).filter( session = self._get_owned_session(session_id, user_id)
DatasetReviewSession.session_id == session_id, logger.reason(
DatasetReviewSession.user_id == user_id "Persisting dataset profile and replacing validation findings",
).first() extra={
"session_id": session_id,
if not session: "user_id": user_id,
raise ValueError("Session not found or access denied") "has_profile": bool(profile),
"findings_count": len(findings),
},
)
if profile: if profile:
# Ensure we update existing profile by session_id if it exists
existing_profile = self.db.query(DatasetProfile).filter_by(session_id=session_id).first() existing_profile = self.db.query(DatasetProfile).filter_by(session_id=session_id).first()
if existing_profile: if existing_profile:
profile.profile_id = existing_profile.profile_id profile.profile_id = existing_profile.profile_id
self.db.merge(profile) self.db.merge(profile)
# Remove old findings for this session to avoid stale data
self.db.query(ValidationFinding).filter( self.db.query(ValidationFinding).filter(
ValidationFinding.session_id == session_id ValidationFinding.session_id == session_id
).delete() ).delete()
# Add new findings
for finding in findings: for finding in findings:
finding.session_id = session_id finding.session_id = session_id
self.db.add(finding) self.db.add(finding)
self.db.commit() self.db.commit()
logger.reflect(
"Dataset profile and validation findings committed",
extra={
"session_id": session.session_id,
"user_id": user_id,
"findings_count": len(findings),
},
)
return self.load_session_detail(session_id, user_id) return self.load_session_detail(session_id, user_id)
# [/DEF:save_prof_find:Function] # [/DEF:save_prof_find:Function]
@@ -151,15 +213,12 @@ class DatasetReviewSessionRepository:
# @DATA_CONTRACT: Input[PreviewMutation] -> Output[CompiledPreview] # @DATA_CONTRACT: Input[PreviewMutation] -> Output[CompiledPreview]
def save_preview(self, session_id: str, user_id: str, preview: CompiledPreview) -> CompiledPreview: def save_preview(self, session_id: str, user_id: str, preview: CompiledPreview) -> CompiledPreview:
with belief_scope("DatasetReviewSessionRepository.save_preview"): with belief_scope("DatasetReviewSessionRepository.save_preview"):
session = self.db.query(DatasetReviewSession).filter( session = self._get_owned_session(session_id, user_id)
DatasetReviewSession.session_id == session_id, logger.reason(
DatasetReviewSession.user_id == user_id "Persisting compiled preview and staling previous preview snapshots",
).first() extra={"session_id": session_id, "user_id": user_id},
)
if not session:
raise ValueError("Session not found or access denied")
# Mark existing previews for this session as stale if they are not the new one
self.db.query(CompiledPreview).filter( self.db.query(CompiledPreview).filter(
CompiledPreview.session_id == session_id CompiledPreview.session_id == session_id
).update({"preview_status": "stale"}) ).update({"preview_status": "stale"})
@@ -170,6 +229,14 @@ class DatasetReviewSessionRepository:
self.db.commit() self.db.commit()
self.db.refresh(preview) self.db.refresh(preview)
logger.reflect(
"Compiled preview committed as latest session preview",
extra={
"session_id": session.session_id,
"preview_id": preview.preview_id,
"user_id": user_id,
},
)
return preview return preview
# [/DEF:save_prev:Function] # [/DEF:save_prev:Function]
@@ -184,13 +251,11 @@ class DatasetReviewSessionRepository:
# @DATA_CONTRACT: Input[RunContextMutation] -> Output[DatasetRunContext] # @DATA_CONTRACT: Input[RunContextMutation] -> Output[DatasetRunContext]
def save_run_context(self, session_id: str, user_id: str, run_context: DatasetRunContext) -> DatasetRunContext: def save_run_context(self, session_id: str, user_id: str, run_context: DatasetRunContext) -> DatasetRunContext:
with belief_scope("DatasetReviewSessionRepository.save_run_context"): with belief_scope("DatasetReviewSessionRepository.save_run_context"):
session = self.db.query(DatasetReviewSession).filter( session = self._get_owned_session(session_id, user_id)
DatasetReviewSession.session_id == session_id, logger.reason(
DatasetReviewSession.user_id == user_id "Persisting dataset run context audit snapshot",
).first() extra={"session_id": session_id, "user_id": user_id},
)
if not session:
raise ValueError("Session not found or access denied")
self.db.add(run_context) self.db.add(run_context)
self.db.flush() self.db.flush()
@@ -198,6 +263,14 @@ class DatasetReviewSessionRepository:
self.db.commit() self.db.commit()
self.db.refresh(run_context) self.db.refresh(run_context)
logger.reflect(
"Dataset run context committed as latest launch snapshot",
extra={
"session_id": session.session_id,
"run_context_id": run_context.run_context_id,
"user_id": user_id,
},
)
return run_context return run_context
# [/DEF:save_run_ctx:Function] # [/DEF:save_run_ctx:Function]
@@ -207,9 +280,18 @@ class DatasetReviewSessionRepository:
# @RELATION: [DEPENDS_ON] -> [DatasetReviewSession] # @RELATION: [DEPENDS_ON] -> [DatasetReviewSession]
def list_sessions_for_user(self, user_id: str) -> List[DatasetReviewSession]: def list_sessions_for_user(self, user_id: str) -> List[DatasetReviewSession]:
with belief_scope("DatasetReviewSessionRepository.list_sessions_for_user"): with belief_scope("DatasetReviewSessionRepository.list_sessions_for_user"):
return self.db.query(DatasetReviewSession).filter( logger.reason(
"Listing dataset review sessions for owner scope",
extra={"user_id": user_id},
)
sessions = self.db.query(DatasetReviewSession).filter(
DatasetReviewSession.user_id == user_id DatasetReviewSession.user_id == user_id
).order_by(DatasetReviewSession.updated_at.desc()).all() ).order_by(DatasetReviewSession.updated_at.desc()).all()
logger.reflect(
"Dataset review session list assembled",
extra={"user_id": user_id, "session_count": len(sessions)},
)
return sessions
# [/DEF:list_user_sess:Function] # [/DEF:list_user_sess:Function]
# [/DEF:SessionRepo:Class] # [/DEF:SessionRepo:Class]

View File

@@ -24,6 +24,7 @@ from src.models.dataset_review import (
CandidateMatchType, CandidateMatchType,
CandidateStatus, CandidateStatus,
FieldProvenance, FieldProvenance,
SemanticSource,
) )
# [/DEF:SemanticSourceResolver.imports:Block] # [/DEF:SemanticSourceResolver.imports:Block]
@@ -259,6 +260,63 @@ class SemanticSourceResolver:
return merged return merged
# [/DEF:SemanticSourceResolver.apply_field_decision:Function] # [/DEF:SemanticSourceResolver.apply_field_decision:Function]
# [DEF:SemanticSourceResolver.propagate_source_version_update:Function]
# @COMPLEXITY: 4
# @PURPOSE: Propagate a semantic source version change to unlocked field entries without silently overwriting manual or locked values.
# @RELATION: [DEPENDS_ON] ->[SemanticSource]
# @RELATION: [DEPENDS_ON] ->[SemanticFieldEntry]
# @PRE: source is persisted and fields belong to the same session aggregate.
# @POST: unlocked fields linked to the source carry the new source version and are marked reviewable; manual or locked fields keep their active values untouched.
# @SIDE_EFFECT: mutates in-memory field state for the caller to persist.
# @DATA_CONTRACT: Input[SemanticSource,List[SemanticFieldEntry]] -> Output[Dict[str,int]]
def propagate_source_version_update(
self,
source: SemanticSource,
fields: Iterable[Any],
) -> Dict[str, int]:
with belief_scope("SemanticSourceResolver.propagate_source_version_update"):
source_id = str(source.source_id or "").strip()
source_version = str(source.source_version or "").strip()
if not source_id or not source_version:
logger.explore(
"Semantic source version propagation rejected due to incomplete source metadata",
extra={"source_id": source_id, "source_version": source_version},
)
raise ValueError("Semantic source must provide source_id and source_version")
propagated = 0
preserved_locked = 0
untouched = 0
for field in fields:
if str(getattr(field, "source_id", "") or "").strip() != source_id:
untouched += 1
continue
if bool(getattr(field, "is_locked", False)) or getattr(field, "provenance", None) == FieldProvenance.MANUAL_OVERRIDE:
preserved_locked += 1
continue
field.source_version = source_version
field.needs_review = True
field.has_conflict = bool(getattr(field, "has_conflict", False))
propagated += 1
logger.reflect(
"Semantic source version propagation completed",
extra={
"source_id": source_id,
"source_version": source_version,
"propagated": propagated,
"preserved_locked": preserved_locked,
"untouched": untouched,
},
)
return {
"propagated": propagated,
"preserved_locked": preserved_locked,
"untouched": untouched,
}
# [/DEF:SemanticSourceResolver.propagate_source_version_update:Function]
# [DEF:SemanticSourceResolver._normalize_dictionary_row:Function] # [DEF:SemanticSourceResolver._normalize_dictionary_row:Function]
# @COMPLEXITY: 2 # @COMPLEXITY: 2
# @PURPOSE: Normalize one dictionary row into a consistent lookup structure. # @PURPOSE: Normalize one dictionary row into a consistent lookup structure.

View File

@@ -0,0 +1,126 @@
# [DEF:SupersetCompatibilityMatrixTests:Module]
# @COMPLEXITY: 3
# @SEMANTICS: dataset_review, superset, compatibility_matrix, preview, sql_lab, tests
# @PURPOSE: Verifies Superset preview and SQL Lab endpoint fallback strategy used by dataset-review orchestration.
# @LAYER: Tests
# @RELATION: [DEPENDS_ON] ->[SupersetCompilationAdapter]
# @RELATION: [DEPENDS_ON] ->[SupersetClient]
from types import SimpleNamespace
from unittest.mock import MagicMock
from src.core.utils.superset_compilation_adapter import (
PreviewCompilationPayload,
SqlLabLaunchPayload,
SupersetCompilationAdapter,
)
# [DEF:make_adapter:Function]
# @COMPLEXITY: 2
# @PURPOSE: Build an adapter with a mock Superset client and deterministic environment for compatibility tests.
def make_adapter():
environment = SimpleNamespace(
id="env-1",
name="Test Env",
url="http://superset.example",
username="user",
password="pass",
verify_ssl=True,
timeout=30,
)
client = MagicMock()
client.network = MagicMock()
return SupersetCompilationAdapter(environment=environment, client=client), client
# [/DEF:make_adapter:Function]
# [DEF:test_preview_prefers_supported_client_method_before_network_fallback:Function]
# @COMPLEXITY: 2
# @PURPOSE: Confirms preview compilation uses a supported client method first when the capability exists.
def test_preview_prefers_supported_client_method_before_network_fallback():
adapter, client = make_adapter()
client.compile_preview = MagicMock(return_value={"compiled_sql": "SELECT 1"})
payload = PreviewCompilationPayload(
session_id="sess-1",
dataset_id=42,
preview_fingerprint="fp-1",
template_params={"country": "RU"},
effective_filters=[{"name": "country", "value": "RU"}],
)
preview = adapter.compile_preview(payload)
assert preview.preview_status.value == "ready"
assert preview.compiled_sql == "SELECT 1"
client.compile_preview.assert_called_once()
client.network.request.assert_not_called()
# [/DEF:test_preview_prefers_supported_client_method_before_network_fallback:Function]
# [DEF:test_preview_falls_back_across_matrix_until_supported_endpoint_returns_sql:Function]
# @COMPLEXITY: 3
# @PURPOSE: Confirms preview fallback walks the compatibility matrix from preferred to legacy endpoints until one returns compiled SQL.
def test_preview_falls_back_across_matrix_until_supported_endpoint_returns_sql():
adapter, client = make_adapter()
payload = PreviewCompilationPayload(
session_id="sess-2",
dataset_id=77,
preview_fingerprint="fp-2",
template_params={"region": "emea"},
effective_filters=[],
)
client.network.request.side_effect = [
RuntimeError("preview endpoint unavailable"),
{"result": {"sql": "SELECT * FROM dataset_77"}},
]
preview = adapter.compile_preview(payload)
assert preview.preview_status.value == "ready"
assert preview.compiled_sql == "SELECT * FROM dataset_77"
assert client.network.request.call_count == 2
first_call = client.network.request.call_args_list[0].kwargs
second_call = client.network.request.call_args_list[1].kwargs
assert first_call["endpoint"] == "/dataset/77/preview"
assert second_call["endpoint"] == "/dataset/77/sql"
# [/DEF:test_preview_falls_back_across_matrix_until_supported_endpoint_returns_sql:Function]
# [DEF:test_sql_lab_launch_falls_back_to_legacy_execute_endpoint:Function]
# @COMPLEXITY: 3
# @PURPOSE: Confirms SQL Lab launch falls back from modern to legacy execute endpoint and preserves canonical session reference extraction.
def test_sql_lab_launch_falls_back_to_legacy_execute_endpoint():
adapter, client = make_adapter()
client.get_dataset.return_value = {
"result": {
"id": 55,
"schema": "public",
"database": {"id": 9},
}
}
client.network.request.side_effect = [
RuntimeError("sqllab execute unavailable"),
{"result": {"id": "query-123"}},
]
payload = SqlLabLaunchPayload(
session_id="sess-3",
dataset_id=55,
preview_id="preview-9",
compiled_sql="SELECT * FROM sales",
template_params={"limit": 10},
)
sql_lab_ref = adapter.create_sql_lab_session(payload)
assert sql_lab_ref == "query-123"
assert client.network.request.call_count == 2
first_call = client.network.request.call_args_list[0].kwargs
second_call = client.network.request.call_args_list[1].kwargs
assert first_call["endpoint"] == "/sqllab/execute/"
assert second_call["endpoint"] == "/sql_lab/execute/"
# [/DEF:test_sql_lab_launch_falls_back_to_legacy_execute_endpoint:Function]
# [/DEF:SupersetCompatibilityMatrixTests:Module]

View File

@@ -0,0 +1,472 @@
<!-- [DEF:ClarificationDialog:Component] -->
<!-- @COMPLEXITY: 4 -->
<!-- @SEMANTICS: dataset-review, clarification, single-question, resumable, expert-review -->
<!-- @PURPOSE: Present one clarification question at a time with explicit why-it-matters context, answer actions, and resumable state transitions for US2. -->
<!-- @LAYER: UI -->
<!-- @RELATION: [BINDS_TO] ->[api_module] -->
<!-- @RELATION: [BINDS_TO] ->[assistantChat] -->
<!-- @RELATION: [BINDS_TO] ->[DatasetReviewWorkspace] -->
<!-- @PRE: Session id is available and clarification state payload belongs to the current ownership-scoped session. -->
<!-- @POST: Users can answer, skip, or route to expert review while the UI keeps only the active clarification question visible and preserves resumable progress. -->
<!-- @SIDE_EFFECT: Persists clarification answers and may resume clarification state from dataset orchestration APIs. -->
<!-- @UX_STATE: Question -> Active clarification question with options and current guess is visible. -->
<!-- @UX_STATE: Saving -> Answer controls are disabled while one mutation is in flight. -->
<!-- @UX_STATE: Completed -> No active question remains and resumable clarification summary stays visible. -->
<!-- @UX_FEEDBACK: Save results and changed findings surface inline without hiding the active or completed state. -->
<!-- @UX_RECOVERY: Users can resume clarification after interruption, skip uncertain questions, or open assistant guidance before escalating to expert review. -->
<!-- @UX_REACTIVITY: Uses $props, $state, and $derived only; no legacy reactive syntax. -->
<script>
import { t } from "$lib/i18n";
import { requestApi } from "$lib/api.js";
import { openAssistantChat } from "$lib/stores/assistantChat.js";
let {
sessionId = "",
clarificationState = null,
disabled = false,
onupdated = () => {},
} = $props();
let selectedOptionValue = $state("");
let customAnswerValue = $state("");
let localMessage = $state("");
let localStatus = $state("");
let isSaving = $state(false);
let lastQuestionId = $state("");
let feedbackQuestionId = $state("");
let feedbackQuestionText = $state("");
let feedbackSaving = $state("");
const clarificationSession = $derived(clarificationState?.clarification_session || null);
const currentQuestion = $derived(clarificationState?.current_question || null);
const dialogState = $derived(
isSaving ? "Saving" : currentQuestion ? "Question" : "Completed",
);
const hasQuestion = $derived(Boolean(currentQuestion?.question_id));
const progressLabel = $derived(
clarificationSession
? `${clarificationSession.resolved_count} / ${
clarificationSession.resolved_count + clarificationSession.remaining_count
}`
: "0 / 0",
);
$effect(() => {
const questionId = currentQuestion?.question_id || "";
if (questionId === lastQuestionId) {
return;
}
lastQuestionId = questionId;
if (!questionId) {
selectedOptionValue = "";
customAnswerValue = "";
return;
}
const recommendedOption = (currentQuestion?.options || []).find(
(option) => option.is_recommended,
);
selectedOptionValue = recommendedOption?.value || "";
customAnswerValue = "";
});
function clearFeedback() {
localMessage = "";
localStatus = "";
}
function normalizeAnswerValue(answerKind) {
if (answerKind === "selected") {
return selectedOptionValue || null;
}
if (answerKind === "custom") {
return customAnswerValue.trim() || null;
}
return null;
}
async function recordClarificationFeedback(feedback) {
if (!sessionId || !feedbackQuestionId || isSaving || feedbackSaving) {
return;
}
feedbackSaving = feedback;
localStatus = "saving";
localMessage = "";
try {
await requestApi(
`/dataset-orchestration/sessions/${sessionId}/clarification/questions/${feedbackQuestionId}/feedback`,
"POST",
{ feedback },
);
localStatus = "success";
localMessage =
feedback === "up"
? $t.dataset_review?.clarification?.messages?.feedback_up
: $t.dataset_review?.clarification?.messages?.feedback_down;
} catch (error) {
localStatus = "error";
localMessage =
error?.message ||
$t.dataset_review?.clarification?.messages?.feedback_failed ||
$t.common?.error;
} finally {
feedbackSaving = "";
}
}
async function resumeClarification() {
if (!sessionId || isSaving) {
return;
}
isSaving = true;
localStatus = "saving";
localMessage = "";
try {
const nextState = await requestApi(
`/dataset-orchestration/sessions/${sessionId}/clarification/resume`,
"POST",
);
localStatus = "success";
localMessage = $t.dataset_review?.clarification?.messages?.resumed;
onupdated(nextState);
} catch (error) {
localStatus = "error";
localMessage =
error?.message ||
$t.dataset_review?.clarification?.messages?.resume_failed ||
$t.common?.error;
} finally {
isSaving = false;
}
}
async function submitAnswer(answerKind) {
if (!sessionId || !currentQuestion?.question_id || isSaving) {
return;
}
const answerValue = normalizeAnswerValue(answerKind);
if (answerKind === "selected" && !answerValue) {
localStatus = "error";
localMessage = $t.dataset_review?.clarification?.messages?.option_required;
return;
}
if (answerKind === "custom" && !answerValue) {
localStatus = "error";
localMessage = $t.dataset_review?.clarification?.messages?.custom_required;
return;
}
isSaving = true;
localStatus = "saving";
localMessage = "";
try {
const answeredQuestionId = currentQuestion.question_id;
const answeredQuestionText = currentQuestion.question_text;
const result = await requestApi(
`/dataset-orchestration/sessions/${sessionId}/clarification/answers`,
"POST",
{
question_id: answeredQuestionId,
answer_kind: answerKind,
answer_value: answerValue,
},
);
feedbackQuestionId = answeredQuestionId;
feedbackQuestionText = answeredQuestionText;
localStatus = "success";
if (answerKind === "skipped") {
localMessage = $t.dataset_review?.clarification?.messages?.skipped;
} else if (answerKind === "expert_review") {
localMessage =
$t.dataset_review?.clarification?.messages?.expert_review_requested;
} else {
localMessage = $t.dataset_review?.clarification?.messages?.saved;
}
onupdated(result);
} catch (error) {
localStatus = "error";
localMessage =
error?.message ||
$t.dataset_review?.clarification?.messages?.save_failed ||
$t.common?.error;
} finally {
isSaving = false;
}
}
function requestAssistantGuidance() {
openAssistantChat();
localStatus = "success";
localMessage = $t.dataset_review?.clarification?.messages?.assistant_opened;
}
</script>
<section id="clarification" class="rounded-2xl border border-slate-200 bg-white p-5 shadow-sm">
<div class="flex flex-col gap-3 md:flex-row md:items-start md:justify-between">
<div>
<p class="text-xs font-semibold uppercase tracking-wide text-slate-500">
{$t.dataset_review?.clarification?.eyebrow}
</p>
<h2 class="text-xl font-semibold text-slate-900">
{$t.dataset_review?.clarification?.title}
</h2>
<p class="mt-1 max-w-3xl text-sm text-slate-600">
{$t.dataset_review?.clarification?.description}
</p>
</div>
<div class="rounded-xl border border-slate-200 bg-slate-50 px-3 py-2 text-right">
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.clarification?.progress_label}
</div>
<div class="mt-1 text-sm font-medium text-slate-900">{progressLabel}</div>
<div class="mt-1 text-xs text-slate-500">
{$t.dataset_review?.clarification?.state_label}: {dialogState}
</div>
</div>
</div>
{#if clarificationSession}
<div class="mt-4 flex flex-wrap gap-2 text-xs text-slate-600">
<span class="rounded-full bg-slate-100 px-2 py-1">
{$t.dataset_review?.clarification?.status_label}: {clarificationSession.status}
</span>
<span class="rounded-full bg-slate-100 px-2 py-1">
{$t.dataset_review?.clarification?.remaining_label}: {clarificationSession.remaining_count}
</span>
{#if clarificationSession.summary_delta}
<span class="rounded-full bg-slate-100 px-2 py-1">
{$t.dataset_review?.clarification?.summary_delta_label}:
{clarificationSession.summary_delta}
</span>
{/if}
</div>
{/if}
{#if hasQuestion}
<article class="mt-5 rounded-2xl border border-violet-200 bg-violet-50 p-4">
<div class="flex flex-col gap-4 lg:flex-row lg:items-start lg:justify-between">
<div class="min-w-0 flex-1">
<p class="text-xs font-semibold uppercase tracking-wide text-violet-700">
{$t.dataset_review?.clarification?.active_question_label}
</p>
<h3 class="mt-1 text-lg font-semibold text-slate-900">
{currentQuestion.question_text}
</h3>
<div class="mt-3 rounded-xl border border-white/70 bg-white/80 p-3">
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.clarification?.why_it_matters_label}
</div>
<p class="mt-1 text-sm text-slate-700">{currentQuestion.why_it_matters}</p>
</div>
<div class="mt-3 grid gap-3 md:grid-cols-2">
<div class="rounded-xl border border-white/70 bg-white/80 p-3">
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.clarification?.current_guess_label}
</div>
<p class="mt-1 text-sm text-slate-700">
{currentQuestion.current_guess ||
$t.dataset_review?.clarification?.current_guess_empty}
</p>
</div>
<div class="rounded-xl border border-white/70 bg-white/80 p-3">
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.clarification?.topic_label}
</div>
<p class="mt-1 text-sm text-slate-700">{currentQuestion.topic_ref}</p>
</div>
</div>
<div class="mt-4">
<h4 class="text-sm font-semibold text-slate-900">
{$t.dataset_review?.clarification?.options_title}
</h4>
<div class="mt-3 space-y-2">
{#each currentQuestion.options || [] as option}
<label
class={`flex cursor-pointer items-start gap-3 rounded-xl border p-3 transition ${
selectedOptionValue === option.value
? "border-violet-300 bg-white text-slate-900"
: "border-white/70 bg-white/80 text-slate-700"
}`}
>
<input
type="radio"
name="clarification-option"
value={option.value}
checked={selectedOptionValue === option.value}
onchange={() => {
selectedOptionValue = option.value;
clearFeedback();
}}
disabled={disabled || isSaving}
/>
<div class="min-w-0 flex-1">
<div class="flex flex-wrap items-center gap-2">
<span class="text-sm font-medium">{option.label}</span>
{#if option.is_recommended}
<span class="rounded-full bg-violet-100 px-2 py-0.5 text-xs text-violet-700">
{$t.dataset_review?.clarification?.recommended_badge}
</span>
{/if}
</div>
<div class="mt-1 text-xs text-slate-500">{option.value}</div>
</div>
</label>
{/each}
</div>
</div>
<label class="mt-4 block space-y-2">
<span class="text-sm font-medium text-slate-700">
{$t.dataset_review?.clarification?.custom_answer_label}
</span>
<textarea
class="min-h-24 w-full rounded-xl border border-slate-300 bg-white px-3 py-2 text-sm text-slate-900 shadow-sm outline-none transition focus:border-violet-500 focus:ring-2 focus:ring-violet-200"
bind:value={customAnswerValue}
disabled={disabled || isSaving}
></textarea>
</label>
</div>
<div class="flex shrink-0 flex-wrap gap-2 lg:w-56 lg:flex-col">
<button
type="button"
class="inline-flex items-center justify-center rounded-xl bg-violet-600 px-4 py-2 text-sm font-medium text-white transition hover:bg-violet-700 disabled:cursor-not-allowed disabled:bg-slate-300"
onclick={() => submitAnswer("selected")}
disabled={disabled || isSaving}
>
{$t.dataset_review?.clarification?.answer_action}
</button>
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-4 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => submitAnswer("custom")}
disabled={disabled || isSaving}
>
{$t.dataset_review?.clarification?.custom_answer_action}
</button>
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-4 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => submitAnswer("skipped")}
disabled={disabled || isSaving}
>
{$t.dataset_review?.clarification?.skip_action}
</button>
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-4 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => requestAssistantGuidance()}
disabled={disabled || isSaving}
>
{$t.dataset_review?.clarification?.assistant_action}
</button>
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-amber-300 px-4 py-2 text-sm font-medium text-amber-800 transition hover:bg-amber-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => submitAnswer("expert_review")}
disabled={disabled || isSaving}
>
{$t.dataset_review?.clarification?.expert_review_action}
</button>
</div>
</div>
</article>
{:else}
<div class="mt-5 rounded-xl border border-emerald-200 bg-emerald-50 px-4 py-4 text-sm text-emerald-800">
{$t.dataset_review?.clarification?.completed}
</div>
<div class="mt-4 flex flex-wrap gap-2">
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-4 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={resumeClarification}
disabled={disabled || isSaving}
>
{$t.dataset_review?.clarification?.resume_action}
</button>
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-4 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => requestAssistantGuidance()}
disabled={isSaving}
>
{$t.dataset_review?.clarification?.assistant_action}
</button>
</div>
{/if}
{#if feedbackQuestionId}
<div class="mt-4 rounded-xl border border-slate-200 bg-slate-50 p-4">
<div class="flex flex-col gap-3 md:flex-row md:items-center md:justify-between">
<div>
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.clarification?.feedback_label}
</div>
<p class="mt-1 text-sm text-slate-700">
{feedbackQuestionText ||
$t.dataset_review?.clarification?.feedback_prompt}
</p>
</div>
<div class="grid grid-cols-2 gap-2">
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-3 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => recordClarificationFeedback("up")}
disabled={disabled || isSaving || Boolean(feedbackSaving)}
>
{$t.dataset_review?.clarification?.feedback_up_action}
</button>
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-3 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => recordClarificationFeedback("down")}
disabled={disabled || isSaving || Boolean(feedbackSaving)}
>
{$t.dataset_review?.clarification?.feedback_down_action}
</button>
</div>
</div>
</div>
{/if}
{#if localMessage || localStatus === "saving"}
<div
class={`mt-4 rounded-xl px-3 py-3 text-sm ${
localStatus === "error"
? "border border-red-200 bg-red-50 text-red-700"
: localStatus === "saving"
? "border border-violet-200 bg-violet-50 text-violet-700"
: "border border-emerald-200 bg-emerald-50 text-emerald-800"
}`}
>
{localStatus === "saving"
? $t.dataset_review?.clarification?.messages?.saving
: localMessage}
</div>
{/if}
</section>
<!-- [/DEF:ClarificationDialog:Component] -->

View File

@@ -0,0 +1,247 @@
<!-- [DEF:CompiledSQLPreview:Component] -->
<!-- @COMPLEXITY: 4 -->
<!-- @SEMANTICS: dataset-review, compiled-sql-preview, superset-preview, stale-state, diagnostics -->
<!-- @PURPOSE: Present the exact Superset-generated compiled SQL preview, expose readiness or staleness clearly, and preserve readable recovery paths when preview generation fails. -->
<!-- @LAYER: UI -->
<!-- @RELATION: [BINDS_TO] ->[api_module] -->
<!-- @RELATION: [BINDS_TO] ->[DatasetReviewWorkspace] -->
<!-- @PRE: Session id is available and preview state comes from the current ownership-scoped session detail payload. -->
<!-- @POST: Users can distinguish missing, pending, ready, stale, and error preview states and can trigger only Superset-backed preview generation. -->
<!-- @SIDE_EFFECT: Requests preview generation through dataset orchestration APIs and updates route shell preview state when Superset responds. -->
<!-- @UX_STATE: Missing -> Prompt the user to generate a Superset preview before launch. -->
<!-- @UX_STATE: Pending -> Show generation-in-progress feedback without fabricating SQL. -->
<!-- @UX_STATE: Ready -> Render read-only SQL preview that is explicitly labeled as compiled by Superset. -->
<!-- @UX_STATE: Stale -> Mark the existing preview as outdated after mapping/input changes until regenerated. -->
<!-- @UX_STATE: Error -> Show readable Superset compilation diagnostics and preserve remediation action. -->
<!-- @UX_FEEDBACK: Preview refresh updates status pill, timestamps, and inline generation feedback. -->
<!-- @UX_RECOVERY: Users can retry preview generation and jump back to mapping review when diagnostics point to execution-input issues. -->
<!-- @UX_REACTIVITY: Uses $props, $state, and $derived only; no legacy reactive syntax. -->
<script>
import { t } from "$lib/i18n";
import { requestApi } from "$lib/api.js";
let {
sessionId = "",
preview = null,
previewState = "",
disabled = false,
onupdated = () => {},
onjump = () => {},
} = $props();
let localStatus = $state("");
let localMessage = $state("");
const effectiveState = $derived(resolvePreviewState());
const compiledBySuperset = $derived(String(preview?.compiled_by || "") === "superset");
const hasSql = $derived(Boolean(String(preview?.compiled_sql || "").trim()));
const previewTimestamp = $derived(preview?.compiled_at || preview?.created_at || "");
function resolvePreviewState() {
const explicit = String(previewState || "").trim();
if (explicit) {
return explicit;
}
const status = String(preview?.preview_status || "").trim();
if (status === "ready" || status === "pending" || status === "failed" || status === "stale") {
return status;
}
if (preview?.error_code || preview?.error_details) {
return "failed";
}
return "missing";
}
function getStateLabel(state) {
return $t.dataset_review?.preview?.state?.[state] || state;
}
function getStateTone(state) {
if (state === "ready") {
return "bg-emerald-100 text-emerald-700";
}
if (state === "stale") {
return "bg-amber-100 text-amber-700";
}
if (state === "failed") {
return "bg-red-100 text-red-700";
}
if (state === "pending") {
return "bg-blue-100 text-blue-700";
}
return "bg-slate-100 text-slate-700";
}
function getCompilerLabel() {
if (compiledBySuperset) {
return $t.dataset_review?.preview?.compiled_by_superset;
}
return $t.dataset_review?.preview?.compiled_source_unknown;
}
function getPreviewBodyText() {
if (effectiveState === "pending") {
return $t.dataset_review?.preview?.pending_body;
}
if (effectiveState === "stale") {
return $t.dataset_review?.preview?.stale_body;
}
if (effectiveState === "failed") {
return (
preview?.error_details ||
preview?.error_code ||
$t.dataset_review?.preview?.error_body
);
}
if (effectiveState === "missing") {
return $t.dataset_review?.preview?.missing_body;
}
return $t.dataset_review?.preview?.ready_body;
}
async function requestPreview() {
if (!sessionId || disabled || localStatus === "saving") {
return;
}
localStatus = "saving";
localMessage = "";
try {
const response = await requestApi(
`/dataset-orchestration/sessions/${sessionId}/preview`,
"POST",
);
const nextPreview = response?.preview || response;
onupdated({
preview: nextPreview,
preview_state: nextPreview?.preview_status || response?.preview_status || "pending",
});
localStatus = "success";
localMessage = $t.dataset_review?.preview?.messages?.generated;
} catch (error) {
localStatus = "error";
localMessage =
error?.message ||
$t.dataset_review?.preview?.messages?.generate_failed ||
$t.common?.error;
}
}
function jumpToMapping() {
onjump({ target: "mapping" });
}
</script>
<section id="sql-preview" class="rounded-2xl border border-slate-200 bg-white p-5 shadow-sm">
<div class="flex flex-col gap-3 md:flex-row md:items-start md:justify-between">
<div>
<p class="text-xs font-semibold uppercase tracking-wide text-slate-500">
{$t.dataset_review?.preview?.eyebrow}
</p>
<h2 class="text-xl font-semibold text-slate-900">
{$t.dataset_review?.preview?.title}
</h2>
<p class="mt-1 max-w-3xl text-sm text-slate-600">
{$t.dataset_review?.preview?.description}
</p>
</div>
<div class="flex flex-wrap items-center gap-2">
<span class={`rounded-full px-3 py-1 text-xs font-medium ${getStateTone(effectiveState)}`}>
{$t.dataset_review?.preview?.state_label}: {getStateLabel(effectiveState)}
</span>
<button
type="button"
class="inline-flex items-center justify-center rounded-xl bg-blue-600 px-4 py-2 text-sm font-medium text-white transition hover:bg-blue-700 disabled:cursor-not-allowed disabled:bg-slate-300"
onclick={requestPreview}
disabled={disabled || localStatus === "saving"}
>
{localStatus === "saving"
? $t.dataset_review?.preview?.generate_loading
: $t.dataset_review?.preview?.generate_action}
</button>
</div>
</div>
<div
class={`mt-5 rounded-xl border px-4 py-4 text-sm ${
effectiveState === "failed"
? "border-red-200 bg-red-50 text-red-700"
: effectiveState === "stale"
? "border-amber-200 bg-amber-50 text-amber-800"
: effectiveState === "pending"
? "border-blue-200 bg-blue-50 text-blue-700"
: "border-slate-200 bg-slate-50 text-slate-700"
}`}
>
{getPreviewBodyText()}
</div>
{#if preview}
<div class="mt-4 flex flex-wrap gap-2 text-xs text-slate-600">
<span class="rounded-full bg-slate-100 px-2 py-1">
{$t.dataset_review?.preview?.compiler_label}: {getCompilerLabel()}
</span>
<span class="rounded-full bg-slate-100 px-2 py-1">
{$t.dataset_review?.preview?.fingerprint_label}: {preview.preview_fingerprint}
</span>
{#if previewTimestamp}
<span class="rounded-full bg-slate-100 px-2 py-1">
{$t.dataset_review?.preview?.compiled_at_label}: {previewTimestamp}
</span>
{/if}
</div>
{/if}
{#if effectiveState === "ready" && hasSql}
<div class="mt-5 overflow-hidden rounded-2xl border border-slate-200 bg-slate-950">
<div class="flex items-center justify-between border-b border-slate-800 px-4 py-3">
<div class="text-sm font-medium text-slate-100">
{$t.dataset_review?.preview?.sql_block_title}
</div>
<div class="text-xs text-slate-400">
{$t.dataset_review?.preview?.compiled_truth_note}
</div>
</div>
<pre class="overflow-x-auto px-4 py-4 text-sm leading-6 text-slate-100"><code>{preview.compiled_sql}</code></pre>
</div>
{:else if effectiveState === "failed"}
<div class="mt-5 flex flex-wrap gap-2">
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-4 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50"
onclick={jumpToMapping}
>
{$t.dataset_review?.preview?.go_to_mapping_action}
</button>
</div>
{:else if effectiveState === "stale"}
<div class="mt-5 flex flex-wrap gap-2">
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-4 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50"
onclick={jumpToMapping}
>
{$t.dataset_review?.preview?.review_inputs_action}
</button>
</div>
{/if}
{#if localMessage}
<div
class={`mt-4 rounded-xl px-3 py-3 text-sm ${
localStatus === "error"
? "border border-red-200 bg-red-50 text-red-700"
: "border border-emerald-200 bg-emerald-50 text-emerald-800"
}`}
>
{localMessage}
</div>
{/if}
</section>
<!-- [/DEF:CompiledSQLPreview:Component] -->

View File

@@ -0,0 +1,578 @@
<!-- [DEF:ExecutionMappingReview:Component] -->
<!-- @COMPLEXITY: 4 -->
<!-- @SEMANTICS: dataset-review, execution-mapping, warning-approval, manual-override, required-values -->
<!-- @PURPOSE: Review imported-filter to template-variable mappings, surface effective values and blockers, and require explicit approval for warning-sensitive execution inputs before preview or launch. -->
<!-- @LAYER: UI -->
<!-- @RELATION: [BINDS_TO] ->[api_module] -->
<!-- @RELATION: [BINDS_TO] ->[DatasetReviewWorkspace] -->
<!-- @PRE: Session id, execution mappings, imported filters, and template variables belong to the current ownership-scoped session payload. -->
<!-- @POST: Users can review effective mapping values, approve warning-sensitive transformations, or manually override them while unresolved required-value blockers remain visible. -->
<!-- @SIDE_EFFECT: Persists mapping approvals or manual overrides through dataset orchestration APIs and may invalidate the current preview truth for the route shell. -->
<!-- @UX_STATE: Incomplete -> Required mapping values remain missing and blockers stay visible. -->
<!-- @UX_STATE: WarningApproval -> Mapping rows with transformation risk require explicit approval before execution can proceed. -->
<!-- @UX_STATE: Approved -> All launch-sensitive mappings are approved or no explicit approval is required. -->
<!-- @UX_FEEDBACK: Mapping approvals and manual overrides expose inline success, saving, and error feedback per row. -->
<!-- @UX_RECOVERY: Users can replace transformed values manually instead of approving them as-is and can retry failed mutations in place. -->
<!-- @UX_REACTIVITY: Uses $props, $state, and $derived only; no legacy reactive syntax. -->
<script>
import { t } from "$lib/i18n";
import { requestApi } from "$lib/api.js";
let {
sessionId = "",
mappings = [],
importedFilters = [],
templateVariables = [],
disabled = false,
onupdated = () => {},
} = $props();
let editingMappingId = $state("");
let overrideValueDraft = $state("");
let overrideNoteDraft = $state("");
let savingMappingId = $state("");
let batchSaving = $state(false);
let rowMessages = $state({});
const filterLookup = $derived(
Object.fromEntries((importedFilters || []).map((item) => [item.filter_id, item])),
);
const variableLookup = $derived(
Object.fromEntries((templateVariables || []).map((item) => [item.variable_id, item])),
);
const sortedMappings = $derived(
[...(mappings || [])].sort((left, right) => {
const leftVariable = variableLookup[left.variable_id];
const rightVariable = variableLookup[right.variable_id];
const leftRequired = Number(Boolean(leftVariable?.is_required));
const rightRequired = Number(Boolean(rightVariable?.is_required));
if (leftRequired !== rightRequired) {
return rightRequired - leftRequired;
}
const leftNeedsApproval = Number(
Boolean(left.requires_explicit_approval && left.approval_state !== "approved"),
);
const rightNeedsApproval = Number(
Boolean(right.requires_explicit_approval && right.approval_state !== "approved"),
);
if (leftNeedsApproval !== rightNeedsApproval) {
return rightNeedsApproval - leftNeedsApproval;
}
return String(leftVariable?.variable_name || left.variable_id || "").localeCompare(
String(rightVariable?.variable_name || right.variable_id || ""),
);
}),
);
const missingRequiredCount = $derived(
sortedMappings.filter((mapping) => getRowState(mapping).missingRequiredValue).length,
);
const pendingApprovalCount = $derived(
sortedMappings.filter((mapping) => getRowState(mapping).needsExplicitApproval).length,
);
const reviewState = $derived(
missingRequiredCount > 0
? "Incomplete"
: pendingApprovalCount > 0
? "WarningApproval"
: "Approved",
);
function setRowMessage(mappingId, patch) {
rowMessages = {
...rowMessages,
[mappingId]: {
...(rowMessages[mappingId] || {}),
...patch,
},
};
}
function resetRowMessage(mappingId) {
rowMessages = {
...rowMessages,
[mappingId]: { status: "", text: "" },
};
}
function getApprovalLabel(approvalState) {
const normalized = String(approvalState || "pending");
return $t.dataset_review?.mapping?.approval_state?.[normalized] || normalized;
}
function getWarningLabel(warningLevel) {
const normalized = String(warningLevel || "none");
return $t.dataset_review?.mapping?.warning_level?.[normalized] || normalized;
}
function getMethodLabel(method) {
const normalized = String(method || "direct_match");
return $t.dataset_review?.mapping?.method?.[normalized] || normalized;
}
function stringifyValue(value) {
if (value === null || value === undefined || value === "") {
return $t.common?.not_available || "N/A";
}
if (typeof value === "string") {
return value;
}
try {
return JSON.stringify(value);
} catch (_error) {
return String(value);
}
}
function getEffectiveValue(mapping) {
const importedFilter = filterLookup[mapping.filter_id];
const templateVariable = variableLookup[mapping.variable_id];
if (mapping.effective_value !== null && mapping.effective_value !== undefined) {
return mapping.effective_value;
}
if (importedFilter?.normalized_value !== null && importedFilter?.normalized_value !== undefined) {
return importedFilter.normalized_value;
}
if (importedFilter?.raw_value !== null && importedFilter?.raw_value !== undefined) {
return importedFilter.raw_value;
}
if (templateVariable?.default_value !== null && templateVariable?.default_value !== undefined) {
return templateVariable.default_value;
}
return null;
}
function getRowState(mapping) {
const templateVariable = variableLookup[mapping.variable_id];
const effectiveValue = getEffectiveValue(mapping);
const missingRequiredValue = Boolean(templateVariable?.is_required) && effectiveValue === null;
const needsExplicitApproval = Boolean(
mapping.requires_explicit_approval && mapping.approval_state !== "approved",
);
return {
templateVariable,
importedFilter: filterLookup[mapping.filter_id],
effectiveValue,
missingRequiredValue,
needsExplicitApproval,
};
}
function startManualOverride(mapping) {
const rowState = getRowState(mapping);
editingMappingId = mapping.mapping_id;
overrideValueDraft =
rowState.effectiveValue === null ? "" : stringifyValue(rowState.effectiveValue);
overrideNoteDraft = mapping.transformation_note || "";
resetRowMessage(mapping.mapping_id);
}
function cancelManualOverride(mappingId) {
if (editingMappingId !== mappingId) {
return;
}
editingMappingId = "";
overrideValueDraft = "";
overrideNoteDraft = "";
resetRowMessage(mappingId);
}
function normalizeOverrideValue() {
const trimmed = String(overrideValueDraft || "").trim();
if (!trimmed) {
return null;
}
try {
return JSON.parse(trimmed);
} catch (_error) {
return trimmed;
}
}
async function approveMapping(mapping) {
if (!sessionId || !mapping?.mapping_id || savingMappingId) {
return;
}
savingMappingId = mapping.mapping_id;
setRowMessage(mapping.mapping_id, { status: "saving", text: "" });
try {
const updatedMapping = await requestApi(
`/dataset-orchestration/sessions/${sessionId}/mappings/${mapping.mapping_id}/approve`,
"POST",
mapping.transformation_note ? { approval_note: mapping.transformation_note } : {},
);
setRowMessage(mapping.mapping_id, {
status: "success",
text: $t.dataset_review?.mapping?.messages?.approval_saved,
});
onupdated({
mapping: updatedMapping,
preview_state: "unchanged",
});
} catch (error) {
setRowMessage(mapping.mapping_id, {
status: "error",
text:
error?.message ||
$t.dataset_review?.mapping?.messages?.approval_failed ||
$t.common?.error,
});
} finally {
savingMappingId = "";
}
}
async function approveAllMappings() {
if (!sessionId || batchSaving || pendingApprovalCount === 0) {
return;
}
batchSaving = true;
try {
const mappingIds = sortedMappings
.filter((mapping) => getRowState(mapping).needsExplicitApproval)
.map((mapping) => mapping.mapping_id);
const updatedMappings = await requestApi(
`/dataset-orchestration/sessions/${sessionId}/mappings/approve-batch`,
"POST",
{ mapping_ids: mappingIds },
);
onupdated({
mappings: updatedMappings,
preview_state: "unchanged",
});
} catch (error) {
const message =
error?.message ||
$t.dataset_review?.mapping?.messages?.approval_failed ||
$t.common?.error;
for (const mapping of sortedMappings.filter((item) => getRowState(item).needsExplicitApproval)) {
setRowMessage(mapping.mapping_id, { status: "error", text: message });
}
} finally {
batchSaving = false;
}
}
async function saveManualOverride(mapping) {
if (!sessionId || !mapping?.mapping_id || savingMappingId) {
return;
}
const effectiveValue = normalizeOverrideValue();
const templateVariable = variableLookup[mapping.variable_id];
if (effectiveValue === null && templateVariable?.is_required) {
setRowMessage(mapping.mapping_id, {
status: "error",
text: $t.dataset_review?.mapping?.messages?.required_value_missing,
});
return;
}
savingMappingId = mapping.mapping_id;
setRowMessage(mapping.mapping_id, { status: "saving", text: "" });
try {
const updatedMapping = await requestApi(
`/dataset-orchestration/sessions/${sessionId}/mappings/${mapping.mapping_id}`,
"PATCH",
{
effective_value: effectiveValue,
mapping_method: "manual_override",
transformation_note: String(overrideNoteDraft || "").trim() || null,
},
);
setRowMessage(mapping.mapping_id, {
status: "success",
text: $t.dataset_review?.mapping?.messages?.override_saved,
});
editingMappingId = "";
overrideValueDraft = "";
overrideNoteDraft = "";
onupdated({
mapping: updatedMapping,
preview_state: "stale",
});
} catch (error) {
setRowMessage(mapping.mapping_id, {
status: "error",
text:
error?.message ||
$t.dataset_review?.mapping?.messages?.override_failed ||
$t.common?.error,
});
} finally {
savingMappingId = "";
}
}
</script>
<section id="mapping" class="rounded-2xl border border-slate-200 bg-white p-5 shadow-sm">
<div class="flex flex-col gap-3 md:flex-row md:items-start md:justify-between">
<div>
<p class="text-xs font-semibold uppercase tracking-wide text-slate-500">
{$t.dataset_review?.mapping?.eyebrow}
</p>
<h2 class="text-xl font-semibold text-slate-900">
{$t.dataset_review?.mapping?.title}
</h2>
<p class="mt-1 max-w-3xl text-sm text-slate-600">
{$t.dataset_review?.mapping?.description}
</p>
</div>
<div class="flex flex-wrap gap-2">
<span
class={`rounded-full px-3 py-1 text-xs font-medium ${
reviewState === "Incomplete"
? "bg-red-100 text-red-700"
: reviewState === "WarningApproval"
? "bg-amber-100 text-amber-700"
: "bg-emerald-100 text-emerald-700"
}`}
>
{$t.dataset_review?.mapping?.state_label}: {$t.dataset_review?.mapping?.state?.[reviewState]}
</span>
<span class="rounded-full bg-slate-100 px-3 py-1 text-xs font-medium text-slate-700">
{$t.dataset_review?.mapping?.pending_approvals_label}: {pendingApprovalCount}
</span>
<span class="rounded-full bg-slate-100 px-3 py-1 text-xs font-medium text-slate-700">
{$t.dataset_review?.mapping?.required_values_label}: {missingRequiredCount}
</span>
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-4 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={approveAllMappings}
disabled={disabled || batchSaving || pendingApprovalCount === 0}
>
{$t.dataset_review?.mapping?.approve_all_action}
</button>
</div>
</div>
{#if sortedMappings.length === 0}
<div class="mt-5 rounded-xl border border-slate-200 bg-slate-50 px-4 py-4 text-sm text-slate-600">
{$t.dataset_review?.mapping?.empty}
</div>
{:else}
{#if missingRequiredCount > 0}
<div class="mt-5 rounded-xl border border-red-200 bg-red-50 px-4 py-4 text-sm text-red-700">
{$t.dataset_review?.mapping?.required_blockers_notice.replace(
"{count}",
String(missingRequiredCount),
)}
</div>
{/if}
{#if pendingApprovalCount > 0}
<div class="mt-3 rounded-xl border border-amber-200 bg-amber-50 px-4 py-4 text-sm text-amber-800">
{$t.dataset_review?.mapping?.approval_notice.replace(
"{count}",
String(pendingApprovalCount),
)}
</div>
{/if}
<div class="mt-5 space-y-4">
{#each sortedMappings as mapping}
{@const rowState = getRowState(mapping)}
{@const templateVariable = rowState.templateVariable}
{@const importedFilter = rowState.importedFilter}
{@const isEditing = editingMappingId === mapping.mapping_id}
{@const isSaving = savingMappingId === mapping.mapping_id}
{@const rowMessage = rowMessages[mapping.mapping_id] || { status: "", text: "" }}
<article
class={`rounded-2xl border p-4 shadow-sm ${
rowState.missingRequiredValue
? "border-red-200 bg-red-50"
: rowState.needsExplicitApproval
? "border-amber-200 bg-amber-50"
: "border-slate-200 bg-white"
}`}
>
<div class="flex flex-col gap-4 xl:flex-row xl:items-start xl:justify-between">
<div class="min-w-0 flex-1">
<div class="flex flex-wrap items-center gap-2">
<h3 class="text-base font-semibold text-slate-900">
{importedFilter?.display_name || importedFilter?.filter_name || mapping.mapping_id}
</h3>
<span class="rounded-full bg-slate-100 px-2 py-0.5 text-xs text-slate-700">
{$t.dataset_review?.mapping?.to_variable_label}: {templateVariable?.variable_name ||
mapping.variable_id}
</span>
{#if templateVariable?.is_required}
<span class="rounded-full bg-red-100 px-2 py-0.5 text-xs text-red-700">
{$t.dataset_review?.mapping?.required_badge}
</span>
{/if}
{#if rowState.needsExplicitApproval}
<span class="rounded-full bg-amber-100 px-2 py-0.5 text-xs text-amber-800">
{$t.dataset_review?.mapping?.approval_required_badge}
</span>
{/if}
{#if mapping.approval_state === "approved"}
<span class="rounded-full bg-emerald-100 px-2 py-0.5 text-xs text-emerald-700">
{$t.dataset_review?.mapping?.approved_badge}
</span>
{/if}
</div>
<div class="mt-3 grid gap-3 lg:grid-cols-2">
<div class="rounded-xl border border-slate-200 bg-white p-3">
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.mapping?.source_filter_label}
</div>
<div class="mt-1 text-sm font-medium text-slate-900">
{importedFilter?.filter_name || mapping.filter_id}
</div>
<div class="mt-2 flex flex-wrap gap-2 text-xs text-slate-600">
<span class="rounded-full bg-slate-100 px-2 py-1">
{$t.dataset_review?.mapping?.source_value_label}:
{stringifyValue(importedFilter?.raw_value)}
</span>
<span class="rounded-full bg-slate-100 px-2 py-1">
{$t.dataset_review?.mapping?.confidence_label}:
{importedFilter?.confidence_state || ($t.common?.unknown || "unknown")}
</span>
<span class="rounded-full bg-slate-100 px-2 py-1">
{$t.dataset_review?.mapping?.recovery_label}:
{importedFilter?.recovery_status || ($t.common?.unknown || "unknown")}
</span>
</div>
</div>
<div class="rounded-xl border border-slate-200 bg-white p-3">
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.mapping?.effective_value_label}
</div>
<div class="mt-1 break-all text-sm font-medium text-slate-900">
{stringifyValue(rowState.effectiveValue)}
</div>
<div class="mt-2 flex flex-wrap gap-2 text-xs text-slate-600">
<span class="rounded-full bg-slate-100 px-2 py-1">
{$t.dataset_review?.mapping?.method_label}: {getMethodLabel(mapping.mapping_method)}
</span>
<span class="rounded-full bg-slate-100 px-2 py-1">
{$t.dataset_review?.mapping?.approval_label}: {getApprovalLabel(mapping.approval_state)}
</span>
{#if mapping.warning_level}
<span class="rounded-full bg-slate-100 px-2 py-1">
{$t.dataset_review?.mapping?.warning_label}: {getWarningLabel(mapping.warning_level)}
</span>
{/if}
</div>
</div>
</div>
{#if mapping.transformation_note}
<div class="mt-3 rounded-xl border border-slate-200 bg-slate-50 px-3 py-3 text-sm text-slate-700">
<span class="font-medium text-slate-900">
{$t.dataset_review?.mapping?.transformation_note_label}:
</span>
{mapping.transformation_note}
</div>
{/if}
{#if rowState.missingRequiredValue}
<div class="mt-3 rounded-xl border border-red-200 bg-red-50 px-3 py-3 text-sm text-red-700">
{$t.dataset_review?.mapping?.missing_required_value}
</div>
{/if}
{#if isEditing}
<div class="mt-4 grid gap-3">
<label class="space-y-1">
<span class="text-sm font-medium text-slate-700">
{$t.dataset_review?.mapping?.manual_value_label}
</span>
<input
class="w-full rounded-xl border border-slate-300 bg-white px-3 py-2 text-sm text-slate-900 shadow-sm outline-none transition focus:border-blue-500 focus:ring-2 focus:ring-blue-200"
bind:value={overrideValueDraft}
disabled={disabled || isSaving}
/>
</label>
<label class="space-y-1">
<span class="text-sm font-medium text-slate-700">
{$t.dataset_review?.mapping?.manual_note_label}
</span>
<textarea
class="min-h-24 w-full rounded-xl border border-slate-300 bg-white px-3 py-2 text-sm text-slate-900 shadow-sm outline-none transition focus:border-blue-500 focus:ring-2 focus:ring-blue-200"
bind:value={overrideNoteDraft}
disabled={disabled || isSaving}
></textarea>
</label>
<div class="flex flex-wrap gap-2">
<button
type="button"
class="inline-flex items-center justify-center rounded-xl bg-blue-600 px-4 py-2 text-sm font-medium text-white transition hover:bg-blue-700 disabled:cursor-not-allowed disabled:bg-slate-300"
onclick={() => saveManualOverride(mapping)}
disabled={disabled || isSaving}
>
{$t.dataset_review?.mapping?.save_override_action}
</button>
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-4 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => cancelManualOverride(mapping.mapping_id)}
disabled={isSaving}
>
{$t.common?.cancel}
</button>
</div>
</div>
{/if}
{#if rowMessage.text || rowMessage.status === "saving"}
<div
class={`mt-4 rounded-xl px-3 py-3 text-sm ${
rowMessage.status === "error"
? "border border-red-200 bg-red-50 text-red-700"
: rowMessage.status === "saving"
? "border border-blue-200 bg-blue-50 text-blue-700"
: "border border-emerald-200 bg-emerald-50 text-emerald-800"
}`}
>
{rowMessage.status === "saving"
? $t.dataset_review?.mapping?.messages?.saving
: rowMessage.text}
</div>
{/if}
</div>
<div class="flex shrink-0 flex-wrap gap-2 xl:w-56 xl:flex-col">
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-4 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => startManualOverride(mapping)}
disabled={disabled || isSaving}
>
{$t.dataset_review?.mapping?.manual_override_action}
</button>
<button
type="button"
class="inline-flex items-center justify-center rounded-xl bg-amber-600 px-4 py-2 text-sm font-medium text-white transition hover:bg-amber-700 disabled:cursor-not-allowed disabled:bg-slate-300"
onclick={() => approveMapping(mapping)}
disabled={disabled || isSaving || !rowState.needsExplicitApproval}
>
{$t.dataset_review?.mapping?.approve_action}
</button>
</div>
</div>
</article>
{/each}
</div>
{/if}
</section>
<!-- [/DEF:ExecutionMappingReview:Component] -->

View File

@@ -0,0 +1,372 @@
<!-- [DEF:LaunchConfirmationPanel:Component] -->
<!-- @COMPLEXITY: 4 -->
<!-- @SEMANTICS: dataset-review, launch-confirmation, run-gates, sql-lab, audited-execution -->
<!-- @PURPOSE: Summarize final execution context, surface launch blockers explicitly, and confirm only a gate-complete SQL Lab launch request. -->
<!-- @LAYER: UI -->
<!-- @RELATION: [BINDS_TO] ->[api_module] -->
<!-- @RELATION: [BINDS_TO] ->[DatasetReviewWorkspace] -->
<!-- @PRE: Session detail, mappings, findings, preview state, and latest run context belong to the current ownership-scoped session payload. -->
<!-- @POST: Users can see why launch is blocked or confirm a run-ready launch with explicit SQL Lab handoff evidence. -->
<!-- @SIDE_EFFECT: Submits the launch request through dataset orchestration APIs and updates the workspace with returned run context state. -->
<!-- @UX_STATE: Blocked -> Explicit blocker list prevents hidden bypass of approvals, readiness, or preview-fingerprint gates. -->
<!-- @UX_STATE: Ready -> Final reviewed run context is visible before confirmation. -->
<!-- @UX_STATE: Submitted -> SQL Lab handoff and audited run context reference are shown after launch request succeeds. -->
<!-- @UX_FEEDBACK: Launch button, blocker list, and success state all reflect current gate truth instead of generic confirmation copy. -->
<!-- @UX_RECOVERY: Blocked launch state provides jump paths back to mapping review, preview generation, or validation remediation. -->
<!-- @UX_REACTIVITY: Uses $props, $state, and $derived only; no legacy reactive syntax. -->
<script>
import { t } from "$lib/i18n";
import { requestApi } from "$lib/api.js";
let {
sessionId = "",
session = null,
findings = [],
mappings = [],
preview = null,
previewState = "",
latestRunContext = null,
disabled = false,
onupdated = () => {},
onjump = () => {},
} = $props();
let launchStatus = $state("");
let launchMessage = $state("");
let launchResult = $state(null);
const activeRunContext = $derived(launchResult?.run_context || latestRunContext || null);
const effectivePreviewState = $derived(
String(previewState || preview?.preview_status || "missing"),
);
const approvedMappingsCount = $derived(
(mappings || []).filter((mapping) => mapping.approval_state === "approved").length,
);
const launchBlockers = $derived(buildLaunchBlockers());
const panelState = $derived(
activeRunContext ? "Submitted" : launchBlockers.length > 0 ? "Blocked" : "Ready",
);
const launchButtonLabel = $derived(
launchStatus === "saving"
? $t.dataset_review?.launch?.launch_loading
: $t.dataset_review?.launch?.launch_action,
);
function buildLaunchBlockers() {
const blockers = [];
for (const finding of findings || []) {
if (
finding.severity === "blocking" &&
finding.resolution_state !== "resolved" &&
finding.resolution_state !== "approved"
) {
blockers.push({
label:
$t.dataset_review?.launch?.blockers?.blocking_finding ||
"Blocking findings remain unresolved",
detail: finding.title || finding.message || finding.code,
target: "summary",
});
}
}
for (const mapping of mappings || []) {
if (mapping.requires_explicit_approval && mapping.approval_state !== "approved") {
blockers.push({
label:
$t.dataset_review?.launch?.blockers?.mapping_approval_required ||
"Mapping approval is still required",
detail: mapping.transformation_note || mapping.mapping_id,
target: "mapping",
});
}
}
if (effectivePreviewState === "missing") {
blockers.push({
label:
$t.dataset_review?.launch?.blockers?.preview_missing ||
"Superset preview is required before launch",
detail: "",
target: "sql-preview",
});
} else if (effectivePreviewState === "pending") {
blockers.push({
label:
$t.dataset_review?.launch?.blockers?.preview_pending ||
"Preview generation is still in progress",
detail: "",
target: "sql-preview",
});
} else if (effectivePreviewState === "stale") {
blockers.push({
label:
$t.dataset_review?.launch?.blockers?.preview_stale ||
"Preview no longer matches the current execution inputs",
detail: "",
target: "sql-preview",
});
} else if (effectivePreviewState === "failed") {
blockers.push({
label:
$t.dataset_review?.launch?.blockers?.preview_failed ||
"Preview failed and launch remains blocked",
detail: preview?.error_details || preview?.error_code || "",
target: "sql-preview",
});
}
if (!activeRunContext && String(session?.readiness_state || "") !== "run_ready") {
blockers.push({
label:
$t.dataset_review?.launch?.blockers?.readiness_not_run_ready ||
"Session is not yet in run-ready state",
detail: session?.readiness_state || "",
target: "next-action",
});
}
if (!activeRunContext && !preview?.preview_fingerprint) {
blockers.push({
label:
$t.dataset_review?.launch?.blockers?.preview_fingerprint_missing ||
"Preview fingerprint is missing, so launch cannot be trusted",
detail: "",
target: "sql-preview",
});
}
return blockers;
}
function getStateLabel(state) {
return $t.dataset_review?.launch?.state?.[state] || state;
}
function getStateClass(state) {
if (state === "Submitted") {
return "bg-emerald-100 text-emerald-700";
}
if (state === "Ready") {
return "bg-blue-100 text-blue-700";
}
return "bg-red-100 text-red-700";
}
function jumpTo(target) {
onjump({ target });
}
async function confirmLaunch() {
if (!sessionId || disabled || panelState !== "Ready" || launchStatus === "saving") {
return;
}
launchStatus = "saving";
launchMessage = "";
try {
const result = await requestApi(`/dataset-orchestration/sessions/${sessionId}/launch`, "POST");
launchResult = result;
launchStatus = "success";
launchMessage = $t.dataset_review?.launch?.messages?.launch_started;
onupdated({
launch_result: result,
preview_state: "unchanged",
});
} catch (error) {
launchStatus = "error";
launchMessage =
error?.message ||
$t.dataset_review?.launch?.messages?.launch_failed ||
$t.common?.error;
}
}
</script>
<section id="launch" class="rounded-2xl border border-slate-200 bg-white p-5 shadow-sm">
<div class="flex flex-col gap-3 md:flex-row md:items-start md:justify-between">
<div>
<p class="text-xs font-semibold uppercase tracking-wide text-slate-500">
{$t.dataset_review?.launch?.eyebrow}
</p>
<h2 class="text-xl font-semibold text-slate-900">
{$t.dataset_review?.launch?.title}
</h2>
<p class="mt-1 max-w-3xl text-sm text-slate-600">
{$t.dataset_review?.launch?.description}
</p>
</div>
<span class={`rounded-full px-3 py-1 text-xs font-medium ${getStateClass(panelState)}`}>
{$t.dataset_review?.launch?.state_label}: {getStateLabel(panelState)}
</span>
</div>
{#if panelState === "Blocked"}
<div class="mt-5 rounded-xl border border-red-200 bg-red-50 p-4">
<h3 class="text-sm font-semibold text-red-800">
{$t.dataset_review?.launch?.blocked_title}
</h3>
<p class="mt-1 text-sm text-red-700">
{$t.dataset_review?.launch?.blocked_body}
</p>
<div class="mt-4 space-y-3">
{#each launchBlockers as blocker}
<div class="rounded-xl border border-red-200 bg-white p-3">
<div class="text-sm font-medium text-slate-900">{blocker.label}</div>
{#if blocker.detail}
<div class="mt-1 break-all text-xs text-slate-600">{blocker.detail}</div>
{/if}
<button
type="button"
class="mt-3 inline-flex items-center justify-center rounded-lg border border-slate-300 bg-white px-3 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50"
onclick={() => jumpTo(blocker.target)}
>
{$t.dataset_review?.launch?.resolve_action}
</button>
</div>
{/each}
</div>
</div>
{/if}
{#if panelState === "Ready"}
<div class="mt-5 space-y-4">
<div class="rounded-xl border border-slate-200 bg-slate-50 p-4">
<div class="grid gap-3 md:grid-cols-2">
<div>
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.launch?.dataset_ref_label}
</div>
<div class="mt-1 text-sm font-medium text-slate-900">
{session?.dataset_ref || ($t.common?.unknown || "unknown")}
</div>
</div>
<div>
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.launch?.readiness_label}
</div>
<div class="mt-1 text-sm font-medium text-slate-900">
{session?.readiness_state || ($t.common?.unknown || "unknown")}
</div>
</div>
<div>
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.launch?.approved_mappings_label}
</div>
<div class="mt-1 text-sm font-medium text-slate-900">
{approvedMappingsCount}
</div>
</div>
<div>
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.launch?.preview_fingerprint_label}
</div>
<div class="mt-1 break-all text-sm font-medium text-slate-900">
{preview?.preview_fingerprint || ($t.common?.not_available || "N/A")}
</div>
</div>
</div>
</div>
<div class="rounded-xl border border-slate-200 bg-slate-50 p-4">
<h3 class="text-sm font-semibold text-slate-900">
{$t.dataset_review?.launch?.sql_lab_target_title}
</h3>
<p class="mt-1 text-sm text-slate-600">
{$t.dataset_review?.launch?.sql_lab_target_body}
</p>
<div class="mt-3 flex flex-wrap gap-2 text-xs text-slate-600">
<span class="rounded-full bg-white px-2 py-1">
{$t.dataset_review?.launch?.preview_status_label}: {preview?.preview_status}
</span>
<span class="rounded-full bg-white px-2 py-1">
{$t.dataset_review?.launch?.compiled_by_label}: {preview?.compiled_by}
</span>
</div>
</div>
<button
type="button"
onclick={confirmLaunch}
disabled={disabled || launchStatus === "saving"}
class="inline-flex w-full items-center justify-center rounded-xl bg-blue-600 px-4 py-3 text-sm font-medium text-white transition hover:bg-blue-700 disabled:cursor-not-allowed disabled:bg-slate-300"
>
{launchButtonLabel}
</button>
</div>
{/if}
{#if panelState === "Submitted"}
<div class="mt-5 space-y-4 rounded-xl border border-emerald-200 bg-emerald-50 p-4">
<div>
<h3 class="text-sm font-semibold text-emerald-900">
{$t.dataset_review?.launch?.submitted_title}
</h3>
<p class="mt-1 text-sm text-emerald-800">
{$t.dataset_review?.launch?.submitted_body}
</p>
</div>
<div class="grid gap-3 md:grid-cols-2">
<div class="rounded-xl border border-emerald-200 bg-white p-3">
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.launch?.run_context_label}
</div>
<div class="mt-1 break-all text-sm font-medium text-slate-900">
{activeRunContext?.run_context_id}
</div>
</div>
<div class="rounded-xl border border-emerald-200 bg-white p-3">
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.launch?.sql_lab_session_label}
</div>
<div class="mt-1 break-all text-sm font-medium text-slate-900">
{activeRunContext?.sql_lab_session_ref}
</div>
</div>
<div class="rounded-xl border border-emerald-200 bg-white p-3">
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.launch?.launch_status_label}
</div>
<div class="mt-1 text-sm font-medium text-slate-900">
{activeRunContext?.launch_status}
</div>
</div>
<div class="rounded-xl border border-emerald-200 bg-white p-3">
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.launch?.preview_ref_label}
</div>
<div class="mt-1 break-all text-sm font-medium text-slate-900">
{activeRunContext?.preview_id}
</div>
</div>
</div>
</div>
{/if}
{#if launchMessage}
<div
class={`mt-4 rounded-xl px-3 py-3 text-sm ${
launchStatus === "error"
? "border border-red-200 bg-red-50 text-red-700"
: launchStatus === "saving"
? "border border-blue-200 bg-blue-50 text-blue-700"
: "border border-emerald-200 bg-emerald-50 text-emerald-800"
}`}
>
{launchMessage}
</div>
{/if}
</section>
<!-- [/DEF:LaunchConfirmationPanel:Component] -->

View File

@@ -0,0 +1,598 @@
<!-- [DEF:SemanticLayerReview:Component] -->
<!-- @COMPLEXITY: 4 -->
<!-- @SEMANTICS: dataset-review, semantic-layer, candidate-review, manual-override, field-lock -->
<!-- @PURPOSE: Surface field-level semantic decisions with provenance, confidence, candidate acceptance, and manual override safeguards for US2 review flow. -->
<!-- @LAYER: UI -->
<!-- @RELATION: [BINDS_TO] ->[api_module] -->
<!-- @RELATION: [BINDS_TO] ->[DatasetReviewWorkspace] -->
<!-- @PRE: Session id is available and semantic field entries come from the current ownership-scoped session detail payload. -->
<!-- @POST: Users can review the current semantic value, accept a candidate, apply manual override, and lock or unlock field state without violating backend provenance rules. -->
<!-- @SIDE_EFFECT: Persists semantic field decisions, lock state, and optional thumbs feedback through dataset orchestration endpoints. -->
<!-- @UX_STATE: Conflicted -> Multiple candidates or review-needed fields remain visible with explicit acceptance actions. -->
<!-- @UX_STATE: Manual -> One field enters local draft mode and persists as locked manual override on save. -->
<!-- @UX_FEEDBACK: Save, lock, unlock, and feedback actions expose inline success or error state on the affected field. -->
<!-- @UX_RECOVERY: Users can cancel local edits, unlock a manual override for re-review, or retry failed mutations in place. -->
<!-- @UX_REACTIVITY: Uses $props, $state, and $derived only; no legacy reactive syntax. -->
<script>
import { t } from "$lib/i18n";
import { requestApi } from "$lib/api.js";
let {
sessionId = "",
fields = [],
semanticSources = [],
disabled = false,
onupdated = () => {},
} = $props();
let editingFieldId = $state("");
let draftVerboseName = $state("");
let draftDescription = $state("");
let draftDisplayFormat = $state("");
let savingFieldId = $state("");
let batchSaving = $state(false);
let fieldMessages = $state({});
const sortedFields = $derived(
[...(fields || [])].sort((left, right) => {
const leftPriority = Number(Boolean(left?.needs_review)) + Number(Boolean(left?.has_conflict));
const rightPriority =
Number(Boolean(right?.needs_review)) + Number(Boolean(right?.has_conflict));
if (leftPriority !== rightPriority) {
return rightPriority - leftPriority;
}
return String(left?.field_name || "").localeCompare(String(right?.field_name || ""));
}),
);
const activeFieldCount = $derived(sortedFields.length);
function updateFieldMessage(fieldId, patch) {
fieldMessages = {
...fieldMessages,
[fieldId]: {
...(fieldMessages[fieldId] || {}),
...patch,
},
};
}
function resetFieldMessage(fieldId) {
fieldMessages = {
...fieldMessages,
[fieldId]: {
status: "",
text: "",
},
};
}
function startManualEdit(field) {
editingFieldId = field.field_id;
draftVerboseName = field.verbose_name || "";
draftDescription = field.description || "";
draftDisplayFormat = field.display_format || "";
resetFieldMessage(field.field_id);
}
function cancelManualEdit(fieldId) {
if (editingFieldId !== fieldId) {
return;
}
editingFieldId = "";
draftVerboseName = "";
draftDescription = "";
draftDisplayFormat = "";
resetFieldMessage(fieldId);
}
function getSourceLabel(sourceId) {
const source = (semanticSources || []).find((item) => item.source_id === sourceId);
if (!source) {
return $t.dataset_review?.semantics?.unknown_source;
}
return `${source.display_name}${source.source_version}`;
}
function getCandidateStatusLabel(status) {
const normalized = String(status || "pending");
return $t.dataset_review?.semantics?.candidate_status?.[normalized] || normalized;
}
function getProvenanceLabel(provenance) {
const normalized = String(provenance || "unresolved");
return $t.dataset_review?.semantics?.provenance?.[normalized] || normalized;
}
function getConfidenceLabel(rank) {
if (rank === null || rank === undefined) {
return $t.dataset_review?.semantics?.confidence_unset;
}
return `${$t.dataset_review?.semantics?.confidence_rank_label} #${rank}`;
}
function getCurrentValueSummary(field) {
const parts = [
field.verbose_name || $t.dataset_review?.semantics?.empty_value,
field.description || "",
field.display_format || "",
].filter(Boolean);
return parts.join(" • ");
}
async function persistFieldSemantic(fieldId, payload, successKey) {
if (!sessionId || !fieldId || savingFieldId) {
return;
}
savingFieldId = fieldId;
updateFieldMessage(fieldId, { status: "saving", text: "" });
try {
const updatedField = await requestApi(
`/dataset-orchestration/sessions/${sessionId}/fields/${fieldId}/semantic`,
"PATCH",
payload,
);
updateFieldMessage(fieldId, {
status: "success",
text: $t.dataset_review?.semantics?.messages?.[successKey],
});
if (editingFieldId === fieldId) {
editingFieldId = "";
}
onupdated(updatedField);
} catch (error) {
updateFieldMessage(fieldId, {
status: "error",
text:
error?.message ||
$t.dataset_review?.semantics?.messages?.save_failed ||
$t.common?.error,
});
} finally {
savingFieldId = "";
}
}
async function acceptCandidate(field, candidateId, lockField) {
await persistFieldSemantic(
field.field_id,
{
candidate_id: candidateId,
lock_field: lockField,
},
lockField ? "candidate_locked" : "candidate_applied",
);
}
async function saveManualOverride(field) {
const payload = {
verbose_name: draftVerboseName.trim() || null,
description: draftDescription.trim() || null,
display_format: draftDisplayFormat.trim() || null,
lock_field: true,
};
const hasManualValue =
payload.verbose_name || payload.description || payload.display_format;
if (!hasManualValue) {
updateFieldMessage(field.field_id, {
status: "error",
text: $t.dataset_review?.semantics?.messages?.manual_override_required,
});
return;
}
await persistFieldSemantic(field.field_id, payload, "manual_saved");
}
async function mutateLock(field, action) {
if (!sessionId || !field?.field_id || savingFieldId) {
return;
}
savingFieldId = field.field_id;
updateFieldMessage(field.field_id, { status: "saving", text: "" });
try {
const updatedField = await requestApi(
`/dataset-orchestration/sessions/${sessionId}/fields/${field.field_id}/${action}`,
"POST",
);
updateFieldMessage(field.field_id, {
status: "success",
text:
action === "lock"
? $t.dataset_review?.semantics?.messages?.locked
: $t.dataset_review?.semantics?.messages?.unlocked,
});
onupdated(updatedField);
} catch (error) {
updateFieldMessage(field.field_id, {
status: "error",
text:
error?.message ||
$t.dataset_review?.semantics?.messages?.save_failed ||
$t.common?.error,
});
} finally {
savingFieldId = "";
}
}
async function approveAllCandidates() {
if (!sessionId || batchSaving) {
return;
}
const items = sortedFields
.filter((field) => !field.is_locked && field.candidates?.length)
.map((field) => ({
field_id: field.field_id,
candidate_id: field.candidates[0].candidate_id,
lock_field: false,
}));
if (!items.length) {
return;
}
batchSaving = true;
try {
const updatedFields = await requestApi(
`/dataset-orchestration/sessions/${sessionId}/fields/semantic/approve-batch`,
"POST",
{ items },
);
onupdated({ fields: updatedFields });
} catch (error) {
const message =
error?.message ||
$t.dataset_review?.semantics?.messages?.save_failed ||
$t.common?.error;
for (const item of items) {
updateFieldMessage(item.field_id, { status: "error", text: message });
}
} finally {
batchSaving = false;
}
}
async function recordFeedback(fieldId, feedback) {
if (!sessionId || !fieldId || savingFieldId) {
return;
}
savingFieldId = fieldId;
updateFieldMessage(fieldId, { status: "saving", text: "" });
try {
await requestApi(
`/dataset-orchestration/sessions/${sessionId}/fields/${fieldId}/feedback`,
"POST",
{ feedback },
);
updateFieldMessage(fieldId, {
status: "success",
text:
feedback === "up"
? $t.dataset_review?.semantics?.messages?.feedback_up
: $t.dataset_review?.semantics?.messages?.feedback_down,
});
} catch (error) {
updateFieldMessage(fieldId, {
status: "error",
text:
error?.message ||
$t.dataset_review?.semantics?.messages?.save_failed ||
$t.common?.error,
});
} finally {
savingFieldId = "";
}
}
</script>
<section id="semantics" class="rounded-2xl border border-slate-200 bg-white p-5 shadow-sm">
<div class="flex flex-col gap-3 md:flex-row md:items-start md:justify-between">
<div>
<p class="text-xs font-semibold uppercase tracking-wide text-slate-500">
{$t.dataset_review?.semantics?.eyebrow}
</p>
<h2 class="text-xl font-semibold text-slate-900">
{$t.dataset_review?.semantics?.title}
</h2>
<p class="mt-1 max-w-3xl text-sm text-slate-600">
{$t.dataset_review?.semantics?.description}
</p>
</div>
<div class="flex flex-col items-end gap-2">
<div class="rounded-xl border border-slate-200 bg-slate-50 px-3 py-2 text-right">
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.semantics?.field_count_label}
</div>
<div class="mt-1 text-sm font-medium text-slate-900">{activeFieldCount}</div>
</div>
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-4 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={approveAllCandidates}
disabled={disabled || batchSaving || activeFieldCount === 0}
>
{$t.dataset_review?.semantics?.approve_all_action}
</button>
</div>
</div>
{#if activeFieldCount === 0}
<div class="mt-5 rounded-xl border border-slate-200 bg-slate-50 px-4 py-4 text-sm text-slate-600">
{$t.dataset_review?.semantics?.empty}
</div>
{:else}
<div class="mt-5 space-y-4">
{#each sortedFields as field}
{@const message = fieldMessages[field.field_id] || { status: "", text: "" }}
{@const isEditing = editingFieldId === field.field_id}
{@const isSaving = savingFieldId === field.field_id}
<article
class={`rounded-2xl border p-4 shadow-sm ${
field.has_conflict || field.needs_review
? "border-amber-200 bg-amber-50"
: "border-slate-200 bg-white"
}`}
>
<div class="flex flex-col gap-4 lg:flex-row lg:items-start lg:justify-between">
<div class="min-w-0 flex-1">
<div class="flex flex-wrap items-center gap-2">
<h3 class="text-base font-semibold text-slate-900">{field.field_name}</h3>
<span class="rounded-full bg-slate-100 px-2 py-0.5 text-xs text-slate-700">
{field.field_kind}
</span>
<span
class={`rounded-full px-2 py-0.5 text-xs ${
field.is_locked
? "bg-blue-100 text-blue-700"
: "bg-slate-100 text-slate-700"
}`}
>
{field.is_locked
? $t.dataset_review?.semantics?.locked_badge
: $t.dataset_review?.semantics?.unlocked_badge}
</span>
{#if field.has_conflict}
<span class="rounded-full bg-amber-100 px-2 py-0.5 text-xs text-amber-800">
{$t.dataset_review?.semantics?.conflict_badge}
</span>
{/if}
{#if field.needs_review}
<span class="rounded-full bg-amber-100 px-2 py-0.5 text-xs text-amber-800">
{$t.dataset_review?.semantics?.needs_review_badge}
</span>
{/if}
</div>
<div class="mt-3 rounded-xl border border-slate-200 bg-slate-50 p-3">
<div class="text-xs uppercase tracking-wide text-slate-500">
{$t.dataset_review?.semantics?.active_value_label}
</div>
<div class="mt-1 text-sm font-medium text-slate-900">
{getCurrentValueSummary(field)}
</div>
<div class="mt-2 flex flex-wrap gap-2 text-xs text-slate-600">
<span class="rounded-full bg-white px-2 py-1">
{$t.dataset_review?.semantics?.provenance_label}:
{getProvenanceLabel(field.provenance)}
</span>
<span class="rounded-full bg-white px-2 py-1">
{$t.dataset_review?.semantics?.confidence_label}:
{getConfidenceLabel(field.confidence_rank)}
</span>
{#if field.source_id}
<span class="rounded-full bg-white px-2 py-1">
{$t.dataset_review?.semantics?.source_label}: {getSourceLabel(field.source_id)}
</span>
{/if}
<span class="rounded-full bg-white px-2 py-1">
{$t.dataset_review?.semantics?.changed_by_label}: {field.last_changed_by}
</span>
</div>
</div>
{#if isEditing}
<div class="mt-4 grid gap-3">
<label class="space-y-1">
<span class="text-sm font-medium text-slate-700">
{$t.dataset_review?.semantics?.manual_verbose_name_label}
</span>
<input
class="w-full rounded-xl border border-slate-300 bg-white px-3 py-2 text-sm text-slate-900 shadow-sm outline-none transition focus:border-blue-500 focus:ring-2 focus:ring-blue-200"
bind:value={draftVerboseName}
disabled={isSaving || disabled}
/>
</label>
<label class="space-y-1">
<span class="text-sm font-medium text-slate-700">
{$t.dataset_review?.semantics?.manual_description_label}
</span>
<textarea
class="min-h-24 w-full rounded-xl border border-slate-300 bg-white px-3 py-2 text-sm text-slate-900 shadow-sm outline-none transition focus:border-blue-500 focus:ring-2 focus:ring-blue-200"
bind:value={draftDescription}
disabled={isSaving || disabled}
></textarea>
</label>
<label class="space-y-1">
<span class="text-sm font-medium text-slate-700">
{$t.dataset_review?.semantics?.manual_display_format_label}
</span>
<input
class="w-full rounded-xl border border-slate-300 bg-white px-3 py-2 text-sm text-slate-900 shadow-sm outline-none transition focus:border-blue-500 focus:ring-2 focus:ring-blue-200"
bind:value={draftDisplayFormat}
disabled={isSaving || disabled}
/>
</label>
<div class="flex flex-wrap gap-2">
<button
type="button"
class="inline-flex items-center justify-center rounded-xl bg-blue-600 px-4 py-2 text-sm font-medium text-white transition hover:bg-blue-700 disabled:cursor-not-allowed disabled:bg-slate-300"
onclick={() => saveManualOverride(field)}
disabled={disabled || isSaving}
>
{$t.dataset_review?.semantics?.save_manual_action}
</button>
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-4 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => cancelManualEdit(field.field_id)}
disabled={isSaving}
>
{$t.common?.cancel}
</button>
</div>
</div>
{/if}
<div class="mt-4">
<div class="flex items-center justify-between gap-2">
<h4 class="text-sm font-semibold text-slate-900">
{$t.dataset_review?.semantics?.candidates_title}
</h4>
<span class="rounded-full bg-slate-100 px-2.5 py-1 text-xs text-slate-700">
{field.candidates?.length || 0}
</span>
</div>
{#if !field.candidates?.length}
<p class="mt-2 text-sm text-slate-600">
{$t.dataset_review?.semantics?.candidates_empty}
</p>
{:else}
<div class="mt-3 space-y-3">
{#each field.candidates as candidate}
<div class="rounded-xl border border-slate-200 bg-white p-3">
<div class="flex flex-col gap-3 lg:flex-row lg:items-start lg:justify-between">
<div class="min-w-0 flex-1">
<div class="flex flex-wrap items-center gap-2">
<div class="text-sm font-medium text-slate-900">
{candidate.proposed_verbose_name ||
$t.dataset_review?.semantics?.empty_value}
</div>
<span class="rounded-full bg-slate-100 px-2 py-0.5 text-xs text-slate-700">
{candidate.match_type}
</span>
<span class="rounded-full bg-slate-100 px-2 py-0.5 text-xs text-slate-700">
{$t.dataset_review?.semantics?.score_label}:
{candidate.confidence_score}
</span>
<span class="rounded-full bg-slate-100 px-2 py-0.5 text-xs text-slate-700">
{getCandidateStatusLabel(candidate.status)}
</span>
</div>
<p class="mt-2 text-sm text-slate-700">
{candidate.proposed_description ||
$t.dataset_review?.semantics?.candidate_description_empty}
</p>
{#if candidate.proposed_display_format}
<p class="mt-1 text-xs text-slate-500">
{$t.dataset_review?.semantics?.display_format_label}:
{candidate.proposed_display_format}
</p>
{/if}
</div>
<div class="flex shrink-0 flex-wrap gap-2">
<button
type="button"
class="inline-flex items-center justify-center rounded-lg border border-slate-300 px-3 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => acceptCandidate(field, candidate.candidate_id, false)}
disabled={disabled || isSaving}
>
{$t.dataset_review?.semantics?.apply_candidate_action}
</button>
<button
type="button"
class="inline-flex items-center justify-center rounded-lg bg-blue-600 px-3 py-2 text-sm font-medium text-white transition hover:bg-blue-700 disabled:cursor-not-allowed disabled:bg-slate-300"
onclick={() => acceptCandidate(field, candidate.candidate_id, true)}
disabled={disabled || isSaving}
>
{$t.dataset_review?.semantics?.apply_and_lock_action}
</button>
</div>
</div>
</div>
{/each}
</div>
{/if}
</div>
{#if message.text || message.status === "saving"}
<div
class={`mt-4 rounded-xl px-3 py-3 text-sm ${
message.status === "error"
? "border border-red-200 bg-red-50 text-red-700"
: message.status === "saving"
? "border border-blue-200 bg-blue-50 text-blue-700"
: "border border-emerald-200 bg-emerald-50 text-emerald-800"
}`}
>
{message.status === "saving"
? $t.dataset_review?.semantics?.messages?.saving
: message.text}
</div>
{/if}
</div>
<div class="flex shrink-0 flex-wrap gap-2 lg:w-52 lg:flex-col">
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-4 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => startManualEdit(field)}
disabled={disabled || isSaving}
>
{$t.dataset_review?.semantics?.manual_override_action}
</button>
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-4 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => mutateLock(field, field.is_locked ? "unlock" : "lock")}
disabled={disabled || isSaving}
>
{field.is_locked
? $t.dataset_review?.semantics?.unlock_action
: $t.dataset_review?.semantics?.lock_action}
</button>
<div class="grid grid-cols-2 gap-2">
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-3 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => recordFeedback(field.field_id, 'up')}
disabled={disabled || isSaving}
>
{$t.dataset_review?.semantics?.feedback_up_action}
</button>
<button
type="button"
class="inline-flex items-center justify-center rounded-xl border border-slate-300 px-3 py-2 text-sm font-medium text-slate-700 transition hover:bg-slate-50 disabled:cursor-not-allowed disabled:opacity-50"
onclick={() => recordFeedback(field.field_id, 'down')}
disabled={disabled || isSaving}
>
{$t.dataset_review?.semantics?.feedback_down_action}
</button>
</div>
</div>
</div>
</article>
{/each}
</div>
{/if}
</section>
<!-- [/DEF:SemanticLayerReview:Component] -->

View File

@@ -0,0 +1,412 @@
/**
* @vitest-environment jsdom
*/
// @ts-nocheck
// [DEF:DatasetReviewUs2UxTests:Module]
// @COMPLEXITY: 3
// @SEMANTICS: dataset-review, semantics, clarification, ux-tests, field-review
// @PURPOSE: Verify US2 semantic field review and one-question clarification UX contracts.
// @LAYER: UI
// @RELATION: [VERIFIES] ->[SemanticLayerReview]
// @RELATION: [VERIFIES] ->[ClarificationDialog]
// @UX_STATE: Conflicted -> Semantic candidates and manual override affordances remain visible.
// @UX_STATE: Question -> Clarification shows one active question, why-it-matters context, and resumable actions.
// @TEST_CONTRACT: Us2DatasetReviewProps -> ObservableSemanticAndClarificationUx
// @TEST_SCENARIO: semantic_review_surfaces_active_value_and_candidates -> Field-level semantic review shows provenance, candidates, and mutation callbacks.
// @TEST_SCENARIO: clarification_dialog_shows_active_question_and_actions -> Clarification dialog renders one active question with answer and escalation actions.
// @TEST_EDGE: missing_field -> Missing optional values still keep semantic summary readable.
// @TEST_EDGE: external_fail -> Failed request surfaces inline recovery message.
// @TEST_INVARIANT: us2_components_remain_actionable -> VERIFIED_BY: [semantic_review_surfaces_active_value_and_candidates, clarification_dialog_shows_active_question_and_actions]
import { describe, it, expect, vi, beforeEach } from "vitest";
import { render, screen, fireEvent, waitFor } from "@testing-library/svelte";
import SemanticLayerReview from "../SemanticLayerReview.svelte";
import ClarificationDialog from "../ClarificationDialog.svelte";
import { requestApi } from "$lib/api.js";
import { openAssistantChat } from "$lib/stores/assistantChat.js";
const mockedRequestApi = /** @type {any} */ (requestApi);
const mockedOpenAssistantChat = /** @type {any} */ (openAssistantChat);
vi.mock("$lib/api.js", () => ({
requestApi: vi.fn(),
}));
vi.mock("$lib/stores/assistantChat.js", () => ({
openAssistantChat: vi.fn(),
}));
vi.mock("$lib/i18n", () => ({
t: {
subscribe: (fn) => {
fn({
common: {
error: "Common error",
cancel: "Cancel",
},
dataset_review: {
semantics: {
eyebrow: "Semantic layer",
title: "Review semantic field values",
description: "Review semantic candidates",
field_count_label: "Fields in review",
empty: "No semantic fields",
unknown_source: "Unknown source",
active_value_label: "Active semantic value",
provenance_label: "Provenance",
confidence_label: "Confidence",
confidence_unset: "No confidence rank",
confidence_rank_label: "Rank",
source_label: "Source",
changed_by_label: "Changed by",
locked_badge: "Locked",
unlocked_badge: "Unlocked",
conflict_badge: "Conflict",
needs_review_badge: "Needs review",
manual_verbose_name_label: "Manual verbose name",
manual_description_label: "Manual description",
manual_display_format_label: "Manual display format",
manual_override_action: "Manual override",
save_manual_action: "Save manual value",
lock_action: "Lock field",
unlock_action: "Unlock field",
feedback_up_action: "Thumbs up",
feedback_down_action: "Thumbs down",
candidates_title: "Candidate options",
candidates_empty: "No candidates",
candidate_description_empty: "No candidate description",
display_format_label: "Display format",
score_label: "Score",
apply_candidate_action: "Apply",
apply_and_lock_action: "Apply and lock",
empty_value: "No value",
provenance: {
unresolved: "Unresolved",
dictionary_exact: "Dictionary exact match",
manual_override: "Manual override",
},
candidate_status: {
pending: "Pending",
accepted: "Accepted",
superseded: "Superseded",
},
messages: {
saving: "Saving semantic decision...",
save_failed: "Failed to save semantic decision.",
manual_override_required: "Manual value required",
manual_saved: "Manual semantic value saved and locked.",
candidate_applied: "Candidate value applied.",
candidate_locked: "Candidate value applied and locked.",
locked: "Field locked against silent overwrite.",
unlocked: "Field unlocked for renewed review.",
feedback_up: "Positive feedback recorded.",
feedback_down: "Negative feedback recorded.",
},
},
clarification: {
eyebrow: "Clarification",
title: "Resolve one ambiguity at a time",
description: "One question at a time",
progress_label: "Clarification progress",
state_label: "Dialog state",
status_label: "Status",
remaining_label: "Remaining questions",
summary_delta_label: "Summary delta",
active_question_label: "Active question",
why_it_matters_label: "Why it matters",
current_guess_label: "Current guess",
current_guess_empty: "No current guess",
topic_label: "Topic",
options_title: "Available options",
recommended_badge: "Recommended",
custom_answer_label: "Custom answer",
answer_action: "Answer with selected option",
custom_answer_action: "Submit custom answer",
skip_action: "Skip for now",
assistant_action: "Open assistant guidance",
expert_review_action: "Send to expert review",
resume_action: "Resume clarification",
completed: "No active clarification question remains.",
feedback_label: "Clarification feedback",
feedback_prompt: "Record whether the clarification result was useful.",
feedback_up_action: "Helpful",
feedback_down_action: "Not helpful",
messages: {
saving: "Saving clarification answer...",
saved: "Clarification answer saved.",
skipped: "Question skipped and left resumable.",
expert_review_requested: "Question escalated to expert review.",
assistant_opened: "Assistant guidance opened.",
resumed: "Clarification resumed.",
resume_failed: "Failed to resume clarification.",
save_failed: "Failed to save clarification answer.",
feedback_up: "Positive clarification feedback recorded.",
feedback_down: "Negative clarification feedback recorded.",
feedback_failed: "Failed to save clarification feedback.",
option_required: "Option required",
custom_required: "Custom answer required",
},
},
},
});
return () => {};
},
},
}));
describe("Dataset review US2 component UX", () => {
beforeEach(() => {
vi.clearAllMocks();
});
it("semantic_review_surfaces_active_value_and_candidates", async () => {
requestApi.mockResolvedValue({
field_id: "field-1",
field_name: "customer_name",
field_kind: "dimension",
verbose_name: "Customer name",
description: "Customer display name",
display_format: "text",
provenance: "dictionary_exact",
source_id: "source-1",
confidence_rank: 1,
is_locked: false,
has_conflict: false,
needs_review: false,
last_changed_by: "user",
candidates: [],
});
const onupdated = vi.fn();
render(SemanticLayerReview, {
sessionId: "session-1",
semanticSources: [
{
source_id: "source-1",
display_name: "Trusted dictionary",
source_version: "v1",
},
],
fields: [
{
field_id: "field-1",
field_name: "customer_name",
field_kind: "dimension",
verbose_name: "Customer name",
description: "Resolved from trusted dictionary",
display_format: "text",
provenance: "dictionary_exact",
source_id: "source-1",
confidence_rank: 1,
is_locked: false,
has_conflict: true,
needs_review: true,
last_changed_by: "system",
candidates: [
{
candidate_id: "candidate-1",
match_type: "exact",
confidence_score: 0.98,
proposed_verbose_name: "Customer name",
proposed_description: "Customer display name",
proposed_display_format: "text",
status: "pending",
},
],
},
],
onupdated,
});
expect(screen.getByText("Review semantic field values")).toBeDefined();
expect(screen.getByText("customer_name")).toBeDefined();
expect(screen.getByText("Conflict")).toBeDefined();
expect(screen.getByText("Needs review")).toBeDefined();
expect(screen.getByText("Candidate options")).toBeDefined();
expect(screen.getByText(/Trusted dictionary/)).toBeDefined();
expect(screen.getByText(/v1/)).toBeDefined();
await fireEvent.click(screen.getByRole("button", { name: "Apply" }));
await waitFor(() => {
expect(requestApi).toHaveBeenCalledWith(
"/dataset-orchestration/sessions/session-1/fields/field-1/semantic",
"PATCH",
{
candidate_id: "candidate-1",
lock_field: false,
},
);
});
expect(onupdated).toHaveBeenCalled();
expect(screen.getByText("Candidate value applied.")).toBeDefined();
});
it("clarification_dialog_shows_active_question_and_actions", async () => {
requestApi.mockResolvedValue({
clarification_state: {
clarification_session: {
clarification_session_id: "clarification-1",
session_id: "session-1",
status: "active",
current_question_id: "question-2",
resolved_count: 2,
remaining_count: 1,
summary_delta: "Updated summary",
},
current_question: {
question_id: "question-2",
clarification_session_id: "clarification-1",
topic_ref: "profile.summary",
question_text: "Which label should reviewers see?",
why_it_matters: "This label is reused in the workspace summary.",
current_guess: "Customer name",
options: [
{
option_id: "option-1",
value: "Customer name",
label: "Customer name",
is_recommended: true,
},
],
},
},
session: {
session_id: "session-1",
readiness_state: "clarification_active",
recommended_action: "answer_next_question",
},
changed_findings: [],
});
const onupdated = vi.fn();
render(ClarificationDialog, {
sessionId: "session-1",
clarificationState: {
clarification_session: {
clarification_session_id: "clarification-1",
session_id: "session-1",
status: "active",
current_question_id: "question-1",
resolved_count: 1,
remaining_count: 2,
summary_delta: "Awaiting confirmation",
},
current_question: {
question_id: "question-1",
clarification_session_id: "clarification-1",
topic_ref: "profile.summary",
question_text: "Which customer label should be used?",
why_it_matters: "This label is shown to reviewers.",
current_guess: "Customer name",
options: [
{
option_id: "option-1",
value: "Customer name",
label: "Customer name",
is_recommended: true,
},
],
},
},
onupdated,
});
expect(screen.getByText("Resolve one ambiguity at a time")).toBeDefined();
expect(screen.getByText("Which customer label should be used?")).toBeDefined();
expect(screen.getByText("This label is shown to reviewers.")).toBeDefined();
expect(screen.getAllByText("Customer name").length).toBeGreaterThan(0);
expect(screen.getByText("Recommended")).toBeDefined();
await fireEvent.click(
screen.getByRole("button", { name: "Open assistant guidance" }),
);
expect(openAssistantChat).toHaveBeenCalledTimes(1);
expect(screen.getByText("Assistant guidance opened.")).toBeDefined();
await fireEvent.click(
screen.getByRole("button", { name: "Answer with selected option" }),
);
await waitFor(() => {
expect(requestApi).toHaveBeenCalledWith(
"/dataset-orchestration/sessions/session-1/clarification/answers",
"POST",
{
question_id: "question-1",
answer_kind: "selected",
answer_value: "Customer name",
},
);
});
expect(onupdated).toHaveBeenCalled();
expect(screen.getByText("Clarification answer saved.")).toBeDefined();
mockedRequestApi.mockResolvedValueOnce({
target_id: "question-1",
feedback: "up",
});
await fireEvent.click(screen.getByRole("button", { name: "Helpful" }));
await waitFor(() => {
expect(requestApi).toHaveBeenCalledWith(
"/dataset-orchestration/sessions/session-1/clarification/questions/question-1/feedback",
"POST",
{ feedback: "up" },
);
});
expect(
screen.getByText("Positive clarification feedback recorded."),
).toBeDefined();
});
it("clarification_dialog_external_fail_surfaces_inline_error", async () => {
requestApi.mockRejectedValue(new Error("Clarification backend unavailable"));
render(ClarificationDialog, {
sessionId: "session-1",
clarificationState: {
clarification_session: {
clarification_session_id: "clarification-1",
session_id: "session-1",
status: "active",
current_question_id: "question-1",
resolved_count: 1,
remaining_count: 2,
},
current_question: {
question_id: "question-1",
clarification_session_id: "clarification-1",
topic_ref: "profile.summary",
question_text: "Which customer label should be used?",
why_it_matters: "This label is shown to reviewers.",
current_guess: "Customer name",
options: [
{
option_id: "option-1",
value: "Customer name",
label: "Customer name",
is_recommended: true,
},
],
},
},
});
await fireEvent.click(
screen.getByRole("button", { name: "Answer with selected option" }),
);
await waitFor(() => {
expect(screen.getByText("Clarification backend unavailable")).toBeDefined();
});
});
});
// [/DEF:DatasetReviewUs2UxTests:Module]

View File

@@ -0,0 +1,499 @@
/**
* @vitest-environment jsdom
*/
// @ts-nocheck
// [DEF:DatasetReviewUs3UxTests:Module]
// @COMPLEXITY: 3
// @SEMANTICS: dataset-review, execution, mapping, preview, launch, ux-tests
// @PURPOSE: Verify US3 mapping review, Superset preview, and launch confirmation UX contracts.
// @LAYER: UI
// @RELATION: [VERIFIES] ->[ExecutionMappingReview]
// @RELATION: [VERIFIES] ->[CompiledSQLPreview]
// @RELATION: [VERIFIES] ->[LaunchConfirmationPanel]
// @UX_STATE: WarningApproval -> Mapping review requires explicit approval or manual override.
// @UX_STATE: Ready -> Preview and launch panels expose reviewed Superset-backed execution context.
// @UX_STATE: Blocked -> Launch panel lists blockers instead of allowing hidden bypass.
// @TEST_CONTRACT: Us3ExecutionProps -> ObservableExecutionUx
// @TEST_SCENARIO: mapping_review_approves_warning_sensitive_row -> Approval persists and keeps blockers visible until cleared.
// @TEST_SCENARIO: preview_panel_requests_superset_compilation_and_renders_sql -> Preview stays explicitly Superset-derived.
// @TEST_SCENARIO: launch_panel_blocks_then_submits_sql_lab_launch -> Launch lists gates first and shows audited handoff after success.
// @TEST_EDGE: missing_field -> Missing preview or run context remains readable.
// @TEST_EDGE: external_fail -> Failed preview request surfaces inline error.
// @TEST_EDGE: invalid_type -> Mixed values stringify without crashing.
// @TEST_INVARIANT: execution_gates_remain_visible -> VERIFIED_BY: [mapping_review_approves_warning_sensitive_row, preview_panel_requests_superset_compilation_and_renders_sql, launch_panel_blocks_then_submits_sql_lab_launch]
import { beforeEach, describe, expect, it, vi } from "vitest";
import { fireEvent, render, screen, waitFor } from "@testing-library/svelte";
import { requestApi } from "$lib/api.js";
import ExecutionMappingReview from "../ExecutionMappingReview.svelte";
import CompiledSQLPreview from "../CompiledSQLPreview.svelte";
import LaunchConfirmationPanel from "../LaunchConfirmationPanel.svelte";
vi.mock("$lib/api.js", () => ({
requestApi: vi.fn(),
}));
vi.mock("$lib/i18n", () => ({
t: {
subscribe: (fn) => {
fn({
common: {
error: "Common error",
cancel: "Cancel",
not_available: "N/A",
unknown: "Unknown",
},
dataset_review: {
mapping: {
eyebrow: "Template mapping",
title: "Review filter-to-template mappings",
description: "Verify imported filter values, effective execution values, and warning-sensitive transformations before preview or launch.",
state_label: "Mapping state",
state: {
Incomplete: "Incomplete",
WarningApproval: "Approval required",
Approved: "Approved",
},
pending_approvals_label: "Pending approvals",
required_values_label: "Missing required values",
empty: "No execution mappings are available yet.",
required_blockers_notice: "{count} required values still need attention before preview or launch can proceed.",
approval_notice: "{count} mapping transformations still need explicit approval.",
to_variable_label: "To variable",
required_badge: "Required",
approval_required_badge: "Approval required",
approved_badge: "Approved",
source_filter_label: "Source filter",
source_value_label: "Source value",
confidence_label: "Confidence",
recovery_label: "Recovery",
effective_value_label: "Effective value",
method_label: "Method",
approval_label: "Approval",
warning_label: "Warning",
transformation_note_label: "Transformation note",
missing_required_value: "This mapping still lacks a required effective value.",
manual_value_label: "Manual effective value",
manual_note_label: "Manual override note",
save_override_action: "Save manual override",
manual_override_action: "Manual override",
approve_action: "Approve mapping",
approve_all_action: "Approve all pending mappings",
approval_state: {
pending: "Pending",
approved: "Approved",
rejected: "Rejected",
not_required: "Not required",
},
warning_level: {
none: "None",
low: "Low",
medium: "Medium",
high: "High",
},
method: {
direct_match: "Direct match",
heuristic_match: "Heuristic match",
semantic_match: "Semantic match",
manual_override: "Manual override",
},
messages: {
saving: "Saving mapping decision...",
approval_saved: "Mapping approval recorded.",
approval_failed: "Failed to save mapping approval.",
override_saved: "Manual mapping override saved. Preview should be refreshed.",
override_failed: "Failed to save manual mapping override.",
required_value_missing: "Provide a required value before saving the manual override.",
},
},
preview: {
eyebrow: "Compiled SQL preview",
title: "Review Superset-compiled SQL",
description: "Preview truth comes only from Superset. Regenerate the preview whenever mappings or required inputs change.",
state_label: "Preview state",
state: {
missing: "Missing",
pending: "Pending",
ready: "Ready",
stale: "Stale",
failed: "Error",
},
generate_action: "Generate SQL preview",
generate_loading: "Generating preview...",
missing_body: "No Superset preview is available yet. Generate one before attempting launch.",
pending_body: "Superset is compiling the current execution context. Launch stays blocked until preview completes.",
ready_body: "This SQL preview was compiled by Superset for the current execution inputs.",
stale_body: "Mappings or effective values changed after the last successful preview. Regenerate before launch.",
error_body: "Superset could not compile the current execution context.",
compiler_label: "Compiled source",
compiled_by_superset: "Compiled by Superset",
compiled_source_unknown: "Compilation source unavailable",
fingerprint_label: "Preview fingerprint",
compiled_at_label: "Compiled at",
sql_block_title: "Superset SQL",
compiled_truth_note: "Exact SQL returned by Superset",
go_to_mapping_action: "Review mapping inputs",
review_inputs_action: "Review changed inputs",
messages: {
generated: "Superset preview refreshed.",
generate_failed: "Failed to generate Superset preview.",
},
},
launch: {
eyebrow: "Launch confirmation",
title: "Confirm SQL Lab launch",
description: "Launch remains blocked until preview truth, approvals, and readiness gates all match the reviewed execution context.",
state_label: "Launch state",
state: {
Blocked: "Blocked",
Ready: "Ready",
Submitted: "Submitted",
},
blocked_title: "Launch blockers",
blocked_body: "Resolve the items below before sending this dataset run to SQL Lab.",
resolve_action: "Open related area",
dataset_ref_label: "Dataset reference",
readiness_label: "Readiness",
approved_mappings_label: "Approved mappings",
preview_fingerprint_label: "Preview fingerprint",
sql_lab_target_title: "Launch target",
sql_lab_target_body: "The canonical launch target is a Superset SQL Lab session using the reviewed preview and effective execution inputs.",
preview_status_label: "Preview status",
compiled_by_label: "Compiled by",
launch_action: "Launch dataset",
launch_loading: "Launching dataset...",
submitted_title: "Launch submitted",
submitted_body: "SQL Lab handoff and audited run context were recorded for this launch request.",
run_context_label: "Run context",
sql_lab_session_label: "SQL Lab session",
launch_status_label: "Launch status",
preview_ref_label: "Preview reference",
blockers: {
blocking_finding: "Blocking findings remain unresolved",
mapping_approval_required: "Mapping approval is still required",
preview_missing: "Superset preview is required before launch",
preview_pending: "Preview generation is still in progress",
preview_stale: "Preview no longer matches the current execution inputs",
preview_failed: "Preview failed and launch remains blocked",
readiness_not_run_ready: "Session is not yet in run-ready state",
preview_fingerprint_missing: "Preview fingerprint is missing, so launch cannot be trusted",
},
messages: {
launch_started: "Dataset launch request sent to SQL Lab.",
launch_failed: "Failed to launch dataset in SQL Lab.",
},
},
},
});
return () => {};
},
},
}));
describe("Dataset review US3 execution UX", () => {
beforeEach(() => {
requestApi.mockReset();
});
it("mapping_review_approves_warning_sensitive_row", async () => {
requestApi.mockResolvedValueOnce({
mapping_id: "map-1",
filter_id: "filter-1",
variable_id: "var-1",
mapping_method: "direct_match",
raw_input_value: "DE",
effective_value: "DE",
transformation_note: "Trimmed imported value",
warning_level: "medium",
requires_explicit_approval: true,
approval_state: "approved",
});
const onupdated = vi.fn();
render(ExecutionMappingReview, {
sessionId: "session-1",
mappings: [
{
mapping_id: "map-1",
filter_id: "filter-1",
variable_id: "var-1",
mapping_method: "direct_match",
raw_input_value: "DE",
effective_value: "DE",
transformation_note: "Trimmed imported value",
warning_level: "medium",
requires_explicit_approval: true,
approval_state: "pending",
},
],
importedFilters: [
{
filter_id: "filter-1",
filter_name: "country",
display_name: "Country",
raw_value: "DE",
normalized_value: "DE",
confidence_state: "imported",
recovery_status: "recovered",
},
],
templateVariables: [
{
variable_id: "var-1",
variable_name: "country",
is_required: true,
default_value: null,
},
],
onupdated,
});
expect(screen.getByText("Review filter-to-template mappings")).toBeDefined();
expect(screen.getByText("Approval required")).toBeDefined();
expect(screen.getByText("Country")).toBeDefined();
expect(screen.getByText(/Trimmed imported value/)).toBeDefined();
await fireEvent.click(screen.getByRole("button", { name: "Approve mapping" }));
await waitFor(() => {
expect(requestApi).toHaveBeenCalledWith(
"/dataset-orchestration/sessions/session-1/mappings/map-1/approve",
"POST",
{ approval_note: "Trimmed imported value" },
);
});
expect(onupdated).toHaveBeenCalledWith({
mapping: expect.objectContaining({
mapping_id: "map-1",
approval_state: "approved",
}),
preview_state: "unchanged",
});
expect(screen.getByText("Mapping approval recorded.")).toBeDefined();
});
it("mapping_review_batch_approves_pending_rows", async () => {
requestApi.mockResolvedValueOnce([
{
mapping_id: "map-1",
filter_id: "filter-1",
variable_id: "var-1",
mapping_method: "direct_match",
raw_input_value: "DE",
effective_value: "DE",
transformation_note: "Trimmed imported value",
warning_level: "medium",
requires_explicit_approval: true,
approval_state: "approved",
},
]);
const onupdated = vi.fn();
render(ExecutionMappingReview, {
sessionId: "session-1",
mappings: [
{
mapping_id: "map-1",
filter_id: "filter-1",
variable_id: "var-1",
mapping_method: "direct_match",
raw_input_value: "DE",
effective_value: "DE",
transformation_note: "Trimmed imported value",
warning_level: "medium",
requires_explicit_approval: true,
approval_state: "pending",
},
],
importedFilters: [
{
filter_id: "filter-1",
filter_name: "country",
display_name: "Country",
raw_value: "DE",
normalized_value: "DE",
confidence_state: "imported",
recovery_status: "recovered",
},
],
templateVariables: [
{
variable_id: "var-1",
variable_name: "country",
is_required: true,
default_value: null,
},
],
onupdated,
});
await fireEvent.click(screen.getByRole("button", { name: "Approve all pending mappings" }));
await waitFor(() => {
expect(requestApi).toHaveBeenCalledWith(
"/dataset-orchestration/sessions/session-1/mappings/approve-batch",
"POST",
{ mapping_ids: ["map-1"] },
);
});
expect(onupdated).toHaveBeenCalledWith({
mappings: expect.any(Array),
preview_state: "unchanged",
});
});
it("preview_panel_requests_superset_compilation_and_renders_sql", async () => {
requestApi.mockResolvedValueOnce({
preview_id: "preview-1",
session_id: "session-1",
preview_status: "ready",
compiled_sql: "SELECT * FROM sales WHERE country = 'DE'",
preview_fingerprint: "fingerprint-1",
compiled_by: "superset",
compiled_at: "2026-03-17T09:00:00Z",
});
const onupdated = vi.fn();
render(CompiledSQLPreview, {
sessionId: "session-1",
preview: null,
previewState: "missing",
onupdated,
});
expect(screen.getByText("Review Superset-compiled SQL")).toBeDefined();
expect(screen.getByText("No Superset preview is available yet. Generate one before attempting launch.")).toBeDefined();
await fireEvent.click(screen.getByRole("button", { name: "Generate SQL preview" }));
await waitFor(() => {
expect(requestApi).toHaveBeenCalledWith(
"/dataset-orchestration/sessions/session-1/preview",
"POST",
);
});
expect(onupdated).toHaveBeenCalledWith({
preview: expect.objectContaining({
preview_id: "preview-1",
compiled_by: "superset",
}),
preview_state: "ready",
});
requestApi.mockRejectedValueOnce(new Error("Superset compile failed"));
const onjump = vi.fn();
render(CompiledSQLPreview, {
sessionId: "session-1",
preview: {
preview_id: "preview-err",
preview_status: "failed",
preview_fingerprint: "fingerprint-err",
compiled_by: "superset",
error_details: "Variable country is invalid",
},
previewState: "failed",
onjump,
});
expect(screen.getByText("Variable country is invalid")).toBeDefined();
await fireEvent.click(screen.getByRole("button", { name: "Review mapping inputs" }));
expect(onjump).toHaveBeenCalledWith({ target: "mapping" });
});
it("launch_panel_blocks_then_submits_sql_lab_launch", async () => {
const onjump = vi.fn();
const { rerender } = render(LaunchConfirmationPanel, {
sessionId: "session-1",
session: {
dataset_ref: "public.sales",
readiness_state: "mapping_review_needed",
},
findings: [
{
severity: "blocking",
resolution_state: "open",
title: "Missing required value",
message: "country is required",
code: "REQ_COUNTRY",
},
],
mappings: [
{
mapping_id: "map-1",
requires_explicit_approval: true,
approval_state: "pending",
transformation_note: "Europe -> EU",
},
],
preview: null,
previewState: "missing",
onjump,
});
expect(screen.getByText("Launch blockers")).toBeDefined();
expect(screen.getByText("Blocking findings remain unresolved")).toBeDefined();
expect(screen.getByText("Mapping approval is still required")).toBeDefined();
expect(screen.getByText("Superset preview is required before launch")).toBeDefined();
await fireEvent.click(screen.getAllByRole("button", { name: "Open related area" })[1]);
expect(onjump).toHaveBeenCalledWith({ target: "mapping" });
requestApi.mockResolvedValueOnce({
run_context: {
run_context_id: "run-1",
sql_lab_session_ref: "sql-lab-77",
launch_status: "started",
preview_id: "preview-1",
},
});
await rerender({
sessionId: "session-1",
session: {
dataset_ref: "public.sales",
readiness_state: "run_ready",
},
findings: [],
mappings: [
{
mapping_id: "map-1",
requires_explicit_approval: true,
approval_state: "approved",
transformation_note: "Europe -> EU",
},
],
preview: {
preview_id: "preview-1",
preview_status: "ready",
preview_fingerprint: "fingerprint-1",
compiled_by: "superset",
},
previewState: "ready",
onupdated: vi.fn(),
onjump,
});
expect(screen.getByText("Confirm SQL Lab launch")).toBeDefined();
expect(screen.getByText("Launch state: Ready")).toBeDefined();
await fireEvent.click(screen.getByRole("button", { name: "Launch dataset" }));
await waitFor(() => {
expect(requestApi).toHaveBeenCalledWith(
"/dataset-orchestration/sessions/session-1/launch",
"POST",
);
});
expect(screen.getByText("Dataset launch request sent to SQL Lab.")).toBeDefined();
});
});
// [/DEF:DatasetReviewUs3UxTests:Module]

View File

@@ -664,6 +664,11 @@
"export_failed": "Failed to export the requested review artifact.", "export_failed": "Failed to export the requested review artifact.",
"empty_state_title": "No session loaded", "empty_state_title": "No session loaded",
"state_label": "Workspace", "state_label": "Workspace",
"state": {
"empty": "Empty",
"importing": "Importing",
"review": "Review"
},
"readiness_label": "Readiness", "readiness_label": "Readiness",
"source_badge_fallback": "review session", "source_badge_fallback": "review session",
"save_session_action": "Save session", "save_session_action": "Save session",
@@ -721,6 +726,8 @@
"preview_compiler_label": "Compiled by", "preview_compiler_label": "Compiled by",
"preview_pending_note": "Compiled preview is not part of this US1 batch yet, but the workspace keeps the state visible when present.", "preview_pending_note": "Compiled preview is not part of this US1 batch yet, but the workspace keeps the state visible when present.",
"jump_target_label": "Focused area", "jump_target_label": "Focused area",
"semantic_review_visible": "Semantic review is active for field-level source decisions.",
"clarification_visible": "Clarification is available with one active question at a time.",
"resume_action": "Resume session", "resume_action": "Resume session",
"pause_action": "Pause session", "pause_action": "Pause session",
"actions": { "actions": {
@@ -765,6 +772,245 @@
"low_confidence": "Low confidence", "low_confidence": "Low confidence",
"unresolved": "Unresolved" "unresolved": "Unresolved"
} }
},
"semantics": {
"eyebrow": "Semantic layer",
"title": "Review semantic field values",
"description": "Compare the active semantic value with ranked candidates, preserve provenance, and lock or unlock manual decisions explicitly.",
"field_count_label": "Fields in review",
"empty": "No semantic fields need review right now.",
"unknown_source": "Unknown source",
"active_value_label": "Active semantic value",
"provenance_label": "Provenance",
"confidence_label": "Confidence",
"confidence_unset": "No confidence rank",
"confidence_rank_label": "Rank",
"source_label": "Source",
"changed_by_label": "Changed by",
"locked_badge": "Locked",
"unlocked_badge": "Unlocked",
"conflict_badge": "Conflict",
"needs_review_badge": "Needs review",
"manual_verbose_name_label": "Manual verbose name",
"manual_description_label": "Manual description",
"manual_display_format_label": "Manual display format",
"manual_override_action": "Manual override",
"approve_all_action": "Approve visible candidates",
"save_manual_action": "Save manual value",
"lock_action": "Lock field",
"unlock_action": "Unlock field",
"feedback_up_action": "👍",
"feedback_down_action": "👎",
"candidates_title": "Candidate options",
"candidates_empty": "No ranked candidate options were returned for this field.",
"candidate_description_empty": "No description proposed for this candidate.",
"display_format_label": "Display format",
"score_label": "Score",
"apply_candidate_action": "Apply",
"apply_and_lock_action": "Apply & lock",
"empty_value": "No current value",
"provenance": {
"unresolved": "Unresolved",
"dictionary_exact": "Dictionary exact match",
"reference_imported": "Imported reference",
"ai_generated": "AI generated",
"fuzzy_inferred": "Fuzzy inferred",
"manual_override": "Manual override"
},
"candidate_status": {
"pending": "Pending",
"accepted": "Accepted",
"superseded": "Superseded",
"rejected": "Rejected"
},
"messages": {
"saving": "Saving semantic decision...",
"save_failed": "Failed to save semantic decision.",
"manual_override_required": "Provide at least one manual semantic value before saving.",
"manual_saved": "Manual semantic value saved and locked.",
"candidate_applied": "Candidate value applied.",
"candidate_locked": "Candidate value applied and locked.",
"locked": "Field locked against silent overwrite.",
"unlocked": "Field unlocked for renewed review.",
"feedback_up": "Positive feedback recorded for this semantic value.",
"feedback_down": "Negative feedback recorded for this semantic value."
}
},
"clarification": {
"eyebrow": "Clarification",
"title": "Resolve one ambiguity at a time",
"description": "Answer the active clarification question, skip it, or send it to expert review while progress stays resumable.",
"progress_label": "Clarification progress",
"state_label": "Dialog state",
"status_label": "Status",
"remaining_label": "Remaining questions",
"summary_delta_label": "Summary delta",
"active_question_label": "Active question",
"why_it_matters_label": "Why it matters",
"current_guess_label": "Current guess",
"current_guess_empty": "No current guess is available.",
"topic_label": "Topic reference",
"options_title": "Available options",
"recommended_badge": "Recommended",
"custom_answer_label": "Custom answer",
"answer_action": "Answer with selected option",
"custom_answer_action": "Submit custom answer",
"skip_action": "Skip for now",
"assistant_action": "Open assistant guidance",
"expert_review_action": "Send to expert review",
"resume_action": "Resume clarification",
"completed": "No active clarification question remains. The session can be resumed later if new ambiguity appears.",
"messages": {
"saving": "Saving clarification answer...",
"saved": "Clarification answer saved.",
"skipped": "Question skipped and left resumable.",
"expert_review_requested": "Question escalated to expert review.",
"assistant_opened": "Assistant guidance opened for this clarification step.",
"resumed": "Clarification resumed from the highest-priority unresolved question.",
"resume_failed": "Failed to resume clarification.",
"save_failed": "Failed to save clarification answer.",
"option_required": "Select an option before submitting the answer.",
"custom_required": "Enter a custom answer before submitting."
}
},
"mapping": {
"eyebrow": "Template mapping",
"title": "Review filter-to-template mappings",
"description": "Verify imported filter values, effective execution values, and warning-sensitive transformations before preview or launch.",
"state_label": "Mapping state",
"state": {
"Incomplete": "Incomplete",
"WarningApproval": "Approval required",
"Approved": "Approved"
},
"pending_approvals_label": "Pending approvals",
"required_values_label": "Missing required values",
"empty": "No execution mappings are available yet.",
"required_blockers_notice": "{count} required values still need attention before preview or launch can proceed.",
"approval_notice": "{count} mapping transformations still need explicit approval.",
"to_variable_label": "To variable",
"required_badge": "Required",
"approval_required_badge": "Approval required",
"approved_badge": "Approved",
"source_filter_label": "Source filter",
"source_value_label": "Source value",
"confidence_label": "Confidence",
"recovery_label": "Recovery",
"effective_value_label": "Effective value",
"method_label": "Method",
"approval_label": "Approval",
"warning_label": "Warning",
"transformation_note_label": "Transformation note",
"missing_required_value": "This mapping still lacks a required effective value.",
"manual_value_label": "Manual effective value",
"manual_note_label": "Manual override note",
"save_override_action": "Save manual override",
"manual_override_action": "Manual override",
"approve_action": "Approve mapping",
"approve_all_action": "Approve all pending mappings",
"approval_state": {
"pending": "Pending",
"approved": "Approved",
"rejected": "Rejected",
"not_required": "Not required"
},
"warning_level": {
"none": "None",
"low": "Low",
"medium": "Medium",
"high": "High"
},
"method": {
"direct_match": "Direct match",
"heuristic_match": "Heuristic match",
"semantic_match": "Semantic match",
"manual_override": "Manual override"
},
"messages": {
"saving": "Saving mapping decision...",
"approval_saved": "Mapping approval recorded.",
"approval_failed": "Failed to save mapping approval.",
"override_saved": "Manual mapping override saved. Preview should be refreshed.",
"override_failed": "Failed to save manual mapping override.",
"required_value_missing": "Provide a required value before saving the manual override."
}
},
"preview": {
"eyebrow": "Compiled SQL preview",
"title": "Review Superset-compiled SQL",
"description": "Preview truth comes only from Superset. Regenerate the preview whenever mappings or required inputs change.",
"state_label": "Preview state",
"state": {
"missing": "Missing",
"pending": "Pending",
"ready": "Ready",
"stale": "Stale",
"failed": "Error"
},
"generate_action": "Generate SQL preview",
"generate_loading": "Generating preview...",
"missing_body": "No Superset preview is available yet. Generate one before attempting launch.",
"pending_body": "Superset is compiling the current execution context. Launch stays blocked until preview completes.",
"ready_body": "This SQL preview was compiled by Superset for the current execution inputs.",
"stale_body": "Mappings or effective values changed after the last successful preview. Regenerate before launch.",
"error_body": "Superset could not compile the current execution context.",
"compiler_label": "Compiled source",
"compiled_by_superset": "Compiled by Superset",
"compiled_source_unknown": "Compilation source unavailable",
"fingerprint_label": "Preview fingerprint",
"compiled_at_label": "Compiled at",
"sql_block_title": "Superset SQL",
"compiled_truth_note": "Exact SQL returned by Superset",
"go_to_mapping_action": "Review mapping inputs",
"review_inputs_action": "Review changed inputs",
"messages": {
"generated": "Superset preview refreshed.",
"generate_failed": "Failed to generate Superset preview."
}
},
"launch": {
"eyebrow": "Launch confirmation",
"title": "Confirm SQL Lab launch",
"description": "Launch remains blocked until preview truth, approvals, and readiness gates all match the reviewed execution context.",
"state_label": "Launch state",
"state": {
"Blocked": "Blocked",
"Ready": "Ready",
"Submitted": "Submitted"
},
"blocked_title": "Launch blockers",
"blocked_body": "Resolve the items below before sending this dataset run to SQL Lab.",
"resolve_action": "Open related area",
"dataset_ref_label": "Dataset reference",
"readiness_label": "Readiness",
"approved_mappings_label": "Approved mappings",
"preview_fingerprint_label": "Preview fingerprint",
"sql_lab_target_title": "Launch target",
"sql_lab_target_body": "The canonical launch target is a Superset SQL Lab session using the reviewed preview and effective execution inputs.",
"preview_status_label": "Preview status",
"compiled_by_label": "Compiled by",
"launch_action": "Launch dataset",
"launch_loading": "Launching dataset...",
"submitted_title": "Launch submitted",
"submitted_body": "SQL Lab handoff and audited run context were recorded for this launch request.",
"run_context_label": "Run context",
"sql_lab_session_label": "SQL Lab session",
"launch_status_label": "Launch status",
"preview_ref_label": "Preview reference",
"blockers": {
"blocking_finding": "Blocking findings remain unresolved",
"mapping_approval_required": "Mapping approval is still required",
"preview_missing": "Superset preview is required before launch",
"preview_pending": "Preview generation is still in progress",
"preview_stale": "Preview no longer matches the current execution inputs",
"preview_failed": "Preview failed and launch remains blocked",
"readiness_not_run_ready": "Session is not yet in run-ready state",
"preview_fingerprint_missing": "Preview fingerprint is missing, so launch cannot be trusted"
},
"messages": {
"launch_started": "Dataset launch request sent to SQL Lab.",
"launch_failed": "Failed to launch dataset in SQL Lab."
}
} }
}, },
"tasks": { "tasks": {

View File

@@ -662,6 +662,11 @@
"export_failed": "Не удалось экспортировать выбранный артефакт review.", "export_failed": "Не удалось экспортировать выбранный артефакт review.",
"empty_state_title": "Сессия не загружена", "empty_state_title": "Сессия не загружена",
"state_label": "Workspace", "state_label": "Workspace",
"state": {
"empty": "Пусто",
"importing": "Импорт",
"review": "Проверка"
},
"readiness_label": "Готовность", "readiness_label": "Готовность",
"source_badge_fallback": "review-сессия", "source_badge_fallback": "review-сессия",
"save_session_action": "Сохранить сессию", "save_session_action": "Сохранить сессию",
@@ -719,6 +724,8 @@
"preview_compiler_label": "Скомпилировано", "preview_compiler_label": "Скомпилировано",
"preview_pending_note": "Скомпилированный preview не входит в этот пакет US1, но workspace сохраняет его видимым, если он уже существует.", "preview_pending_note": "Скомпилированный preview не входит в этот пакет US1, но workspace сохраняет его видимым, если он уже существует.",
"jump_target_label": "Выбранная зона", "jump_target_label": "Выбранная зона",
"semantic_review_visible": "Семантический review активен для полей с решениями по источникам.",
"clarification_visible": "Уточнение доступно в режиме одного активного вопроса.",
"resume_action": "Возобновить сессию", "resume_action": "Возобновить сессию",
"pause_action": "Поставить сессию на паузу", "pause_action": "Поставить сессию на паузу",
"actions": { "actions": {
@@ -763,6 +770,245 @@
"low_confidence": "Низкая уверенность", "low_confidence": "Низкая уверенность",
"unresolved": "Не разрешено" "unresolved": "Не разрешено"
} }
},
"semantics": {
"eyebrow": "Семантический слой",
"title": "Проверьте семантические значения полей",
"description": "Сравнивайте активное семантическое значение с ранжированными кандидатами, сохраняйте provenance и явно блокируйте или разблокируйте ручные решения.",
"field_count_label": "Поля в review",
"empty": "Семантические поля сейчас не требуют review.",
"unknown_source": "Неизвестный источник",
"active_value_label": "Активное семантическое значение",
"provenance_label": "Provenance",
"confidence_label": "Уверенность",
"confidence_unset": "Ранг уверенности отсутствует",
"confidence_rank_label": "Ранг",
"source_label": "Источник",
"changed_by_label": "Изменил",
"locked_badge": "Заблокировано",
"unlocked_badge": "Разблокировано",
"conflict_badge": "Конфликт",
"needs_review_badge": "Нужен review",
"manual_verbose_name_label": "Ручное verbose name",
"manual_description_label": "Ручное описание",
"manual_display_format_label": "Ручной display format",
"manual_override_action": "Ручное переопределение",
"approve_all_action": "Подтвердить видимые кандидаты",
"save_manual_action": "Сохранить ручное значение",
"lock_action": "Заблокировать поле",
"unlock_action": "Разблокировать поле",
"feedback_up_action": "👍",
"feedback_down_action": "👎",
"candidates_title": "Варианты кандидатов",
"candidates_empty": "Для этого поля не возвращены ранжированные кандидаты.",
"candidate_description_empty": "Для этого кандидата описание не предложено.",
"display_format_label": "Формат отображения",
"score_label": "Оценка",
"apply_candidate_action": "Применить",
"apply_and_lock_action": "Применить и заблокировать",
"empty_value": "Текущее значение отсутствует",
"provenance": {
"unresolved": "Не разрешено",
"dictionary_exact": "Точное совпадение словаря",
"reference_imported": "Импортированная ссылка",
"ai_generated": "Сгенерировано AI",
"fuzzy_inferred": "Нечеткий вывод",
"manual_override": "Ручное переопределение"
},
"candidate_status": {
"pending": "В ожидании",
"accepted": "Принято",
"superseded": "Заменено",
"rejected": "Отклонено"
},
"messages": {
"saving": "Сохранение семантического решения...",
"save_failed": "Не удалось сохранить семантическое решение.",
"manual_override_required": "Перед сохранением укажите хотя бы одно ручное семантическое значение.",
"manual_saved": "Ручное семантическое значение сохранено и заблокировано.",
"candidate_applied": "Значение кандидата применено.",
"candidate_locked": "Значение кандидата применено и заблокировано.",
"locked": "Поле заблокировано от тихой перезаписи.",
"unlocked": "Поле разблокировано для повторного review.",
"feedback_up": "Позитивный feedback для этого семантического значения сохранен.",
"feedback_down": "Негативный feedback для этого семантического значения сохранен."
}
},
"clarification": {
"eyebrow": "Уточнение",
"title": "Разрешайте неоднозначности по одному вопросу",
"description": "Ответьте на активный вопрос уточнения, пропустите его или отправьте на экспертный review, сохраняя возможность возобновления.",
"progress_label": "Прогресс уточнения",
"state_label": "Состояние диалога",
"status_label": "Статус",
"remaining_label": "Оставшиеся вопросы",
"summary_delta_label": "Изменение summary",
"active_question_label": "Активный вопрос",
"why_it_matters_label": "Почему это важно",
"current_guess_label": "Текущая гипотеза",
"current_guess_empty": "Текущая гипотеза отсутствует.",
"topic_label": "Ссылка на тему",
"options_title": "Доступные варианты",
"recommended_badge": "Рекомендовано",
"custom_answer_label": "Свой ответ",
"answer_action": "Ответить выбранным вариантом",
"custom_answer_action": "Отправить свой ответ",
"skip_action": "Пропустить пока",
"assistant_action": "Открыть помощь ассистента",
"expert_review_action": "Передать на экспертный review",
"resume_action": "Возобновить уточнение",
"completed": "Активных вопросов уточнения не осталось. Сессию можно возобновить позже, если появится новая неоднозначность.",
"messages": {
"saving": "Сохранение ответа на уточнение...",
"saved": "Ответ на уточнение сохранен.",
"skipped": "Вопрос пропущен и сохранен как возобновляемый.",
"expert_review_requested": "Вопрос отправлен на экспертный review.",
"assistant_opened": "Для этого шага уточнения открыта помощь ассистента.",
"resumed": "Уточнение возобновлено с самого приоритетного неразрешенного вопроса.",
"resume_failed": "Не удалось возобновить уточнение.",
"save_failed": "Не удалось сохранить ответ на уточнение.",
"option_required": "Перед отправкой выберите один вариант.",
"custom_required": "Перед отправкой введите свой ответ."
}
},
"mapping": {
"eyebrow": "Маппинг шаблонов",
"title": "Проверьте маппинг фильтров и шаблонов",
"description": "Проверьте импортированные значения фильтров, итоговые execution values и warning-преобразования перед preview или запуском.",
"state_label": "Состояние маппинга",
"state": {
"Incomplete": "Неполный",
"WarningApproval": "Требуется подтверждение",
"Approved": "Подтверждено"
},
"pending_approvals_label": "Ожидают подтверждения",
"required_values_label": "Отсутствуют обязательные значения",
"empty": "Execution mappings пока отсутствуют.",
"required_blockers_notice": "Еще {count} обязательных значений требуют внимания перед preview или запуском.",
"approval_notice": "Еще {count} преобразований маппинга требуют явного подтверждения.",
"to_variable_label": "В переменную",
"required_badge": "Обязательно",
"approval_required_badge": "Нужно подтверждение",
"approved_badge": "Подтверждено",
"source_filter_label": "Исходный фильтр",
"source_value_label": "Исходное значение",
"confidence_label": "Уверенность",
"recovery_label": "Восстановление",
"effective_value_label": "Итоговое значение",
"method_label": "Метод",
"approval_label": "Подтверждение",
"warning_label": "Предупреждение",
"transformation_note_label": "Комментарий к преобразованию",
"missing_required_value": "Для этого маппинга все еще отсутствует обязательное итоговое значение.",
"manual_value_label": "Ручное итоговое значение",
"manual_note_label": "Комментарий к ручному переопределению",
"save_override_action": "Сохранить ручное переопределение",
"manual_override_action": "Ручное переопределение",
"approve_action": "Подтвердить маппинг",
"approve_all_action": "Подтвердить все ожидающие маппинги",
"approval_state": {
"pending": "Ожидает",
"approved": "Подтверждено",
"rejected": "Отклонено",
"not_required": "Не требуется"
},
"warning_level": {
"none": "Нет",
"low": "Низкий",
"medium": "Средний",
"high": "Высокий"
},
"method": {
"direct_match": "Прямое совпадение",
"heuristic_match": "Эвристическое совпадение",
"semantic_match": "Семантическое совпадение",
"manual_override": "Ручное переопределение"
},
"messages": {
"saving": "Сохранение решения по маппингу...",
"approval_saved": "Подтверждение маппинга сохранено.",
"approval_failed": "Не удалось сохранить подтверждение маппинга.",
"override_saved": "Ручное переопределение маппинга сохранено. Preview нужно обновить.",
"override_failed": "Не удалось сохранить ручное переопределение маппинга.",
"required_value_missing": "Перед сохранением ручного переопределения укажите обязательное значение."
}
},
"preview": {
"eyebrow": "Скомпилированный SQL preview",
"title": "Проверьте SQL, скомпилированный Superset",
"description": "Истина preview приходит только от Superset. Перегенерируйте preview после изменений маппинга или обязательных входных значений.",
"state_label": "Состояние preview",
"state": {
"missing": "Отсутствует",
"pending": "Генерируется",
"ready": "Готов",
"stale": "Устарел",
"failed": "Ошибка"
},
"generate_action": "Сгенерировать SQL preview",
"generate_loading": "Генерация preview...",
"missing_body": "Preview от Superset пока отсутствует. Сгенерируйте его перед запуском.",
"pending_body": "Superset компилирует текущий execution context. Запуск остается заблокированным до завершения preview.",
"ready_body": "Этот SQL preview был скомпилирован Superset для текущих execution inputs.",
"stale_body": "После последнего успешного preview были изменены маппинги или итоговые значения. Перед запуском нужно перегенерировать preview.",
"error_body": "Superset не смог скомпилировать текущий execution context.",
"compiler_label": "Источник компиляции",
"compiled_by_superset": "Скомпилировано Superset",
"compiled_source_unknown": "Источник компиляции недоступен",
"fingerprint_label": "Fingerprint preview",
"compiled_at_label": "Время компиляции",
"sql_block_title": "SQL от Superset",
"compiled_truth_note": "Точный SQL, возвращенный Superset",
"go_to_mapping_action": "Проверить mapping inputs",
"review_inputs_action": "Проверить измененные inputs",
"messages": {
"generated": "Preview от Superset обновлен.",
"generate_failed": "Не удалось сгенерировать preview от Superset."
}
},
"launch": {
"eyebrow": "Подтверждение запуска",
"title": "Подтвердите запуск в SQL Lab",
"description": "Запуск остается заблокированным, пока preview, approvals и readiness gates не соответствуют проверенному execution context.",
"state_label": "Состояние запуска",
"state": {
"Blocked": "Заблокирован",
"Ready": "Готов",
"Submitted": "Отправлен"
},
"blocked_title": "Блокеры запуска",
"blocked_body": "Разрешите следующие пункты перед отправкой запуска датасета в SQL Lab.",
"resolve_action": "Открыть связанный раздел",
"dataset_ref_label": "Ссылка на датасет",
"readiness_label": "Готовность",
"approved_mappings_label": "Подтвержденные маппинги",
"preview_fingerprint_label": "Fingerprint preview",
"sql_lab_target_title": "Цель запуска",
"sql_lab_target_body": "Каноническая цель запуска — сессия Superset SQL Lab с использованием проверенного preview и итоговых execution inputs.",
"preview_status_label": "Статус preview",
"compiled_by_label": "Скомпилировано",
"launch_action": "Запустить датасет",
"launch_loading": "Запуск датасета...",
"submitted_title": "Запуск отправлен",
"submitted_body": "Передача в SQL Lab и аудитируемый run context были записаны для этого запроса запуска.",
"run_context_label": "Run context",
"sql_lab_session_label": "Сессия SQL Lab",
"launch_status_label": "Статус запуска",
"preview_ref_label": "Ссылка на preview",
"blockers": {
"blocking_finding": "Есть неразрешенные блокирующие findings",
"mapping_approval_required": "Маппинг все еще требует подтверждения",
"preview_missing": "Перед запуском требуется preview от Superset",
"preview_pending": "Preview все еще генерируется",
"preview_stale": "Preview больше не соответствует текущим execution inputs",
"preview_failed": "Preview завершился ошибкой, запуск заблокирован",
"readiness_not_run_ready": "Сессия еще не находится в состоянии run-ready",
"preview_fingerprint_missing": "Fingerprint preview отсутствует, поэтому запуску нельзя доверять"
},
"messages": {
"launch_started": "Запрос запуска датасета отправлен в SQL Lab.",
"launch_failed": "Не удалось запустить датасет в SQL Lab."
}
} }
}, },
"tasks": { "tasks": {

View File

@@ -14,8 +14,8 @@
<!-- @UX_STATE: Empty -> Show source intake with Superset link and dataset-selection entry actions. --> <!-- @UX_STATE: Empty -> Show source intake with Superset link and dataset-selection entry actions. -->
<!-- @UX_STATE: Importing -> Show progressive recovery milestones as context is assembled. --> <!-- @UX_STATE: Importing -> Show progressive recovery milestones as context is assembled. -->
<!-- @UX_STATE: Review -> Show summary, findings, semantic layer, filters, mapping, and next action. --> <!-- @UX_STATE: Review -> Show summary, findings, semantic layer, filters, mapping, and next action. -->
<!-- @UX_STATE: Clarification -> Reserved for later batch; current session still exposes the next-step intent. --> <!-- @UX_STATE: Clarification -> Clarification panel mounts one active question at a time and exposes resumable answer plus feedback actions. -->
<!-- @UX_STATE: Ready -> Reserved for later batch; warnings stay visible and never disappear silently. --> <!-- @UX_STATE: Ready -> Preview, warnings, and next-step evidence stay visible without hiding US2 review controls. -->
<!-- @UX_FEEDBACK: Main CTA changes by readiness state and reflects current highest-value next action. --> <!-- @UX_FEEDBACK: Main CTA changes by readiness state and reflects current highest-value next action. -->
<!-- @UX_RECOVERY: Users can save, resume, or reopen an unfinished session without losing context. --> <!-- @UX_RECOVERY: Users can save, resume, or reopen an unfinished session without losing context. -->
<!-- @UX_REACTIVITY: Uses Svelte runes for session, readiness, preview, and task state derivation. --> <!-- @UX_REACTIVITY: Uses Svelte runes for session, readiness, preview, and task state derivation. -->
@@ -26,6 +26,11 @@
import { api } from "$lib/api.js"; import { api } from "$lib/api.js";
import SourceIntakePanel from "$lib/components/dataset-review/SourceIntakePanel.svelte"; import SourceIntakePanel from "$lib/components/dataset-review/SourceIntakePanel.svelte";
import ValidationFindingsPanel from "$lib/components/dataset-review/ValidationFindingsPanel.svelte"; import ValidationFindingsPanel from "$lib/components/dataset-review/ValidationFindingsPanel.svelte";
import SemanticLayerReview from "$lib/components/dataset-review/SemanticLayerReview.svelte";
import ClarificationDialog from "$lib/components/dataset-review/ClarificationDialog.svelte";
import ExecutionMappingReview from "$lib/components/dataset-review/ExecutionMappingReview.svelte";
import CompiledSQLPreview from "$lib/components/dataset-review/CompiledSQLPreview.svelte";
import LaunchConfirmationPanel from "$lib/components/dataset-review/LaunchConfirmationPanel.svelte";
import { import {
setError, setError,
setLoading, setLoading,
@@ -52,6 +57,9 @@
let summaryDraft = $state(""); let summaryDraft = $state("");
let jumpTarget = $state(""); let jumpTarget = $state("");
let intakeAcknowledgment = $state(""); let intakeAcknowledgment = $state("");
let clarificationState = $state(null);
let previewUiState = $state("");
let launchResult = $state(null);
const routeSessionId = $derived(page.params.id || ""); const routeSessionId = $derived(page.params.id || "");
const environments = $derived(environmentContextState.current?.environments || []); const environments = $derived(environmentContextState.current?.environments || []);
@@ -62,10 +70,10 @@
); );
const currentWorkspaceState = $derived( const currentWorkspaceState = $derived(
!session !session
? (isSubmitting ? "Importing" : "Empty") ? (isSubmitting ? "importing" : "empty")
: session.readiness_state === "importing" : session.readiness_state === "importing"
? "Importing" ? "importing"
: "Review", : "review",
); );
const profile = $derived(session?.profile || null); const profile = $derived(session?.profile || null);
const findings = $derived(session?.findings || []); const findings = $derived(session?.findings || []);
@@ -73,8 +81,14 @@
const latestPreview = $derived( const latestPreview = $derived(
session?.previews?.length ? session.previews[session.previews.length - 1] : null, session?.previews?.length ? session.previews[session.previews.length - 1] : null,
); );
const executionMappings = $derived(session?.execution_mappings || []);
const templateVariables = $derived(session?.template_variables || []);
const latestRunContext = $derived(
session?.run_contexts?.length ? session.run_contexts[session.run_contexts.length - 1] : null,
);
const exportDisabled = $derived(!session || isExporting); const exportDisabled = $derived(!session || isExporting);
const saveDisabled = $derived(!session || isSubmitting); const saveDisabled = $derived(!session || isSubmitting);
const launchDisabled = $derived(!session || isSubmitting || isExporting);
const blockingCount = $derived( const blockingCount = $derived(
findings.filter((finding) => finding.severity === "blocking").length, findings.filter((finding) => finding.severity === "blocking").length,
); );
@@ -104,6 +118,14 @@
const importMilestones = $derived(buildImportMilestones()); const importMilestones = $derived(buildImportMilestones());
const recentActions = $derived(buildRecentActions()); const recentActions = $derived(buildRecentActions());
function getWorkspaceStateLabel(state) {
return $t.dataset_review?.workspace?.state?.[state] || state;
}
function getFallbackLabel(key, fallback = "") {
return $t.dataset_review?.workspace?.fallbacks?.[key] || fallback;
}
function getRecommendedActionLabel(action) { function getRecommendedActionLabel(action) {
const normalized = String(action || "import_from_superset"); const normalized = String(action || "import_from_superset");
return $t.dataset_review?.workspace?.actions?.[normalized] || normalized; return $t.dataset_review?.workspace?.actions?.[normalized] || normalized;
@@ -114,7 +136,7 @@
const filtersDone = importedFilters.length > 0; const filtersDone = importedFilters.length > 0;
const variablesDone = Boolean(importedFilters.length || latestPreview); const variablesDone = Boolean(importedFilters.length || latestPreview);
const semanticsDone = Boolean(profile?.business_summary || findings.length > 0); const semanticsDone = Boolean(profile?.business_summary || findings.length > 0);
const importState = currentWorkspaceState === "Importing"; const importState = currentWorkspaceState === "importing";
return [ return [
{ {
@@ -154,7 +176,10 @@
{ {
key: "source", key: "source",
title: $t.dataset_review?.workspace?.timeline?.source, title: $t.dataset_review?.workspace?.timeline?.source,
detail: session.source_input || session.dataset_ref || "—", detail:
session.source_input ||
session.dataset_ref ||
getFallbackLabel("empty_value", "—"),
}, },
{ {
key: "status", key: "status",
@@ -197,6 +222,7 @@
async function loadSessionDetail(sessionId) { async function loadSessionDetail(sessionId) {
if (!sessionId) { if (!sessionId) {
session = null; session = null;
clarificationState = null;
loadError = ""; loadError = "";
summaryDraft = ""; summaryDraft = "";
resetSession(); resetSession();
@@ -210,8 +236,11 @@
try { try {
const detail = await api.fetchApi(`/dataset-orchestration/sessions/${sessionId}`); const detail = await api.fetchApi(`/dataset-orchestration/sessions/${sessionId}`);
session = detail; session = detail;
previewUiState = "";
launchResult = null;
summaryDraft = detail?.profile?.business_summary || ""; summaryDraft = detail?.profile?.business_summary || "";
setSession(detail); setSession(detail);
await loadClarificationState(detail);
} catch (error) { } catch (error) {
loadError = loadError =
error?.message || error?.message ||
@@ -223,6 +252,21 @@
} }
} }
async function loadClarificationState(sessionDetail) {
if (!sessionDetail?.session_id || !sessionDetail?.clarification_sessions?.length) {
clarificationState = null;
return;
}
try {
clarificationState = await api.fetchApi(
`/dataset-orchestration/sessions/${sessionDetail.session_id}/clarification`,
);
} catch (error) {
clarificationState = null;
}
}
async function bootstrapWorkspace() { async function bootstrapWorkspace() {
isBootstrapping = true; isBootstrapping = true;
await initializeEnvironmentContext(); await initializeEnvironmentContext();
@@ -341,6 +385,121 @@
isEditingSummary = false; isEditingSummary = false;
} }
function handleSemanticUpdated(updatedField) {
if (!session || !updatedField) {
return;
}
if (Array.isArray(updatedField.fields)) {
const updatedLookup = new Map(updatedField.fields.map((field) => [field.field_id, field]));
session = {
...session,
semantic_fields: (session.semantic_fields || []).map((field) =>
updatedLookup.get(field.field_id) || field,
),
};
return;
}
if (!updatedField?.field_id) {
return;
}
session = {
...session,
semantic_fields: (session.semantic_fields || []).map((field) =>
field.field_id === updatedField.field_id ? updatedField : field,
),
};
}
function handleClarificationUpdated(result) {
if (!result) {
return;
}
if (result.clarification_state) {
clarificationState = result.clarification_state;
} else if (result.clarification_session || result.current_question !== undefined) {
clarificationState = result;
}
if (result.session && session) {
session = {
...session,
...result.session,
};
}
}
function handleExecutionMappingUpdated(result) {
if (!result || !session) {
return;
}
if (Array.isArray(result.mappings)) {
const updatedLookup = new Map(result.mappings.map((mapping) => [mapping.mapping_id, mapping]));
session = {
...session,
execution_mappings: executionMappings.map((mapping) =>
updatedLookup.get(mapping.mapping_id) || mapping,
),
};
} else if (result.mapping) {
session = {
...session,
execution_mappings: executionMappings.map((mapping) =>
mapping.mapping_id === result.mapping.mapping_id ? result.mapping : mapping,
),
};
} else {
return;
}
if (result.preview_state) {
previewUiState = result.preview_state;
}
}
function handlePreviewUpdated(result) {
if (!session || !result?.preview) {
return;
}
previewUiState = result.preview_state || result.preview.preview_status || "";
const existingPreviews = session.previews || [];
const nextPreviews = existingPreviews.some(
(item) => item.preview_id === result.preview.preview_id,
)
? existingPreviews.map((item) =>
item.preview_id === result.preview.preview_id ? result.preview : item,
)
: [...existingPreviews, result.preview];
session = {
...session,
previews: nextPreviews,
readiness_state:
result.preview.preview_status === "ready"
? session.readiness_state === "run_ready"
? "run_ready"
: "compiled_preview_ready"
: session.readiness_state,
};
}
function handleLaunchUpdated(result) {
if (!result) {
return;
}
launchResult = result.launch_result || result;
}
function handleSectionJump(event) {
jumpTarget = event?.target || "";
}
function primaryActionHandler() { function primaryActionHandler() {
if (!session) { if (!session) {
return; return;
@@ -351,6 +510,21 @@
return; return;
} }
if (session.recommended_action === "generate_sql_preview") {
jumpTarget = "sql-preview";
return;
}
if (session.recommended_action === "approve_mapping") {
jumpTarget = "mapping";
return;
}
if (session.recommended_action === "launch_dataset") {
jumpTarget = "launch";
return;
}
if (session.status === "active") { if (session.status === "active") {
void updateSessionLifecycle("paused"); void updateSessionLifecycle("paused");
} }
@@ -383,7 +557,7 @@
</span> </span>
{/if} {/if}
<span class="rounded-full bg-slate-100 px-3 py-1 text-xs font-medium text-slate-700"> <span class="rounded-full bg-slate-100 px-3 py-1 text-xs font-medium text-slate-700">
{$t.dataset_review?.workspace?.state_label}: {currentWorkspaceState} {$t.dataset_review?.workspace?.state_label}: {getWorkspaceStateLabel(currentWorkspaceState)}
</span> </span>
{#if session} {#if session}
<span class="rounded-full bg-blue-50 px-3 py-1 text-xs font-medium text-blue-700"> <span class="rounded-full bg-blue-50 px-3 py-1 text-xs font-medium text-blue-700">
@@ -429,7 +603,7 @@
onsubmit={handleSourceSubmit} onsubmit={handleSourceSubmit}
/> />
{#if currentWorkspaceState === "Importing"} {#if currentWorkspaceState === "importing"}
<section class="rounded-2xl border border-blue-200 bg-blue-50 p-5 shadow-sm"> <section class="rounded-2xl border border-blue-200 bg-blue-50 p-5 shadow-sm">
<div class="flex items-start justify-between gap-3"> <div class="flex items-start justify-between gap-3">
<div> <div>
@@ -444,7 +618,7 @@
</p> </p>
</div> </div>
<span class="rounded-full bg-white px-3 py-1 text-xs font-medium text-blue-700"> <span class="rounded-full bg-white px-3 py-1 text-xs font-medium text-blue-700">
{currentWorkspaceState} {getWorkspaceStateLabel(currentWorkspaceState)}
</span> </span>
</div> </div>
@@ -499,7 +673,7 @@
{$t.dataset_review?.workspace?.import_status_title} {$t.dataset_review?.workspace?.import_status_title}
</div> </div>
<div class="mt-1 text-sm font-medium text-slate-900"> <div class="mt-1 text-sm font-medium text-slate-900">
{currentWorkspaceState} {getWorkspaceStateLabel(currentWorkspaceState)}
</div> </div>
</div> </div>
<span class="rounded-full bg-white px-2.5 py-1 text-xs font-medium text-slate-700"> <span class="rounded-full bg-white px-2.5 py-1 text-xs font-medium text-slate-700">
@@ -571,7 +745,8 @@
{profile?.dataset_name || session.dataset_ref} {profile?.dataset_name || session.dataset_ref}
</h2> </h2>
<p class="mt-1 text-sm text-slate-600"> <p class="mt-1 text-sm text-slate-600">
{profile?.schema_name || "—"}{profile?.database_name || "—"} {profile?.schema_name || getFallbackLabel("empty_value", "—")}{profile?.database_name ||
getFallbackLabel("empty_value", "—")}
</p> </p>
</div> </div>
@@ -741,6 +916,32 @@
{/if} {/if}
</div> </div>
<SemanticLayerReview
sessionId={session.session_id}
semanticSources={session.semantic_sources || []}
fields={session.semantic_fields || []}
disabled={isSubmitting}
onupdated={handleSemanticUpdated}
/>
{#if clarificationState || session.clarification_sessions?.length}
<ClarificationDialog
sessionId={session.session_id}
clarificationState={clarificationState}
disabled={isSubmitting}
onupdated={handleClarificationUpdated}
/>
{/if}
<ExecutionMappingReview
sessionId={session.session_id}
mappings={executionMappings}
importedFilters={importedFilters}
templateVariables={templateVariables}
disabled={launchDisabled}
onupdated={handleExecutionMappingUpdated}
/>
<ValidationFindingsPanel <ValidationFindingsPanel
findings={findings} findings={findings}
recommendedAction={session.recommended_action} recommendedAction={session.recommended_action}
@@ -866,27 +1067,27 @@
{/if} {/if}
</section> </section>
<section class="rounded-2xl border border-slate-200 bg-white p-5 shadow-sm"> <CompiledSQLPreview
<h2 class="text-lg font-semibold text-slate-900"> sessionId={session.session_id}
{$t.dataset_review?.workspace?.preview_title} preview={latestPreview}
</h2> previewState={previewUiState}
{#if latestPreview} disabled={launchDisabled}
<div class="mt-3 space-y-2 text-sm text-slate-700"> onupdated={handlePreviewUpdated}
<p> onjump={handleSectionJump}
{$t.dataset_review?.workspace?.preview_status_label}: />
{latestPreview.preview_status}
</p> <LaunchConfirmationPanel
<p> sessionId={session.session_id}
{$t.dataset_review?.workspace?.preview_compiler_label}: session={session}
{latestPreview.compiled_by} findings={findings}
</p> mappings={executionMappings}
</div> preview={latestPreview}
{:else} previewState={previewUiState}
<p class="mt-3 text-sm text-slate-600"> latestRunContext={launchResult?.run_context || latestRunContext}
{$t.dataset_review?.workspace?.preview_pending_note} disabled={launchDisabled}
</p> onupdated={handleLaunchUpdated}
{/if} onjump={handleSectionJump}
</section> />
</aside> </aside>
</div> </div>
{/if} {/if}

View File

@@ -5,15 +5,16 @@
// [DEF:DatasetReviewWorkspaceUxTests:Module] // [DEF:DatasetReviewWorkspaceUxTests:Module]
// @COMPLEXITY: 3 // @COMPLEXITY: 3
// @SEMANTICS: dataset-review, workspace, route, ux-tests, review-state, exports, recovery // @SEMANTICS: dataset-review, workspace, route, ux-tests, review-state, exports, recovery
// @PURPOSE: Verify US1 dataset review workspace flow evidence for empty, load, import, recovery, and export behaviors. // @PURPOSE: Verify US1-safe and US2-integrated dataset review workspace flow evidence for empty, load, import, semantics, clarification, and export behaviors.
// @LAYER: UI // @LAYER: UI
// @RELATION: [VERIFIES] ->[DatasetReviewWorkspace] // @RELATION: [VERIFIES] ->[DatasetReviewWorkspace]
// @UX_STATE: Empty -> Intake-first workspace renders clear starting actions. // @UX_STATE: Empty -> Intake-first workspace renders clear starting actions.
// @UX_STATE: Importing -> Progressive milestones remain visible while review context is assembled. // @UX_STATE: Importing -> Progressive milestones remain visible while review context is assembled.
// @UX_STATE: Review -> Workspace exposes summary controls, timeline, findings, and export affordances. // @UX_STATE: Review -> Workspace exposes summary controls, mounted semantic review, clarification dialog, findings, and export affordances.
// @TEST_CONTRACT: DatasetReviewWorkspaceDependencies -> ObservableWorkspaceUX // @TEST_CONTRACT: DatasetReviewWorkspaceDependencies -> ObservableWorkspaceUX
// @TEST_SCENARIO: empty_state_renders_source_intake -> No session route data shows intake-first workspace. // @TEST_SCENARIO: empty_state_renders_source_intake -> No session route data shows intake-first workspace.
// @TEST_SCENARIO: session_load_surfaces_partial_recovery_state -> Existing session route shows review summary, findings, and partial recovery evidence. // @TEST_SCENARIO: session_load_surfaces_partial_recovery_state -> Existing session route shows review summary, findings, and partial recovery evidence.
// @TEST_SCENARIO: session_load_mounts_us2_semantic_and_clarification_panels -> Existing session route mounts semantic review and clarification UI from real route integration.
// @TEST_SCENARIO: import_submission_transitions_to_review_state -> Intake submit creates a session, navigates, and renders review state. // @TEST_SCENARIO: import_submission_transitions_to_review_state -> Intake submit creates a session, navigates, and renders review state.
// @TEST_SCENARIO: export_feedback_surfaces_success_and_error -> Export action exposes both artifact success detail and failure feedback. // @TEST_SCENARIO: export_feedback_surfaces_success_and_error -> Export action exposes both artifact success detail and failure feedback.
// @TEST_EDGE: missing_field -> Empty session context still renders stable workspace copy. // @TEST_EDGE: missing_field -> Empty session context still renders stable workspace copy.
@@ -44,6 +45,7 @@ function createSessionDetail(overrides = {}) {
active_task_id: "task-77", active_task_id: "task-77",
findings: [ findings: [
{ {
finding_id: "finding-1",
title: "Missing semantic description", title: "Missing semantic description",
code: "SEM-1", code: "SEM-1",
area: "dataset_profile", area: "dataset_profile",
@@ -80,6 +82,82 @@ function createSessionDetail(overrides = {}) {
confidence_state: "mostly_confirmed", confidence_state: "mostly_confirmed",
business_summary: "Imported business description", business_summary: "Imported business description",
}, },
semantic_sources: [
{
source_id: "source-1",
display_name: "Trusted dictionary",
source_version: "v1",
},
],
semantic_fields: [
{
field_id: "field-1",
field_name: "customer_name",
field_kind: "dimension",
verbose_name: "Customer name",
description: "Resolved from trusted dictionary",
display_format: "text",
provenance: "dictionary_exact",
source_id: "source-1",
confidence_rank: 1,
is_locked: false,
has_conflict: true,
needs_review: true,
last_changed_by: "system",
candidates: [
{
candidate_id: "candidate-1",
match_type: "exact",
confidence_score: 0.98,
proposed_verbose_name: "Customer name",
proposed_description: "Customer display name",
proposed_display_format: "text",
status: "pending",
},
],
},
],
clarification_sessions: [
{
clarification_session_id: "clarification-1",
status: "active",
current_question_id: "question-1",
resolved_count: 1,
remaining_count: 2,
summary_delta: "Awaiting user confirmation",
},
],
...overrides,
};
}
function createClarificationState(overrides = {}) {
return {
clarification_session: {
clarification_session_id: "clarification-1",
session_id: "session-1",
status: "active",
current_question_id: "question-1",
resolved_count: 1,
remaining_count: 2,
summary_delta: "Awaiting user confirmation",
},
current_question: {
question_id: "question-1",
clarification_session_id: "clarification-1",
topic_ref: "profile.summary",
question_text: "Which customer label should be used?",
why_it_matters: "This label is shown to reviewers.",
current_guess: "Customer name",
options: [
{
option_id: "option-1",
value: "Customer name",
label: "Customer name",
is_recommended: true,
},
],
},
...overrides, ...overrides,
}; };
} }
@@ -239,6 +317,8 @@ vi.mock("$lib/i18n", () => ({
resume_action: "Resume session", resume_action: "Resume session",
pause_action: "Pause session", pause_action: "Pause session",
jump_target_label: "Jump target", jump_target_label: "Jump target",
semantic_review_visible: "Semantic review visible",
clarification_visible: "Clarification visible",
health_title: "Review health", health_title: "Review health",
open_findings_label: "Open findings", open_findings_label: "Open findings",
exports_title: "Exports", exports_title: "Exports",
@@ -252,6 +332,108 @@ vi.mock("$lib/i18n", () => ({
preview_status_label: "Preview status", preview_status_label: "Preview status",
preview_compiler_label: "Compiled by", preview_compiler_label: "Compiled by",
}, },
semantics: {
eyebrow: "Semantic layer",
title: "Review semantic field values",
description: "Review semantic candidates",
field_count_label: "Fields in review",
empty: "No semantic fields",
unknown_source: "Unknown source",
active_value_label: "Active semantic value",
provenance_label: "Provenance",
confidence_label: "Confidence",
confidence_unset: "No confidence rank",
confidence_rank_label: "Rank",
source_label: "Source",
changed_by_label: "Changed by",
locked_badge: "Locked",
unlocked_badge: "Unlocked",
conflict_badge: "Conflict",
needs_review_badge: "Needs review",
manual_verbose_name_label: "Manual verbose name",
manual_description_label: "Manual description",
manual_display_format_label: "Manual display format",
manual_override_action: "Manual override",
save_manual_action: "Save manual value",
lock_action: "Lock field",
unlock_action: "Unlock field",
feedback_up_action: "Thumbs up",
feedback_down_action: "Thumbs down",
candidates_title: "Candidate options",
candidates_empty: "No candidates",
candidate_description_empty: "No candidate description",
display_format_label: "Display format",
score_label: "Score",
apply_candidate_action: "Apply",
apply_and_lock_action: "Apply and lock",
empty_value: "No value",
provenance: {
unresolved: "Unresolved",
dictionary_exact: "Dictionary exact match",
manual_override: "Manual override",
},
candidate_status: {
pending: "Pending",
accepted: "Accepted",
superseded: "Superseded",
},
messages: {
saving: "Saving semantic decision...",
save_failed: "Failed to save semantic decision.",
manual_override_required: "Manual value required",
manual_saved: "Manual semantic value saved and locked.",
candidate_applied: "Candidate value applied.",
candidate_locked: "Candidate value applied and locked.",
locked: "Field locked against silent overwrite.",
unlocked: "Field unlocked for renewed review.",
feedback_up: "Positive feedback recorded.",
feedback_down: "Negative feedback recorded.",
},
},
clarification: {
eyebrow: "Clarification",
title: "Resolve one ambiguity at a time",
description: "One question at a time",
progress_label: "Clarification progress",
state_label: "Dialog state",
status_label: "Status",
remaining_label: "Remaining questions",
summary_delta_label: "Summary delta",
active_question_label: "Active question",
why_it_matters_label: "Why it matters",
current_guess_label: "Current guess",
current_guess_empty: "No current guess",
topic_label: "Topic",
options_title: "Available options",
recommended_badge: "Recommended",
custom_answer_label: "Custom answer",
answer_action: "Answer with selected option",
custom_answer_action: "Submit custom answer",
skip_action: "Skip for now",
assistant_action: "Open assistant guidance",
expert_review_action: "Send to expert review",
resume_action: "Resume clarification",
completed: "No active clarification question remains.",
feedback_label: "Clarification feedback",
feedback_prompt: "Record whether the clarification result was useful.",
feedback_up_action: "Helpful",
feedback_down_action: "Not helpful",
messages: {
saving: "Saving clarification answer...",
saved: "Clarification answer saved.",
skipped: "Question skipped and left resumable.",
expert_review_requested: "Question escalated to expert review.",
assistant_opened: "Assistant guidance opened.",
resumed: "Clarification resumed.",
resume_failed: "Failed to resume clarification.",
save_failed: "Failed to save clarification answer.",
feedback_up: "Positive clarification feedback recorded.",
feedback_down: "Negative clarification feedback recorded.",
feedback_failed: "Failed to save clarification feedback.",
option_required: "Option required",
custom_required: "Custom answer required",
},
},
}, },
}); });
return () => {}; return () => {};
@@ -311,7 +493,9 @@ describe("DatasetReviewWorkspace UX Contract", () => {
it("session_load_surfaces_partial_recovery_state", async () => { it("session_load_surfaces_partial_recovery_state", async () => {
routeState.id = "session-1"; routeState.id = "session-1";
api.fetchApi.mockResolvedValue(createSessionDetail()); api.fetchApi
.mockResolvedValueOnce(createSessionDetail())
.mockResolvedValueOnce(createClarificationState());
render(DatasetReviewWorkspace); render(DatasetReviewWorkspace);
@@ -319,10 +503,13 @@ describe("DatasetReviewWorkspace UX Contract", () => {
expect(api.fetchApi).toHaveBeenCalledWith("/dataset-orchestration/sessions/session-1"); expect(api.fetchApi).toHaveBeenCalledWith("/dataset-orchestration/sessions/session-1");
}); });
await waitFor(() => {
expect(screen.getByText("Business summary")).toBeDefined();
});
expect(screen.getByText("Workspace state: Review")).toBeDefined(); expect(screen.getByText("Workspace state: Review")).toBeDefined();
expect(screen.getByText("Readiness: Partially ready")).toBeDefined(); expect(screen.getByText("Readiness: Partially ready")).toBeDefined();
expect(screen.getAllByText("Sales Dataset").length).toBeGreaterThan(0); expect(screen.getAllByText("Sales Dataset").length).toBeGreaterThan(0);
expect(screen.getByText("Partial recovery")).toBeDefined();
expect(screen.getAllByText("Recovered filters").length).toBeGreaterThan(0); expect(screen.getAllByText("Recovered filters").length).toBeGreaterThan(0);
expect(screen.getByText("Imported business description")).toBeDefined(); expect(screen.getByText("Imported business description")).toBeDefined();
expect(screen.getAllByRole("button", { name: "Save session" }).length).toBeGreaterThan(0); expect(screen.getAllByRole("button", { name: "Save session" }).length).toBeGreaterThan(0);
@@ -332,9 +519,29 @@ describe("DatasetReviewWorkspace UX Contract", () => {
expect(screen.getByText("Source accepted")).toBeDefined(); expect(screen.getByText("Source accepted")).toBeDefined();
}); });
it("session_load_mounts_us2_semantic_and_clarification_panels", async () => {
routeState.id = "session-1";
api.fetchApi
.mockResolvedValueOnce(createSessionDetail())
.mockResolvedValueOnce(createClarificationState());
render(DatasetReviewWorkspace);
await waitFor(() => {
expect(screen.getByText("Review semantic field values")).toBeDefined();
});
expect(screen.getByText("Resolve one ambiguity at a time")).toBeDefined();
expect(screen.getByText("customer_name")).toBeDefined();
expect(screen.getByText("Which customer label should be used?")).toBeDefined();
expect(screen.getAllByText("Customer name").length).toBeGreaterThan(0);
});
it("summary_edit_affordance_updates_visible_draft", async () => { it("summary_edit_affordance_updates_visible_draft", async () => {
routeState.id = "session-1"; routeState.id = "session-1";
api.fetchApi.mockResolvedValue(createSessionDetail()); api.fetchApi
.mockResolvedValueOnce(createSessionDetail())
.mockResolvedValueOnce(createClarificationState());
render(DatasetReviewWorkspace); render(DatasetReviewWorkspace);
@@ -344,7 +551,7 @@ describe("DatasetReviewWorkspace UX Contract", () => {
await fireEvent.click(screen.getByRole("button", { name: "[Edit]" })); await fireEvent.click(screen.getByRole("button", { name: "[Edit]" }));
const summaryEditor = screen.getByRole("textbox"); const summaryEditor = screen.getAllByRole("textbox")[0];
await fireEvent.input(summaryEditor, { await fireEvent.input(summaryEditor, {
target: { value: "Updated analyst-facing summary" }, target: { value: "Updated analyst-facing summary" },
}); });
@@ -361,12 +568,17 @@ describe("DatasetReviewWorkspace UX Contract", () => {
const createdSession = createSessionDetail(); const createdSession = createSessionDetail();
let resolveSessionLoad; let resolveSessionLoad;
api.postApi.mockResolvedValue({ session_id: "session-1" }); api.postApi.mockResolvedValue({ session_id: "session-1" });
api.fetchApi.mockImplementation( api.fetchApi.mockImplementation((url) => {
() => if (url === "/dataset-orchestration/sessions/session-1") {
new Promise((resolve) => { return new Promise((resolve) => {
resolveSessionLoad = resolve; resolveSessionLoad = resolve;
}), });
); }
if (url === "/dataset-orchestration/sessions/session-1/clarification") {
return Promise.resolve(createClarificationState());
}
return Promise.resolve(undefined);
});
render(DatasetReviewWorkspace); render(DatasetReviewWorkspace);
@@ -417,6 +629,7 @@ describe("DatasetReviewWorkspace UX Contract", () => {
routeState.id = "session-1"; routeState.id = "session-1";
api.fetchApi api.fetchApi
.mockResolvedValueOnce(createSessionDetail()) .mockResolvedValueOnce(createSessionDetail())
.mockResolvedValueOnce(createClarificationState())
.mockResolvedValueOnce({ .mockResolvedValueOnce({
artifact_type: "documentation", artifact_type: "documentation",
format: "json", format: "json",

View File

@@ -115,6 +115,7 @@ This document defines the semantic contracts for the core components of the Data
# @PRE: selected source and target field set must be known. # @PRE: selected source and target field set must be known.
# @POST: candidate ranking follows the configured confidence hierarchy and unresolved fuzzy matches remain reviewable. # @POST: candidate ranking follows the configured confidence hierarchy and unresolved fuzzy matches remain reviewable.
# @SIDE_EFFECT: may create conflict findings and semantic candidate records. # @SIDE_EFFECT: may create conflict findings and semantic candidate records.
# @DATA_CONTRACT: Input[SemanticSourceSelection | SemanticFieldSet | ManualFieldDecision] -> Output[SemanticCandidateSet | RankedSemanticResolution | ValidationFindingSet]
# @INVARIANT: Manual overrides are never silently replaced by imported, inferred, or AI-generated values. # @INVARIANT: Manual overrides are never silently replaced by imported, inferred, or AI-generated values.
# @TEST_CONTRACT: rank_candidates -> exact dictionary beats reference import beats fuzzy beats AI draft # @TEST_CONTRACT: rank_candidates -> exact dictionary beats reference import beats fuzzy beats AI draft
# @TEST_SCENARIO: manual_lock_survives_reimport -> locked field remains active after another source is applied # @TEST_SCENARIO: manual_lock_survives_reimport -> locked field remains active after another source is applied
@@ -156,6 +157,7 @@ This document defines the semantic contracts for the core components of the Data
# @PRE: target session contains unresolved or contradictory review state. # @PRE: target session contains unresolved or contradictory review state.
# @POST: every recorded answer updates the clarification session and associated session state deterministically. # @POST: every recorded answer updates the clarification session and associated session state deterministically.
# @SIDE_EFFECT: creates clarification questions, persists answers, updates findings/profile state. # @SIDE_EFFECT: creates clarification questions, persists answers, updates findings/profile state.
# @DATA_CONTRACT: Input[ClarificationSessionState | ClarificationAnswerCommand] -> Output[ClarificationQuestionPayload | ClarificationProgressSnapshot | SessionReadinessDelta]
# @INVARIANT: Clarification answers are persisted before the current question pointer or readiness state is advanced. # @INVARIANT: Clarification answers are persisted before the current question pointer or readiness state is advanced.
# @TEST_CONTRACT: next_question_selection -> returns only one highest-priority unresolved question at a time # @TEST_CONTRACT: next_question_selection -> returns only one highest-priority unresolved question at a time
# @TEST_SCENARIO: save_and_resume_clarification -> reopening session restores current question and prior answers # @TEST_SCENARIO: save_and_resume_clarification -> reopening session restores current question and prior answers
@@ -184,9 +186,10 @@ This document defines the semantic contracts for the core components of the Data
# @COMPLEXITY: 4 # @COMPLEXITY: 4
# @PURPOSE: Recover dataset, dashboard, filter, and runtime-template context from Superset links and related API payloads. # @PURPOSE: Recover dataset, dashboard, filter, and runtime-template context from Superset links and related API payloads.
# @LAYER: Infra # @LAYER: Infra
# @RELATION: [CALLS] ->[SupersetClient]
# @RELATION: [DEPENDS_ON] ->[ImportedFilter] # @RELATION: [DEPENDS_ON] ->[ImportedFilter]
# @RELATION: [DEPENDS_ON] ->[TemplateVariable] # @RELATION: [DEPENDS_ON] ->[TemplateVariable]
# @RELATION: [DEPENDS_ON] ->[backend.src.core.superset_client.SupersetClient]
# @DATA_CONTRACT: Input[SupersetLink | DatasetReference | EnvironmentContext] -> Output[RecoveredSupersetContext | ImportedFilterSet | TemplateVariableSet | RecoverySummary]
# @PRE: Superset link or dataset reference must be parseable enough to resolve an environment-scoped target resource. # @PRE: Superset link or dataset reference must be parseable enough to resolve an environment-scoped target resource.
# @POST: returns the best available recovered context with explicit provenance and partial-recovery markers when necessary. # @POST: returns the best available recovered context with explicit provenance and partial-recovery markers when necessary.
# @SIDE_EFFECT: performs upstream Superset API reads. # @SIDE_EFFECT: performs upstream Superset API reads.
@@ -218,9 +221,10 @@ This document defines the semantic contracts for the core components of the Data
# @COMPLEXITY: 4 # @COMPLEXITY: 4
# @PURPOSE: Interact with Superset preview compilation and SQL Lab execution endpoints using the current approved execution context. # @PURPOSE: Interact with Superset preview compilation and SQL Lab execution endpoints using the current approved execution context.
# @LAYER: Infra # @LAYER: Infra
# @RELATION: [CALLS] ->[SupersetClient]
# @RELATION: [DEPENDS_ON] ->[CompiledPreview] # @RELATION: [DEPENDS_ON] ->[CompiledPreview]
# @RELATION: [DEPENDS_ON] ->[DatasetRunContext] # @RELATION: [DEPENDS_ON] ->[DatasetRunContext]
# @RELATION: [DEPENDS_ON] ->[backend.src.core.superset_client.SupersetClient]
# @DATA_CONTRACT: Input[ApprovedExecutionContext | PreviewFingerprint | LaunchRequest] -> Output[CompiledPreview | PreviewFailureArtifact | DatasetRunContext | LaunchFailureAudit]
# @PRE: effective template params and dataset execution reference are available. # @PRE: effective template params and dataset execution reference are available.
# @POST: preview and launch calls return Superset-originated artifacts or explicit errors. # @POST: preview and launch calls return Superset-originated artifacts or explicit errors.
# @SIDE_EFFECT: performs upstream Superset preview and SQL Lab calls. # @SIDE_EFFECT: performs upstream Superset preview and SQL Lab calls.

View File

@@ -50,12 +50,12 @@
**Independent Test**: Open a session with unresolved findings; answer questions one by one and verify readiness state updates in real-time. **Independent Test**: Open a session with unresolved findings; answer questions one by one and verify readiness state updates in real-time.
- [ ] T021 [P] [US2] Implement `ClarificationEngine.build_question_payload` (CRITICAL: C4, PRE: unresolved state, POST: prioritized question) in `backend/src/services/dataset_review/clarification_engine.py` - [X] T021 [P] [US2] Implement `ClarificationEngine.build_question_payload` (CRITICAL: C4, PRE: unresolved state, POST: prioritized question) in `backend/src/services/dataset_review/clarification_engine.py`
- [ ] T022 [US2] Implement `ClarificationEngine.record_answer` (CRITICAL: C4, PRE: question active, POST: answer persisted before state advance) in `backend/src/services/dataset_review/clarification_engine.py` - [X] T022 [US2] Implement `ClarificationEngine.record_answer` (CRITICAL: C4, PRE: question active, POST: answer persisted before state advance) in `backend/src/services/dataset_review/clarification_engine.py`
- [ ] T023 [P] [US2] Implement field-level semantic override and lock endpoints in `backend/src/api/routes/dataset_review.py` - [X] T023 [P] [US2] Implement field-level semantic override and lock endpoints in `backend/src/api/routes/dataset_review.py`
- [ ] T024 [US2] Implement `SemanticLayerReview` component (C3, UX_STATE: Conflicted/Manual) in `frontend/src/lib/components/dataset-review/SemanticLayerReview.svelte` - [X] T024 [US2] Implement `SemanticLayerReview` component (C3, UX_STATE: Conflicted/Manual) in `frontend/src/lib/components/dataset-review/SemanticLayerReview.svelte`
- [ ] T025 [P] [US2] Implement `ClarificationDialog` (C3, UX_STATE: Question/Saving/Completed, REL: binds to `assistantChat`) in `frontend/src/lib/components/dataset-review/ClarificationDialog.svelte` - [X] T025 [P] [US2] Implement `ClarificationDialog` (C3, UX_STATE: Question/Saving/Completed, REL: binds to `assistantChat`) in `frontend/src/lib/components/dataset-review/ClarificationDialog.svelte`
- [ ] T026 [US2] Implement LLM feedback (👍/👎) storage and UI handlers in `backend/src/api/routes/dataset_review.py` - [X] T026 [US2] Implement LLM feedback (👍/👎) storage and UI handlers in `backend/src/api/routes/dataset_review.py`
- [ ] T027 [US2] Verify implementation matches ux_reference.md (Happy Path & Errors) - [ ] T027 [US2] Verify implementation matches ux_reference.md (Happy Path & Errors)
- [ ] T028 [US2] Acceptance: Perform semantic audit & algorithm emulation by Tester - [ ] T028 [US2] Acceptance: Perform semantic audit & algorithm emulation by Tester
@@ -67,13 +67,13 @@
**Independent Test**: Map filters to variables; trigger preview; verify launch blocked until preview succeeds; verify SQL Lab session creation. **Independent Test**: Map filters to variables; trigger preview; verify launch blocked until preview succeeds; verify SQL Lab session creation.
- [ ] T029 [P] [US3] Implement `SupersetContextExtractor.recover_imported_filters` and variable discovery in `backend/src/core/utils/superset_context_extractor.py` - [X] T029 [P] [US3] Implement `SupersetContextExtractor.recover_imported_filters` and variable discovery in `backend/src/core/utils/superset_context_extractor.py`
- [ ] T030 [US3] Implement `SupersetCompilationAdapter.compile_preview` (CRITICAL: C4, PRE: effective inputs available, POST: Superset-compiled SQL only) in `backend/src/core/utils/superset_compilation_adapter.py` - [X] T030 [US3] Implement `SupersetCompilationAdapter.compile_preview` (CRITICAL: C4, PRE: effective inputs available, POST: Superset-compiled SQL only) in `backend/src/core/utils/superset_compilation_adapter.py`
- [ ] T031 [US3] Implement `DatasetReviewOrchestrator.launch_dataset` (CRITICAL: C5, PRE: run-ready + preview match, POST: audited run context) in `backend/src/services/dataset_review/orchestrator.py` - [X] T031 [US3] Implement `DatasetReviewOrchestrator.launch_dataset` (CRITICAL: C5, PRE: run-ready + preview match, POST: audited run context) in `backend/src/services/dataset_review/orchestrator.py`
- [ ] T032 [P] [US3] Implement mapping approval and preview trigger endpoints in `backend/src/api/routes/dataset_review.py` - [X] T032 [P] [US3] Implement mapping approval and preview trigger endpoints in `backend/src/api/routes/dataset_review.py`
- [ ] T033 [P] [US3] Implement `ExecutionMappingReview` component (C3, UX_STATE: WarningApproval/Approved) in `frontend/src/lib/components/dataset-review/ExecutionMappingReview.svelte` - [X] T033 [P] [US3] Implement `ExecutionMappingReview` component (C3, UX_STATE: WarningApproval/Approved) in `frontend/src/lib/components/dataset-review/ExecutionMappingReview.svelte`
- [ ] T034 [P] [US3] Implement `CompiledSQLPreview` component (C3, UX_STATE: Ready/Stale/Error) in `frontend/src/lib/components/dataset-review/CompiledSQLPreview.svelte` - [X] T034 [P] [US3] Implement `CompiledSQLPreview` component (C3, UX_STATE: Ready/Stale/Error) in `frontend/src/lib/components/dataset-review/CompiledSQLPreview.svelte`
- [ ] T035 [US3] Implement `LaunchConfirmationPanel` (C3, UX_STATE: Blocked/Ready/Submitted) in `frontend/src/lib/components/dataset-review/LaunchConfirmationPanel.svelte` - [X] T035 [US3] Implement `LaunchConfirmationPanel` (C3, UX_STATE: Blocked/Ready/Submitted) in `frontend/src/lib/components/dataset-review/LaunchConfirmationPanel.svelte`
- [ ] T036 [US3] Verify implementation matches ux_reference.md (Happy Path & Errors) - [ ] T036 [US3] Verify implementation matches ux_reference.md (Happy Path & Errors)
- [ ] T037 [US3] Acceptance: Perform semantic audit & algorithm emulation by Tester - [ ] T037 [US3] Acceptance: Perform semantic audit & algorithm emulation by Tester