FR-019 (was a false [x]): MCP list_checkpoints + decide_checkpoint over the
044 CAS lifecycle (server-resolved pending checkpoint, decision_version CAS,
continue_after_human_decision, typed conflict/not_found); catalog scenario:RUN
and service_allowed=False — automation has no path to checkpoints.
tests/test_mcp_checkpoints.py: 3 passed.
FR-013/SC-007: client_credentials grant for machine clients — confidential DCR
(one-time client_secret, sha256-only via migration 0017_oauth_client_secret
with idempotent guard), signed identity-only service-principal tokens
(principal_type=service, aud=mcp, no refresh), McpTokenVerifier service
short-circuit, AS metadata grants/auth-methods, INSTALL.md §MCP-client docs.
tests/test_mcp_client_flow_http.py: full scripted-client flows over real HTTP
(machine: discovery→DCR→client_credentials→/mcp initialize→tools/list→call;
user: DCR→PKCE S256 authorize via Bearer web session→exchange→/mcp live-RBAC
listing). 2 passed; oauth suite 10 passed.
PRODUCTION DEFECTS fixed en route: call_tool argument-inspection gates parsed
only the FLAT shape while FastMCP delivers {"request": {...}} —
start_scenario_run was uncallable over MCP and the PROD-SQL terminal denial was
bypassable by the wrapped shape. Both gates now unwrap via _gate_arguments;
regression pinned by flat x wrapped PROD matrix (6 combos).
E2E-AUTH-001..003 release-gate rows closed with evidence: T028 chain gained
propose_test_plan (single-trace 001); T023 extended with post-activation
start_scenario_run on the canonical runner-shaped fixture graph, asserting the
queued run pins promoted revision_id + content_hash (003); 002 already proven.
Record honesty: tasks.md T025/T008/T008b/T032 downgraded at review time;
T025+T008 re-closed with executable evidence; T032/T008b/T005a remain open in
the round-2 queue (recorded in WORKSTATE checkpoint with the full P1/P2 list:
MCL intent repair, dispatch/sandbox EXPLORE traces, INV_6 tombstones,
SC-005 remnants, FR-010 versioning, depth/rate limits).
Evidence: combined MCP slice 95 passed; alembic single head 0017; ruff and
compileall clean; anchors balanced in all touched files.
Stop preview/env fallbacks when a configured datasource is gone, skip the
duplicate final insert after streaming, and surface retry/scheduler/LLM
edge cases as FAILED instead of COMPLETED.
The send-path repair branch set _request_result but not
_request_error_code, so the AGENT_REQUEST_FAILED lifecycle event fell back
to the misleading 'agent lifecycle failure'. Set THREAD_REPAIRED so logs
and middleware show the real outcome.
The repair paths (send-path _repair_pending_tool_calls and resume fallback)
called graph.update_state — the SYNC method — which internally invokes
AsyncPostgresSaver.get_tuple() and raises InvalidStateError ('Synchronous
calls to AsyncPostgresSaver are only allowed from a different thread').
The repair therefore always failed (surfacing as 'Event loop is closed' +
a dangling aget_tuple coroutine) and broken threads stayed broken.
Switch both repair sites to agent.aupdate_state (async checkpointer
interface) and align the mocked agent in tests. Verified live: aupdate_state
repaired the broken checkpoint of conversation 69651ca1 (pending calls → 0).
Two recurring failures from live logs (conversations 69651ca1 / a8c0dff8):
1. A checkpoint whose AI messages carry tool_calls without ToolMessages
(run crashed after the LLM emitted a call) makes every send raise
INVALID_CHAT_HISTORY with no recovery. The send path now repairs the
thread via _repair_pending_tool_calls: pending calls are answered with
synthetic error ToolMessages (THREAD_REPAIRED) so the user can retry.
2. Scenario tools scheduled from plain chat (no build_dashboard_test_scenario
UI intent) had no durable AgentRun, so the resume fallback refused with
SCENARIO_RUN_REQUIRED. _ensure_scenario_run now lazily creates the run
from the tool args (dashboard_context from scenario_json for
validate/resolve), mirroring the UIContextV2 scenario contract.
Observed in a live checkpoint: the pending scenario_validate tool call
carried a scenario_json that the LLM stream cut mid-document (ended
inside the unclosed outer object, len 3837). _parse_json_value failed
with a generic 'Could not extract JSON value' that neither the operator
nor the LLM could act on. Add _looks_truncated (unbalanced structure /
unterminated string at end) and report 'truncated/incomplete JSON' with
input length, so the model regenerates the full document.
The backend route binds the search query to `search`; a bare `q` param is
not bound, so search_dashboards and prefetch_dashboards silently returned
the unfiltered catalog (e.g. query 'Sales' listed all 11 dashboards).
Switch both to `search` and update the URL-contract test.
- search_dashboards: call /api/dashboards with page_context=other and
page_size=100 so the profile 'My Dashboards Only' filter can no longer
hide the whole catalog; parse available_total/effective_profile_filter
and report hidden-by-filter instead of a false 'no dashboards' answer
- prefetch_dashboards: same full-catalog context; fix dead code where
data=resp.json() sat after return '' inside the error branch, making
every 200 response raise NameError and the prefetch always return ''
- llm-status: ?force=1 bypasses the 30s health cache so the 'Retry now'
button performs a fresh probe instead of re-reading the stale status;
frontend keeps a single retry interval (previously stacked intervals
decayed the countdown faster than 1/s and fired duplicate probes)
- tests: agent tool/prefetch, backend route bypass + force param,
frontend retry/force coverage
- Replace the global env <select> in TopNavbar with an expandable
EnvironmentStatsWidget showing per-env total/mine/published/drafts
and health status (latency, unreachable), preserving env switching.
- Add GET /api/environments/stats: per-env counts (profile-actor matched)
+ lightweight health probe, gathered concurrently with an 8s probe
timeout and a process-local TTL cache (30s, coalescing) so the full
Superset dashboard catalog is not re-fetched on every dropdown open.
- Add available_total to GET /api/dashboards so grids can show how many
dashboards exist when the profile-default filter hides everything.
- Share ProfileFilterBanner across the dashboards hub and validation
task form: 'showing X of Y' reference info + explicit Show all /
Restore filter actions.
- Russian plural forms for dashboard counts (pluralRu helper) and
compact 'Опубл.' label; i18n keys en/ru.
- Ignore :memory:test_* SQLite test artifacts and drop them from the index.
- Tests: env stats endpoint (incl. caching), widget, model fallback,
plural helper, api client, integration.
- handleSearchInput now receives the selected envId, so the debounced
search actually fires API requests instead of hitting the !envId guard
and clearing results immediately.
- The dashboard section of the global search sends page_context=other,
apply_profile_default=false, override_show_all=true (same pattern as
DashboardHubModel.loadDashboardSearchOptions), so the "show only my
dashboards" profile filter no longer zeroes out dashboards that lack
owner metadata.
- Updated unit tests: debounce now asserts API calls + profile-off flags.
- skill §11 rewritten: one invariant (byte-identical prefix) + explicit
preserve/invalidate lists + discipline (load self-orchestration once;
persona/toolFilter/model fixed for a worker's whole life)
- worker skills: "disposable context" -> "long-lived context"; role lines
now say "leaf, long-lived, refined in place via send_message"
Workers are continuable long-lived children: spawn once, refine in place
via send_message as the feature evolves; a worker's own session persists
and compacts independently. Re-spawn only when the context is poisoned.
- skill §3: delegation tree rewritten — "refine, don't re-spawn"
- skill §5: "refine, don't re-spawn" coordination rule
- skill §7: stalled worker → send_message (keep context), fresh spawn
only for poisoned context
- contracts: Self.Worker.Implement is "long-lived, refined in place";
add long-lived invariant
Drop the "never read raw files" rule — it was a false economy. The
architect's context is large and auto-compacting, so reading is cheap and
necessary for decomposition and verification. The real boundary is
EXECUTION (edits/builds/tests) stays with workers, not READING.
- skill §1/§6/§10: "read freely" replaces "never read raw files";
prefer read_outline/search_contracts to locate, read/grep/glob to understand
- @RATIONALE reframed: reading is cheap; only DECISIONS must not live
solely in context (they go to files)
- contract: invariant becomes "reads freely for decisions, delegates execution"
Session-log review found the orchestrator diverging from its own design.
Encode every divergence as a rule + invariant:
- no polling: get_goal/list_agents are state tools, not completion checks
(settlement/report are the signals) — was 101 get_goal + 98 list_agents
- no raw reads: structure via read_outline/search_contracts; file CONTENT
is delegated (was 77 read vs 1 read_outline)
- <RESULT> enforcement: a worker result with no envelope is blocked
(was 4/19 envelopes)
- closed role taxonomy: only Implement/Verify/Curate (was ad-hoc
"code reviewer"/"auditor"/"adversarial")
- fork semantics: fork inherits MY context, not a stalled worker's;
no "takeover" via fork
- interrupt only to redirect; let workers finish (was 7 interrupts)
- decision memory written by me to files (was zero file writes)
- skill hygiene: load self-orchestration once; never load worker skills
- cross-workspace: point Axiom at the target or delegate all reading
Child subagents join the parent's preset composition, so by default they
inherited the orchestrator persona AND the delegation tools — and drifted
into orchestrating instead of working (observed: a "worker" called
list_agents/get_goal/send_message and spawned its own grandchildren).
Fixes:
- preset: subagent/subagent_fork now carry a role-agnostic worker `persona`,
`toolFilter.deny` for all delegation/coordination tools, and `maxDepth: 1`
(children cannot spawn grandchildren) — hard enforcement at the boundary
- skill: every worker prompt opens with a mandatory role-reset block
- contracts: Self.Worker.Implement/Verify gain a LEAF invariant (no subagents,
no delegation tools)
- ENCRYPTION_KEY: smoke test generates a fresh key inline; templates use a placeholder
- drop RUSAL_ROOT.cer (client-specific public cert) + gitignore it
