diff --git a/AGENTS.md b/AGENTS.md index 64fba19..f64f775 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -24,7 +24,7 @@ disinto/ │ preflight.sh — pre-flight data collection for supervisor formula │ supervisor/journal/ — daily health logs from each run │ supervisor-poll.sh — legacy bash orchestrator (superseded) -├── vault/ vault-poll.sh, vault-agent.sh, vault-fire.sh — action gating +├── vault/ vault-poll.sh, vault-agent.sh, vault-fire.sh — action gating + procurement ├── action/ action-poll.sh, action-agent.sh — operational task execution ├── lib/ env.sh, agent-session.sh, ci-helpers.sh, ci-debug.sh, load-project.sh, parse-deps.sh, matrix_listener.sh ├── projects/ *.toml — per-project config @@ -78,7 +78,7 @@ bash dev/phase-test.sh | Planner | `planner/` | Strategic planning | [planner/AGENTS.md](planner/AGENTS.md) | | Predictor | `predictor/` | Infrastructure pattern detection | [predictor/AGENTS.md](predictor/AGENTS.md) | | Action | `action/` | Operational task execution | [action/AGENTS.md](action/AGENTS.md) | -| Vault | `vault/` | Safety gate for dangerous actions | [vault/AGENTS.md](vault/AGENTS.md) | +| Vault | `vault/` | Action gating + resource procurement | [vault/AGENTS.md](vault/AGENTS.md) | See [lib/AGENTS.md](lib/AGENTS.md) for the full shared helper reference. diff --git a/formulas/run-planner.toml b/formulas/run-planner.toml index 8bc3931..ab14325 100644 --- a/formulas/run-planner.toml +++ b/formulas/run-planner.toml @@ -212,12 +212,22 @@ Update the tree by applying these operations: 5. **Propose new capabilities**: If you identify a capability the factory needs (e.g., "marketing formula, runs weekly"), add it to the tree as - a proposed prerequisite. Anything with recurring cost should note: - "→ vault approval required" so the planner files it to vault next run. + a proposed prerequisite. Anything with recurring cost (new accounts, + new infra, new cron entries, new formulas) should be procured through + the vault — see the file-at-constraints step for how to file requests. -6. **Check vault decisions**: Read any open vault issues to see if - previously proposed capabilities have been approved or rejected. - Update the tree accordingly. +6. **Check vault state**: Scan vault directories for procurement status: + - `$FACTORY_ROOT/vault/pending/*.md` — requests awaiting human action. + Any prerequisite that depends on a pending procurement request should + be marked: `[ ] ⏳ blocked-on-vault (vault/pending/.md)` + - `$FACTORY_ROOT/vault/approved/*.md` — fulfilled, being processed. + - `$FACTORY_ROOT/vault/fired/*.md` — completed. Check if the resource + now appears in RESOURCES.md and mark the prerequisite resolved. + - Do NOT file issues for objectives blocked on pending vault items. + +7. **Re-read RESOURCES.md**: Check for newly available capabilities that + were not present last run. If a new resource appears, mark the + corresponding prerequisite as resolved. Write the updated tree to: $FACTORY_ROOT/planner/prerequisite-tree.md Use this format: @@ -228,7 +238,8 @@ Use this format: ## Objective: (#issue or description) - [x] Resolved prerequisite (reference) - [ ] Unresolved prerequisite (#issue or description) - Status: READY | BLOCKED — | DONE + - [ ] Resource need ⏳ blocked-on-vault (vault/pending/.md) + Status: READY | BLOCKED — | BLOCKED — awaiting vault | DONE Keep the tree focused — only include objectives from VISION.md milestones and their genuine prerequisites. Do not inflate the tree with nice-to-haves. @@ -293,9 +304,53 @@ Rules: - When deploying/operating, reference the resource alias from RESOURCES.md - Promoted predictions from triage may become constraints if they block downstream objectives — rank them the same way +- **Do NOT file issues for objectives blocked on pending vault items** — + these are waiting for human procurement, not dev work -If all top 3 constraints already have open issues, note that the backlog -is aligned with the constraint focus. No new issues needed. +### Filing vault procurement requests + +If a constraint requires a resource the factory does not have (check +RESOURCES.md), and that resource has recurring cost (account, infra, +domain, API key, new cron job), file a procurement request instead of +an issue: + +1. Check if a request already exists in vault/pending/ or vault/approved/ + for this resource (match by filename). + +2. If no request exists, create a markdown file at: + $FACTORY_ROOT/vault/pending/.md + + Format: + ``` + # Procurement Request: + + ## What + + + ## Why + + + ## Unblocks + + + ## Proposed RESOURCES.md Entry + ## + - type: + - capability: + - env: + ``` + +3. Mark the prerequisite in the tree as blocked-on-vault: + `[ ] ⏳ blocked-on-vault (vault/pending/.md)` + +4. vault-poll.sh will notify the human automatically. + +Procurement requests count toward the 3-item-per-run limit (issues + +procurement requests combined). + +If all top 3 constraints already have open issues or pending vault +requests, note that the backlog is aligned with the constraint focus. +No new items needed. """ needs = ["update-prerequisite-tree"] diff --git a/vault/AGENTS.md b/vault/AGENTS.md index c14d9d3..aa7416d 100644 --- a/vault/AGENTS.md +++ b/vault/AGENTS.md @@ -1,19 +1,35 @@ # Vault Agent -**Role**: Safety gate for dangerous or irreversible actions. Actions enter a -pending queue and are classified by Claude via `vault-agent.sh`, which can -auto-approve (call `vault-fire.sh` directly), auto-reject (call -`vault-reject.sh`), or escalate to a human via Matrix for APPROVE/REJECT. +**Role**: Dual-purpose gate — action safety classification and resource procurement. + +**Pipeline A — Action Gating (*.json)**: Actions enter a pending queue and are +classified by Claude via `vault-agent.sh`, which can auto-approve (call +`vault-fire.sh` directly), auto-reject (call `vault-reject.sh`), or escalate +to a human via Matrix for APPROVE/REJECT. + +**Pipeline B — Procurement (*.md)**: The planner files resource requests as +markdown files in `vault/pending/`. `vault-poll.sh` notifies the human via +Matrix. The human fulfills the request (creates accounts, provisions infra, +adds secrets to `.env`) and moves the file to `vault/approved/`. +`vault-fire.sh` then extracts the proposed entry and appends it to +`RESOURCES.md`. **Trigger**: `vault-poll.sh` runs every 30 min via cron. **Key files**: -- `vault/vault-poll.sh` — Processes pending actions: retry approved, auto-reject after 48h timeout, invoke vault-agent for new items -- `vault/vault-agent.sh` — Classifies and routes pending actions via `claude -p`: auto-approve, auto-reject, or escalate to human +- `vault/vault-poll.sh` — Processes pending items: retry approved, auto-reject after 48h timeout, invoke vault-agent for JSON actions, notify human for procurement requests +- `vault/vault-agent.sh` — Classifies and routes pending JSON actions via `claude -p`: auto-approve, auto-reject, or escalate to human - `vault/PROMPT.md` — System prompt for the vault agent's Claude invocation -- `vault/vault-fire.sh` — Executes an approved action -- `vault/vault-reject.sh` — Marks an action as rejected +- `vault/vault-fire.sh` — Executes an approved action (JSON) or writes RESOURCES.md entry (procurement MD) +- `vault/vault-reject.sh` — Marks a JSON action as rejected + +**Procurement flow**: +1. Planner drops `vault/pending/.md` with what/why/proposed RESOURCES.md entry +2. `vault-poll.sh` notifies human via Matrix +3. Human fulfills: creates account, adds secrets to `.env`, moves file to `vault/approved/` +4. `vault-fire.sh` extracts proposed entry, appends to RESOURCES.md, moves to `vault/fired/` +5. Next planner run reads RESOURCES.md → new capability available → unblocks prerequisite tree **Environment variables consumed**: - All from `lib/env.sh` diff --git a/vault/PROMPT.md b/vault/PROMPT.md index 3dec30c..9d37aa3 100644 --- a/vault/PROMPT.md +++ b/vault/PROMPT.md @@ -4,9 +4,24 @@ You are the vault agent for `$CODEBERG_REPO`. You were called by `vault-poll.sh` because one or more actions in `vault/pending/` need classification and routing. -## Your Job +## Two Pipelines -For each pending action, decide: **auto-approve**, **escalate**, or **reject**. +The vault handles two kinds of items: + +### A. Action Gating (*.json) +Actions from agents that need safety classification before execution. +You classify and route these: auto-approve, escalate, or reject. + +### B. Procurement Requests (*.md) +Resource requests from the planner. These always escalate to the human — +you do NOT auto-approve or reject procurement requests. The human fulfills +the request (creates accounts, provisions infra, adds secrets to .env) +and moves the file from `vault/pending/` to `vault/approved/`. +`vault-fire.sh` then writes the RESOURCES.md entry. + +## Your Job (Action Gating only) + +For each pending JSON action, decide: **auto-approve**, **escalate**, or **reject**. ## Routing Table (risk × reversibility) @@ -28,6 +43,8 @@ For each pending action, decide: **auto-approve**, **escalate**, or **reject**. 4. **Malformed JSON → reject** with reason `malformed`. 5. **Payload validation:** Check that the payload has the minimum required fields for the action type. Missing fields → reject with reason. +6. **Procurement requests (*.md) → skip.** These are handled by the human + directly. Do not attempt to classify, approve, or reject them. ## Action Type Defaults @@ -41,6 +58,29 @@ For each pending action, decide: **auto-approve**, **escalate**, or **reject**. | `webhook-call` | medium | depends | | `stripe-charge` | high | no | +## Procurement Request Format (reference only) + +Procurement requests dropped by the planner look like: + +```markdown +# Procurement Request: + +## What + + +## Why + + +## Unblocks + + +## Proposed RESOURCES.md Entry +## +- type: +- capability: +- env: +``` + ## Available Tools You have shell access. Use these for routing decisions: @@ -83,8 +123,10 @@ ROUTE: ## Important -- Process ALL pending actions in the batch. Never skip silently. +- Process ALL pending JSON actions in the batch. Never skip silently. - For auto-approved actions, fire them immediately via `vault-fire.sh`. - For escalated actions, move to `vault/approved/` only AFTER human approval (vault-poll handles this via matrix_listener dispatch). - Read the action JSON carefully. Check the payload, not just the metadata. +- Ignore `.md` files in pending/ — those are procurement requests handled + separately by vault-poll.sh and the human. diff --git a/vault/vault-fire.sh b/vault/vault-fire.sh index e943327..136e9ed 100755 --- a/vault/vault-fire.sh +++ b/vault/vault-fire.sh @@ -1,11 +1,14 @@ #!/usr/bin/env bash -# vault-fire.sh — Execute an approved vault action by ID +# vault-fire.sh — Execute an approved vault item by ID # -# Two-phase: pending/ → approved/ → fired/ -# If action is in pending/, moves to approved/ first. -# If action is already in approved/, fires directly (crash recovery). +# Handles two pipelines: +# A. Action gating (*.json): pending/ → approved/ → fired/ +# B. Procurement (*.md): approved/ → fired/ (writes RESOURCES.md entry) # -# Usage: bash vault-fire.sh +# If item is in pending/, moves to approved/ first. +# If item is already in approved/, fires directly (crash recovery). +# +# Usage: bash vault-fire.sh set -euo pipefail @@ -15,27 +18,38 @@ source "${SCRIPT_DIR}/../lib/env.sh" VAULT_DIR="${FACTORY_ROOT}/vault" LOCKS_DIR="${VAULT_DIR}/.locks" LOGFILE="${VAULT_DIR}/vault.log" +RESOURCES_FILE="${PROJECT_REPO_ROOT:-${FACTORY_ROOT}}/RESOURCES.md" log() { printf '[%s] vault-fire: %s\n' "$(date -u '+%Y-%m-%d %H:%M:%S UTC')" "$*" >> "$LOGFILE" } -ACTION_ID="${1:?Usage: vault-fire.sh }" +ACTION_ID="${1:?Usage: vault-fire.sh }" -# Locate the action file +# ============================================================================= +# Detect pipeline: procurement (.md) or action gating (.json) +# ============================================================================= +IS_PROCUREMENT=false ACTION_FILE="" -if [ -f "${VAULT_DIR}/approved/${ACTION_ID}.json" ]; then + +if [ -f "${VAULT_DIR}/approved/${ACTION_ID}.md" ]; then + IS_PROCUREMENT=true + ACTION_FILE="${VAULT_DIR}/approved/${ACTION_ID}.md" +elif [ -f "${VAULT_DIR}/pending/${ACTION_ID}.md" ]; then + IS_PROCUREMENT=true + mv "${VAULT_DIR}/pending/${ACTION_ID}.md" "${VAULT_DIR}/approved/${ACTION_ID}.md" + ACTION_FILE="${VAULT_DIR}/approved/${ACTION_ID}.md" + log "$ACTION_ID: pending → approved (procurement)" +elif [ -f "${VAULT_DIR}/approved/${ACTION_ID}.json" ]; then ACTION_FILE="${VAULT_DIR}/approved/${ACTION_ID}.json" elif [ -f "${VAULT_DIR}/pending/${ACTION_ID}.json" ]; then - # Phase 1: move pending → approved mv "${VAULT_DIR}/pending/${ACTION_ID}.json" "${VAULT_DIR}/approved/${ACTION_ID}.json" ACTION_FILE="${VAULT_DIR}/approved/${ACTION_ID}.json" - # Update status in the file TMP=$(mktemp) jq '.status = "approved"' "$ACTION_FILE" > "$TMP" && mv "$TMP" "$ACTION_FILE" log "$ACTION_ID: pending → approved" else - log "ERROR: action $ACTION_ID not found in pending/ or approved/" + log "ERROR: item $ACTION_ID not found in pending/ or approved/" exit 1 fi @@ -52,7 +66,41 @@ fi echo $$ > "$LOCKFILE" trap 'rm -f "$LOCKFILE"' EXIT -# Read action metadata +# ============================================================================= +# Pipeline A: Procurement — extract RESOURCES.md entry and append +# ============================================================================= +if [ "$IS_PROCUREMENT" = true ]; then + log "$ACTION_ID: firing procurement request" + + # Extract the proposed RESOURCES.md entry from the markdown file. + # The entry is between "## Proposed RESOURCES.md Entry" and the next "## " heading or EOF. + ENTRY="" + ENTRY=$(sed -n '/^## Proposed RESOURCES\.md Entry/,/^## /{/^## Proposed RESOURCES\.md Entry/d;/^## /d;p}' "$ACTION_FILE" 2>/dev/null || true) + + # Strip leading/trailing blank lines and markdown code fences + ENTRY=$(echo "$ENTRY" | sed '/^```/d' | sed -e '/./,$!d' -e :a -e '/^\n*$/{$d;N;ba;}') + + if [ -z "$ENTRY" ]; then + log "ERROR: $ACTION_ID has no '## Proposed RESOURCES.md Entry' section" + matrix_send "vault" "❌ Procurement $ACTION_ID has no RESOURCES.md entry — cannot fire" 2>/dev/null || true + exit 1 + fi + + # Append entry to RESOURCES.md + printf '\n%s\n' "$ENTRY" >> "$RESOURCES_FILE" + log "$ACTION_ID: wrote RESOURCES.md entry" + + # Move to fired/ + mv "$ACTION_FILE" "${VAULT_DIR}/fired/${ACTION_ID}.md" + rm -f "${LOCKS_DIR}/${ACTION_ID}.notified" + log "$ACTION_ID: approved → fired (procurement)" + matrix_send "vault" "✅ Procurement fulfilled: ${ACTION_ID} — RESOURCES.md updated" 2>/dev/null || true + exit 0 +fi + +# ============================================================================= +# Pipeline B: Action gating — dispatch to handler +# ============================================================================= ACTION_TYPE=$(jq -r '.type // ""' < "$ACTION_FILE") ACTION_SOURCE=$(jq -r '.source // ""' < "$ACTION_FILE") PAYLOAD=$(jq -c '.payload // {}' < "$ACTION_FILE") @@ -64,9 +112,6 @@ fi log "$ACTION_ID: firing type=$ACTION_TYPE source=$ACTION_SOURCE" -# ============================================================================= -# Dispatch to handler -# ============================================================================= FIRE_EXIT=0 case "$ACTION_TYPE" in diff --git a/vault/vault-poll.sh b/vault/vault-poll.sh index 13e01ef..706e67a 100755 --- a/vault/vault-poll.sh +++ b/vault/vault-poll.sh @@ -1,10 +1,15 @@ #!/usr/bin/env bash -# vault-poll.sh — Vault gate agent: process pending actions, retry approved, timeout escalations +# vault-poll.sh — Vault: process pending actions + procurement requests # -# Runs every 30min via cron. Processes actions through the vault pipeline: -# 1. Retry any approved/ actions that weren't fired (crash recovery) +# Runs every 30min via cron. Two pipelines: +# A. Action gating (*.json): auto-approve/escalate/reject via vault-agent.sh +# B. Procurement (*.md): notify human, fire approved requests via vault-fire.sh +# +# Phases: +# 1. Retry any approved/ items that weren't fired (crash recovery) # 2. Auto-reject escalations with no reply for 48h -# 3. Invoke vault-agent.sh for new pending/ actions +# 3. Invoke vault-agent.sh for new pending JSON actions +# 4. Notify human about new pending procurement requests (.md) # # Cron: */30 * * * * /path/to/disinto/vault/vault-poll.sh # @@ -67,9 +72,9 @@ unlock_action() { } # ============================================================================= -# PHASE 1: Retry approved actions (crash recovery) +# PHASE 1: Retry approved items (crash recovery — JSON actions + MD procurement) # ============================================================================= -status "phase 1: retrying approved actions" +status "phase 1: retrying approved items" for action_file in "${VAULT_DIR}/approved/"*.json; do [ -f "$action_file" ] || continue @@ -92,6 +97,27 @@ for action_file in "${VAULT_DIR}/approved/"*.json; do unlock_action "$ACTION_ID" done +# Retry approved procurement requests (.md) +for req_file in "${VAULT_DIR}/approved/"*.md; do + [ -f "$req_file" ] || continue + REQ_ID=$(basename "$req_file" .md) + + if ! lock_action "$REQ_ID"; then + log "skip procurement $REQ_ID — locked by another process" + continue + fi + + log "retrying approved procurement: $REQ_ID" + if bash "${VAULT_DIR}/vault-fire.sh" "$REQ_ID" >> "$LOGFILE" 2>&1; then + log "fired procurement $REQ_ID (retry)" + else + log "ERROR: fire failed for procurement $REQ_ID (retry)" + matrix_send "vault" "❌ Vault fire failed on retry: ${REQ_ID} (procurement)" 2>/dev/null || true + fi + + unlock_action "$REQ_ID" +done + # ============================================================================= # PHASE 2: Timeout escalations (48h no reply → auto-reject) # ============================================================================= @@ -122,7 +148,7 @@ for action_file in "${VAULT_DIR}/pending/"*.json; do done # ============================================================================= -# PHASE 3: Process new pending actions +# PHASE 3: Process new pending actions (JSON — action gating) # ============================================================================= status "phase 3: processing pending actions" @@ -152,17 +178,62 @@ for action_file in "${VAULT_DIR}/pending/"*.json; do unlock_action "$ACTION_ID" done -if [ "$PENDING_COUNT" -eq 0 ]; then - status "all clear — no pending actions" - exit 0 +if [ "$PENDING_COUNT" -gt 0 ]; then + log "found $PENDING_COUNT pending action(s), invoking vault-agent" + status "invoking vault-agent for $PENDING_COUNT action(s)" + + bash "${VAULT_DIR}/vault-agent.sh" >> "$LOGFILE" 2>&1 || { + log "ERROR: vault-agent failed" + matrix_send "vault" "❌ vault-agent.sh failed — check vault.log" 2>/dev/null || true + } fi -log "found $PENDING_COUNT pending action(s), invoking vault-agent" -status "invoking vault-agent for $PENDING_COUNT action(s)" +# ============================================================================= +# PHASE 4: Notify human about new pending procurement requests (.md) +# ============================================================================= +status "phase 4: processing pending procurement requests" -bash "${VAULT_DIR}/vault-agent.sh" >> "$LOGFILE" 2>&1 || { - log "ERROR: vault-agent failed" - matrix_send "vault" "❌ vault-agent.sh failed — check vault.log" 2>/dev/null || true -} +PROCURE_COUNT=0 -status "poll complete" +for req_file in "${VAULT_DIR}/pending/"*.md; do + [ -f "$req_file" ] || continue + REQ_ID=$(basename "$req_file" .md) + + # Check if already notified (marker file) + if [ -f "${VAULT_DIR}/.locks/${REQ_ID}.notified" ]; then + continue + fi + + if ! lock_action "$REQ_ID"; then + log "skip procurement $REQ_ID — locked" + continue + fi + + PROCURE_COUNT=$((PROCURE_COUNT + 1)) + + # Extract title from first heading + REQ_TITLE=$(grep -m1 '^# ' "$req_file" | sed 's/^# //' || echo "$REQ_ID") + + log "new procurement request: $REQ_ID — $REQ_TITLE" + + # Notify human via Matrix + matrix_send "vault" "🔑 PROCUREMENT REQUEST — ${REQ_TITLE} + +ID: ${REQ_ID} +Action: review vault/pending/${REQ_ID}.md +To approve: fulfill the request, add secrets to .env, move file to vault/approved/ + +$(head -20 "$req_file")" 2>/dev/null || true + + # Mark as notified so we don't re-send + mkdir -p "${VAULT_DIR}/.locks" + touch "${VAULT_DIR}/.locks/${REQ_ID}.notified" + + unlock_action "$REQ_ID" +done + +if [ "$PENDING_COUNT" -eq 0 ] && [ "$PROCURE_COUNT" -eq 0 ]; then + status "all clear — no pending items" +else + status "poll complete — ${PENDING_COUNT} action(s), ${PROCURE_COUNT} procurement request(s)" +fi