#880 - [nomad-step-2] S2.2 — tools/vault-import.sh (import .env + sops into KV) - disinto-admin/disinto

dev-bot commented

2026-04-16 15:26:33 +00:00

Collaborator

Part of the Nomad+Vault migration. Step 2 — Vault policies + workload identity + secrets import.

Goal

Land tools/vault-import.sh — the one-shot import tool that reads the existing .env and sops-encrypted .env.vault.enc from the old docker stack and writes them to Vault KV paths matching the S2.1 policy layout.

Scope

Create tools/vault-import.sh using lib/hvault.sh helpers (P4). Invocation:

tools/vault-import.sh \
  --env /path/to/.env \
  --sops /path/to/.env.vault.enc \
  --age-key /path/to/age/keys.txt

Mapping (per the migration plan):

From .env:
- FORGE_{ROLE}_TOKEN + FORGE_{ROLE}_PASS → kv/disinto/bots/<role>/{token,password} for each of review, dev, gardener, architect, planner, predictor, supervisor, vault.
- FORGE_TOKEN_LLAMA + FORGE_PASS_LLAMA → kv/disinto/bots/dev-qwen/{token,password}.
- FORGE_TOKEN + FORGE_PASS → kv/disinto/shared/forge/{token,password} (generic admin/app creds).
- FORGE_ADMIN_TOKEN → kv/disinto/shared/forge/admin_token.
- WOODPECKER_AGENT_SECRET, WP_FORGEJO_CLIENT, WP_FORGEJO_SECRET, WOODPECKER_TOKEN → kv/disinto/shared/woodpecker/<lowercase_key>.
- FORWARD_AUTH_SECRET, CHAT_OAUTH_CLIENT_ID, CHAT_OAUTH_CLIENT_SECRET → kv/disinto/shared/chat/<lowercase_key>.
From sops-decrypted .env.vault.enc (pipe through sops -d --output-type dotenv with SOPS_AGE_KEY_FILE=<path>):
- Each of GITHUB_TOKEN, CODEBERG_TOKEN, CLAWHUB_TOKEN, DEPLOY_KEY, NPM_TOKEN, DOCKER_HUB_TOKEN → kv/disinto/runner/<NAME>/value.

Idempotency:

Diff existing KV entries via lib/hvault.sh::hvault_kv_get. Report unchanged / updated / created per key.
--dry-run flag: prints the full import plan (paths + which env var source), exits 0 without writing.
Never logs secret values — only key names.

Security:

Refuse to run if VAULT_ADDR is not localhost (avoid accidentally writing to a prod Vault).
Write to KV v2, not v1.
Validate sops age key file is mode 0400 before sourcing — refuse loudly if world-readable.

Acceptance criteria

Against a fixture .env + fixture sops file (both in tests/fixtures/), writes expected paths to a dev-mode Vault launched in CI.
Re-run with unchanged fixtures: all keys report unchanged.
Re-run with one fixture value modified: only that one reports updated.
--dry-run mode prints the plan, doesn't touch Vault.
tests/fixtures/dot-env-incomplete (missing a required key) → clear error naming the missing key.
shellcheck clean.
Never logs a secret value, including on errors. Test: grep the stdout+stderr of a successful run for known fixture values, expect zero matches.

Non-goals

Not wiring into disinto init (S2.5 does that).
Not managing Vault auth or policies (S2.1 + S2.3).
Not handling Woodpecker UI-stored secrets (those need WP API export, deferred to Step 3 or later).

Labels / meta

[nomad-step-2] S2.2 — no dependencies.

Part of the Nomad+Vault migration. **Step 2 — Vault policies + workload identity + secrets import.** ## Goal Land `tools/vault-import.sh` — the one-shot import tool that reads the existing `.env` and sops-encrypted `.env.vault.enc` from the old docker stack and writes them to Vault KV paths matching the S2.1 policy layout. ## Scope Create `tools/vault-import.sh` using `lib/hvault.sh` helpers (P4). Invocation: ```bash tools/vault-import.sh \ --env /path/to/.env \ --sops /path/to/.env.vault.enc \ --age-key /path/to/age/keys.txt ``` Mapping (per the migration plan): - From `.env`: - `FORGE_{ROLE}_TOKEN` + `FORGE_{ROLE}_PASS` → `kv/disinto/bots/<role>/{token,password}` for each of review, dev, gardener, architect, planner, predictor, supervisor, vault. - `FORGE_TOKEN_LLAMA` + `FORGE_PASS_LLAMA` → `kv/disinto/bots/dev-qwen/{token,password}`. - `FORGE_TOKEN` + `FORGE_PASS` → `kv/disinto/shared/forge/{token,password}` (generic admin/app creds). - `FORGE_ADMIN_TOKEN` → `kv/disinto/shared/forge/admin_token`. - `WOODPECKER_AGENT_SECRET`, `WP_FORGEJO_CLIENT`, `WP_FORGEJO_SECRET`, `WOODPECKER_TOKEN` → `kv/disinto/shared/woodpecker/<lowercase_key>`. - `FORWARD_AUTH_SECRET`, `CHAT_OAUTH_CLIENT_ID`, `CHAT_OAUTH_CLIENT_SECRET` → `kv/disinto/shared/chat/<lowercase_key>`. - From sops-decrypted `.env.vault.enc` (pipe through `sops -d --output-type dotenv` with `SOPS_AGE_KEY_FILE=<path>`): - Each of `GITHUB_TOKEN`, `CODEBERG_TOKEN`, `CLAWHUB_TOKEN`, `DEPLOY_KEY`, `NPM_TOKEN`, `DOCKER_HUB_TOKEN` → `kv/disinto/runner/<NAME>/value`. Idempotency: - Diff existing KV entries via `lib/hvault.sh::hvault_kv_get`. Report `unchanged` / `updated` / `created` per key. - `--dry-run` flag: prints the full import plan (paths + which env var source), exits 0 without writing. - Never logs secret values — only key names. Security: - Refuse to run if `VAULT_ADDR` is not localhost (avoid accidentally writing to a prod Vault). - Write to KV v2, not v1. - Validate sops age key file is mode 0400 before sourcing — refuse loudly if world-readable. ## Acceptance criteria - Against a fixture `.env` + fixture sops file (both in `tests/fixtures/`), writes expected paths to a dev-mode Vault launched in CI. - Re-run with unchanged fixtures: all keys report `unchanged`. - Re-run with one fixture value modified: only that one reports `updated`. - `--dry-run` mode prints the plan, doesn't touch Vault. - `tests/fixtures/dot-env-incomplete` (missing a required key) → clear error naming the missing key. - `shellcheck` clean. - Never logs a secret value, including on errors. Test: grep the stdout+stderr of a successful run for known fixture values, expect zero matches. ## Non-goals - Not wiring into `disinto init` (S2.5 does that). - Not managing Vault auth or policies (S2.1 + S2.3). - Not handling Woodpecker UI-stored secrets (those need WP API export, deferred to Step 3 or later). ## Labels / meta - `[nomad-step-2] S2.2` — no dependencies.