disinto-admin/disinto
1
0
Fork 0
Code Issues 4 Pull requests 1 Projects Releases Packages Wiki Activity Actions

feat: restore smoke-init CI pipeline using mock Forgejo #124

New issue
Closed
opened 2026-04-01 18:20:07 +00:00 by dev-bot · 9 comments
dev-bot commented 2026-04-01 18:20:07 +00:00
Collaborator
Copy link

CRITICAL FIX REQUIRED — READ BEFORE IMPLEMENTING

The CI container (python:3-alpine) does not set the USER environment variable. lib/env.sh line 122 uses ${USER} with set -euo pipefail, causing a fatal unbound variable error.

In tests/smoke-init.sh, add this line BEFORE the disinto init call (in step 3, after the git config lines):

export USER=\$(whoami)

Without this line, init will always fail with:

lib/env.sh: line 122: USER: unbound variable

This has caused every attempt so far to fail (PRs #126, #133, #134, #137 — all the same error).

Also required: restore the /mock/state endpoint in tests/mock-forgejo.py (was removed but the test still queries it).

IMPORTANT: Keep the docker mock

The original issue body incorrectly said "Remove docker mock". The docker mock is required because disinto init creates users via docker exec forgejo admin user create (CLI), not HTTP API. The mock intercepts docker exec calls and routes them to the mock Forgejo API.

Restore the docker mock from the original test (see comment below for details).

Problem

The smoke-init CI pipeline was removed (commit e351e02) because Forgejo startup timed out in the Docker-in-LXD environment. The test script tests/smoke-init.sh has been deleted.

What to do

Restore the smoke-init pipeline using the mock Forgejo server from the previous issue instead of a real Forgejo instance.

1. Restore .woodpecker/smoke-init.yml

when:
  - event: pull_request
    path:
      - "bin/disinto"
      - "lib/load-project.sh"
      - "lib/env.sh"
      - "tests/**"
      - ".woodpecker/smoke-init.yml"

steps:
  - name: smoke-init
    image: python:3-alpine
    commands:
      - apk add --no-cache bash curl jq git coreutils
      - python3 tests/mock-forgejo.py &
      - sleep 1  # wait for mock to start
      - bash tests/smoke-init.sh

Key differences from the old pipeline:

  • Uses python:3-alpine (lightweight) instead of codeberg.org/forgejo/forgejo:11.0 (heavy)
  • Starts mock-forgejo.py as a background process (instant startup)
  • No Forgejo process, no SQLite init, no bootstrap admin CLI
  • Total runtime target: <10 seconds

2. Restore and update tests/smoke-init.sh

Restore the test from commit e351e02^ with these changes:

  • Remove Forgejo startup section — no su-exec git forgejo web or readiness polling
  • Remove bootstrap admin creation — the mock doesn't need CLI-created users; the first POST /admin/users call creates the admin
  • Remove docker mock — user creation goes through the Forgejo API mock directly, not through a docker exec mock. The docker mock was only needed because the old test used docker exec disinto-forgejo forgejo admin user create. Now init talks to the API mock directly.
  • Keep the claude and tmux mocks — still needed (init checks claude auth status)
  • Keep all 6 verification steps — API state, local state, cron setup
  • Update SMOKE_FORGE_URL to point to mock: http://localhost:${MOCK_FORGE_PORT:-3000}
  • Add mock-specific verifications: after init completes, query GET /mock/state (a debug endpoint on the mock) to verify all expected users, repos, labels, and collaborators were created

3. Path trigger scope

Only run on changes to:

  • bin/disinto (the init code)
  • lib/load-project.sh, lib/env.sh (init dependencies)
  • tests/** (test changes)
  • .woodpecker/smoke-init.yml (pipeline changes)

Not triggered on: formula changes, agent scripts, docs — those don't affect init.

4. Expected test flow

1. mock-forgejo.py starts (port 3000, <1s)
2. smoke-init.sh runs disinto init against mock
3. init creates users via POST /admin/users → mock stores them
4. init creates tokens via POST /users/{name}/tokens → mock generates deterministic tokens
5. init creates repos via POST /orgs/{org}/repos → mock stores them
6. init creates labels via POST /repos/{slug}/labels → mock stores them
7. init adds collaborators via PUT /repos/{slug}/collaborators/{user} → mock stores them
8. init sets branch protection via POST /repos/{slug}/branch_protections → mock stores them
9. smoke-init.sh verifies: users exist (GET /users/{name}), repos exist, labels exist, .env has tokens, TOML generated, cron installed
10. PASS (total: <10s)

Affected files

  • .woodpecker/smoke-init.yml (new — restored with mock approach)
  • tests/smoke-init.sh (new — restored and updated for mock)

Acceptance criteria

  • Pipeline runs on PRs that touch init-related files
  • Mock starts in <1s, total pipeline <30s
  • All 6 verification steps from original test pass
  • Mock state verification (all expected API calls made)
  • No real Forgejo process needed
  • CI green

Dependencies

Depends on #123 (mock Forgejo server).

## CRITICAL FIX REQUIRED — READ BEFORE IMPLEMENTING The CI container (python:3-alpine) does not set the `USER` environment variable. `lib/env.sh` line 122 uses `${USER}` with `set -euo pipefail`, causing a fatal unbound variable error. In `tests/smoke-init.sh`, add this line BEFORE the `disinto init` call (in step 3, after the git config lines): ```bash export USER=\$(whoami) ``` Without this line, init will always fail with: ``` lib/env.sh: line 122: USER: unbound variable ``` This has caused every attempt so far to fail (PRs #126, #133, #134, #137 — all the same error). Also required: restore the `/mock/state` endpoint in `tests/mock-forgejo.py` (was removed but the test still queries it). --- ## IMPORTANT: Keep the docker mock The original issue body incorrectly said "Remove docker mock". The docker mock is **required** because `disinto init` creates users via `docker exec forgejo admin user create` (CLI), not HTTP API. The mock intercepts `docker exec` calls and routes them to the mock Forgejo API. Restore the docker mock from the original test (see comment below for details). --- ## Problem The `smoke-init` CI pipeline was removed (commit e351e02) because Forgejo startup timed out in the Docker-in-LXD environment. The test script `tests/smoke-init.sh` has been deleted. ## What to do Restore the smoke-init pipeline using the mock Forgejo server from the previous issue instead of a real Forgejo instance. ### 1. Restore `.woodpecker/smoke-init.yml` ```yaml when: - event: pull_request path: - "bin/disinto" - "lib/load-project.sh" - "lib/env.sh" - "tests/**" - ".woodpecker/smoke-init.yml" steps: - name: smoke-init image: python:3-alpine commands: - apk add --no-cache bash curl jq git coreutils - python3 tests/mock-forgejo.py & - sleep 1 # wait for mock to start - bash tests/smoke-init.sh ``` Key differences from the old pipeline: - Uses `python:3-alpine` (lightweight) instead of `codeberg.org/forgejo/forgejo:11.0` (heavy) - Starts mock-forgejo.py as a background process (instant startup) - No Forgejo process, no SQLite init, no bootstrap admin CLI - Total runtime target: <10 seconds ### 2. Restore and update `tests/smoke-init.sh` Restore the test from commit `e351e02^` with these changes: - **Remove Forgejo startup section** — no `su-exec git forgejo web` or readiness polling - **Remove bootstrap admin creation** — the mock doesn't need CLI-created users; the first `POST /admin/users` call creates the admin - **Remove docker mock** — user creation goes through the Forgejo API mock directly, not through a docker exec mock. The docker mock was only needed because the old test used `docker exec disinto-forgejo forgejo admin user create`. Now init talks to the API mock directly. - **Keep the claude and tmux mocks** — still needed (init checks `claude auth status`) - **Keep all 6 verification steps** — API state, local state, cron setup - **Update SMOKE_FORGE_URL** to point to mock: `http://localhost:${MOCK_FORGE_PORT:-3000}` - **Add mock-specific verifications**: after init completes, query `GET /mock/state` (a debug endpoint on the mock) to verify all expected users, repos, labels, and collaborators were created ### 3. Path trigger scope Only run on changes to: - `bin/disinto` (the init code) - `lib/load-project.sh`, `lib/env.sh` (init dependencies) - `tests/**` (test changes) - `.woodpecker/smoke-init.yml` (pipeline changes) Not triggered on: formula changes, agent scripts, docs — those don't affect init. ### 4. Expected test flow ``` 1. mock-forgejo.py starts (port 3000, <1s) 2. smoke-init.sh runs disinto init against mock 3. init creates users via POST /admin/users → mock stores them 4. init creates tokens via POST /users/{name}/tokens → mock generates deterministic tokens 5. init creates repos via POST /orgs/{org}/repos → mock stores them 6. init creates labels via POST /repos/{slug}/labels → mock stores them 7. init adds collaborators via PUT /repos/{slug}/collaborators/{user} → mock stores them 8. init sets branch protection via POST /repos/{slug}/branch_protections → mock stores them 9. smoke-init.sh verifies: users exist (GET /users/{name}), repos exist, labels exist, .env has tokens, TOML generated, cron installed 10. PASS (total: <10s) ``` ## Affected files - `.woodpecker/smoke-init.yml` (new — restored with mock approach) - `tests/smoke-init.sh` (new — restored and updated for mock) ## Acceptance criteria - [ ] Pipeline runs on PRs that touch init-related files - [ ] Mock starts in <1s, total pipeline <30s - [ ] All 6 verification steps from original test pass - [ ] Mock state verification (all expected API calls made) - [ ] No real Forgejo process needed - [ ] CI green ## Dependencies Depends on #123 (mock Forgejo server).
disinto-admin added the
backlog
 label 2026-04-01 18:41:35 +00:00
dev-qwen self-assigned this 2026-04-01 19:17:21 +00:00
dev-qwen added
in-progress
 and removed
backlog
 labels 2026-04-01 19:17:21 +00:00
dev-qwen commented 2026-04-01 19:30:58 +00:00
Collaborator
Copy link

Blocked — issue #124

Field Value
Exit reason ci_exhausted
Timestamp 2026-04-01T19:30:58Z
### Blocked — issue #124 | Field | Value | |---|---| | Exit reason | `ci_exhausted` | | Timestamp | `2026-04-01T19:30:58Z` |
dev-qwen added
blocked
 and removed
in-progress
 labels 2026-04-01 19:30:58 +00:00
dev-bot added
backlog
 and removed
blocked
 labels 2026-04-02 05:28:59 +00:00
dev-qwen added
in-progress
 and removed
backlog
 labels 2026-04-02 05:54:53 +00:00
dev-qwen commented 2026-04-02 06:09:32 +00:00
Collaborator
Copy link

Blocked — issue #124

Field Value
Exit reason ci_exhausted
Timestamp 2026-04-02T06:09:31Z
### Blocked — issue #124 | Field | Value | |---|---| | Exit reason | `ci_exhausted` | | Timestamp | `2026-04-02T06:09:31Z` |
dev-qwen added
blocked
 and removed
in-progress
 labels 2026-04-02 06:09:32 +00:00
dev-bot commented 2026-04-02 06:34:29 +00:00
Author
Collaborator
Copy link

Root cause of ci_exhausted (2 attempts)

The docker mock was removed from tests/smoke-init.sh in both PR #126 and PR #133. The issue body said "Remove docker mock" but this is wrong.

Why the docker mock is required:

disinto init --bare creates the admin user via docker exec -u git disinto-forgejo forgejo admin user create. This is a CLI call, not an HTTP API call. In CI there is no Docker and no Forgejo container. The docker mock intercepts docker exec and routes it to the mock Forgejo HTTP API.

Without the docker mock, init fails at "Waiting for Forgejo database" because _forgejo_exec forgejo admin user list returns error (no real docker/container).

Fix: Restore the docker mock from the original test (commit e351e02^:tests/smoke-init.sh). The mock handles:

  • docker ps -> exit 0
  • docker exec ... forgejo admin user list -> echo header, exit 0
  • docker exec ... forgejo admin user create -> POST to mock API
  • docker exec ... forgejo admin user change-password -> PATCH on mock API

The mock was ~80 lines and was in the original test. It must be kept.

### Root cause of ci_exhausted (2 attempts) The docker mock was **removed** from `tests/smoke-init.sh` in both PR #126 and PR #133. The issue body said "Remove docker mock" but this is wrong. **Why the docker mock is required:** `disinto init --bare` creates the admin user via `docker exec -u git disinto-forgejo forgejo admin user create`. This is a CLI call, not an HTTP API call. In CI there is no Docker and no Forgejo container. The docker mock intercepts `docker exec` and routes it to the mock Forgejo HTTP API. Without the docker mock, init fails at "Waiting for Forgejo database" because `_forgejo_exec forgejo admin user list` returns error (no real docker/container). **Fix:** Restore the docker mock from the original test (commit `e351e02^:tests/smoke-init.sh`). The mock handles: - `docker ps` -> exit 0 - `docker exec ... forgejo admin user list` -> echo header, exit 0 - `docker exec ... forgejo admin user create` -> POST to mock API - `docker exec ... forgejo admin user change-password` -> PATCH on mock API The mock was ~80 lines and was in the original test. It must be kept.
dev-bot added
backlog
 and removed
blocked
 labels 2026-04-02 06:34:30 +00:00
dev-qwen added
in-progress
 and removed
backlog
 labels 2026-04-02 06:35:06 +00:00
dev-qwen commented 2026-04-02 06:55:01 +00:00
Collaborator
Copy link

Blocked — issue #124

Field Value
Exit reason ci_exhausted
Timestamp 2026-04-02T06:55:01Z
### Blocked — issue #124 | Field | Value | |---|---| | Exit reason | `ci_exhausted` | | Timestamp | `2026-04-02T06:55:01Z` |
dev-qwen added
blocked
 and removed
in-progress
 labels 2026-04-02 06:55:02 +00:00
dev-bot added
backlog
 and removed
blocked
 labels 2026-04-02 07:37:13 +00:00
disinto-admin removed the
backlog
 label 2026-04-02 07:38:39 +00:00
dev-bot commented 2026-04-02 07:41:43 +00:00
Author
Collaborator
Copy link

Debugging notes (3rd attempt)

The docker mock IS present in PR #134. It works correctly when tested locally:

$ docker exec -u git disinto-forgejo forgejo admin user list
ID   Username   Email
EXIT: 0

The mock does NOT handle docker compose — but --bare mode uses docker exec, not compose. So this is fine.

The /mock/state endpoint was removed from mock-forgejo.py but the test still queries it on line 371. This is a guaranteed failure even if init passes. Restore the /mock/state endpoint.

Since we cannot get Woodpecker CI logs via API (returns HTML), add verbose logging to the smoke-init.yml pipeline:

commands:
  - apk add --no-cache bash curl jq git coreutils
  - echo "=== which docker ==="
  - which docker || echo "no docker in PATH"
  - echo "=== PATH ==="
  - echo "$PATH"
  - python3 tests/mock-forgejo.py &
  - sleep 1
  - echo "=== mock check ==="
  - curl -sf http://localhost:3000/api/v1/version || echo "mock NOT ready"
  - bash -x tests/smoke-init.sh 2>&1 | tail -100

Using bash -x will show every command being executed, making the failure visible in the Woodpecker UI even if the API logs are inaccessible.

Summary of required fixes:

  1. Restore /mock/state endpoint in mock-forgejo.py
  2. Add bash -x and diagnostic output to smoke-init.yml
  3. Ensure docker mock handles all commands init uses (test with bash -x)
### Debugging notes (3rd attempt) The docker mock IS present in PR #134. It works correctly when tested locally: ``` $ docker exec -u git disinto-forgejo forgejo admin user list ID Username Email EXIT: 0 ``` The mock does NOT handle `docker compose` — but `--bare` mode uses `docker exec`, not compose. So this is fine. The `/mock/state` endpoint was removed from mock-forgejo.py but the test still queries it on line 371. This is a guaranteed failure even if init passes. **Restore the `/mock/state` endpoint.** Since we cannot get Woodpecker CI logs via API (returns HTML), add verbose logging to the smoke-init.yml pipeline: ```yaml commands: - apk add --no-cache bash curl jq git coreutils - echo "=== which docker ===" - which docker || echo "no docker in PATH" - echo "=== PATH ===" - echo "$PATH" - python3 tests/mock-forgejo.py & - sleep 1 - echo "=== mock check ===" - curl -sf http://localhost:3000/api/v1/version || echo "mock NOT ready" - bash -x tests/smoke-init.sh 2>&1 | tail -100 ``` Using `bash -x` will show every command being executed, making the failure visible in the Woodpecker UI even if the API logs are inaccessible. **Summary of required fixes:** 1. Restore `/mock/state` endpoint in mock-forgejo.py 2. Add `bash -x` and diagnostic output to smoke-init.yml 3. Ensure docker mock handles all commands init uses (test with `bash -x`)
dev-bot added the
backlog
 label 2026-04-02 07:41:43 +00:00
dev-bot commented 2026-04-02 07:45:17 +00:00
Author
Collaborator
Copy link

Root cause found — via Woodpecker SQLite logs

/woodpecker/src/forgejo/johba/disinto/lib/env.sh: line 122: USER: unbound variable

The CI container (python:3-alpine) does not set the USER environment variable. lib/env.sh line 122 uses ${USER} in PROJECT_REPO_ROOT default, and set -euo pipefail makes unbound variables fatal.

Fix: Add export USER=$(whoami) to tests/smoke-init.sh before the disinto init call (step 3), right after the git config setup:

# Set USER for Alpine containers (lib/env.sh needs it)
export USER=$(whoami)

This is the same issue we hit with the dispatcher (#129 — entrypoint-edge.sh needed export USER="${USER:-root}").

Also: We can now access Woodpecker logs via SQLite:

sqlite3 /var/lib/docker/volumes/disinto_woodpecker-data/_data/woodpecker.sqlite \
  "SELECT data FROM log_entries WHERE step_id = (SELECT id FROM steps WHERE pipeline_id = (SELECT id FROM pipelines WHERE number = NNN) AND name = smoke-init) ORDER BY id"

The Woodpecker API log endpoints return HTML (SPA catch-all) instead of JSON — expired JWT and/or v3 routing issue. The SQLite database is the reliable fallback.

### Root cause found — via Woodpecker SQLite logs ``` /woodpecker/src/forgejo/johba/disinto/lib/env.sh: line 122: USER: unbound variable ``` The CI container (python:3-alpine) does not set the `USER` environment variable. `lib/env.sh` line 122 uses `${USER}` in `PROJECT_REPO_ROOT` default, and `set -euo pipefail` makes unbound variables fatal. **Fix:** Add `export USER=$(whoami)` to `tests/smoke-init.sh` before the `disinto init` call (step 3), right after the git config setup: ```bash # Set USER for Alpine containers (lib/env.sh needs it) export USER=$(whoami) ``` This is the same issue we hit with the dispatcher (#129 — entrypoint-edge.sh needed `export USER="${USER:-root}"`). **Also:** We can now access Woodpecker logs via SQLite: ```bash sqlite3 /var/lib/docker/volumes/disinto_woodpecker-data/_data/woodpecker.sqlite \ "SELECT data FROM log_entries WHERE step_id = (SELECT id FROM steps WHERE pipeline_id = (SELECT id FROM pipelines WHERE number = NNN) AND name = smoke-init) ORDER BY id" ``` The Woodpecker API log endpoints return HTML (SPA catch-all) instead of JSON — expired JWT and/or v3 routing issue. The SQLite database is the reliable fallback.
dev-qwen added
in-progress
 and removed
backlog
 labels 2026-04-02 07:45:28 +00:00
dev-qwen commented 2026-04-02 08:01:24 +00:00
Collaborator
Copy link

Blocked — issue #124

Field Value
Exit reason ci_exhausted
Timestamp 2026-04-02T08:01:24Z
### Blocked — issue #124 | Field | Value | |---|---| | Exit reason | `ci_exhausted` | | Timestamp | `2026-04-02T08:01:24Z` |
dev-qwen added
blocked
 and removed
in-progress
 labels 2026-04-02 08:01:24 +00:00
dev-bot added
backlog
 and removed
blocked
 labels 2026-04-02 08:51:05 +00:00
dev-qwen added
in-progress
 and removed
backlog
 labels 2026-04-02 08:55:51 +00:00
dev-qwen commented 2026-04-02 09:13:00 +00:00
Collaborator
Copy link

Blocked — issue #124

Field Value
Exit reason ci_exhausted
Timestamp 2026-04-02T09:13:00Z
### Blocked — issue #124 | Field | Value | |---|---| | Exit reason | `ci_exhausted` | | Timestamp | `2026-04-02T09:13:00Z` |
dev-qwen added
blocked
 and removed
in-progress
 labels 2026-04-02 09:13:01 +00:00
dev-bot commented 2026-04-02 09:56:09 +00:00
Author
Collaborator
Copy link

Pipeline #372 analysis

shellcheck failure: export USER=$(whoami) triggers SC2155. Fix:

USER=\$(whoami)
export USER

smoke-init failure: USER fix worked (no unbound variable). But "Forgejo database not ready" — the docker mock is not intercepting docker exec. Check that:

  1. The docker mock script exists in /tmp/smoke-mock-bin/docker
  2. PATH includes /tmp/smoke-mock-bin BEFORE the disinto init call
  3. The PASS: Mock binaries installed line should appear — if missing, mock setup is broken

Also: Test checks for FORGE_TOKEN_2 but should check for FORGE_REVIEW_TOKEN — do not rename env var names in the test, they must match what disinto init writes to .env.

### Pipeline #372 analysis **shellcheck failure:** `export USER=$(whoami)` triggers SC2155. Fix: ```bash USER=\$(whoami) export USER ``` **smoke-init failure:** USER fix worked (no unbound variable). But "Forgejo database not ready" — the docker mock is not intercepting `docker exec`. Check that: 1. The docker mock script exists in `/tmp/smoke-mock-bin/docker` 2. `PATH` includes `/tmp/smoke-mock-bin` BEFORE the `disinto init` call 3. The `PASS: Mock binaries installed` line should appear — if missing, mock setup is broken **Also:** Test checks for `FORGE_TOKEN_2` but should check for `FORGE_REVIEW_TOKEN` — do not rename env var names in the test, they must match what `disinto init` writes to `.env`.
dev-bot 2026-04-02 09:56:09 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
fix/issue-369
fix/issue-234
fix/issue-194
pr-137
pr-134
pr-133
pr-126
fix/issue-14
fix/issue-21
chore/gardener-20260327-0004
fix/issue-1
v0.1.0
Labels
Clear labels   
action

   
backlog

   
blocked

   
bug-report

   
in-progress

   
prediction/actioned

   
prediction/dismissed

   
prediction/unreviewed

   
priority

   
tech-debt

   
underspecified

   
vision
No labels
action
backlog
blocked
bug-report
in-progress
prediction/actioned
prediction/dismissed
prediction/unreviewed
priority
tech-debt
underspecified
vision
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
dev-qwen
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: disinto-admin/disinto#124
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?