feat: CI log access — disinto ci-logs + dev-agent CI failure context #136

New issue

Closed

opened 2026-04-02 08:11:06 +00:00 by dev-bot · 0 comments

dev-bot commented

2026-04-02 08:11:06 +00:00

Collaborator

Problem

When CI fails on a dev-agent PR, the agent has no access to the actual CI log output. It only gets "CI failed" status and guesses what's wrong. This leads to repeated blind fix attempts (issue #124 took 8+ attempts because the agent couldn't see USER: unbound variable in the logs).

The Woodpecker API log endpoints return HTML (SPA catch-all) instead of JSON — a v3 routing issue. But the logs are stored in the Woodpecker SQLite database and are accessible directly.

What to do

1. `disinto ci-logs <pipeline-number>` CLI command

Add a subcommand to bin/disinto that reads CI logs from the Woodpecker SQLite database:

disinto ci-logs 346
# or with step filter:
disinto ci-logs 346 --step smoke-init

Implementation: use Python's built-in sqlite3 module (no new dependency):

import sqlite3
db = sqlite3.connect("/path/to/woodpecker.sqlite")
cursor = db.execute("""
    SELECT le.data FROM log_entries le
    JOIN steps s ON le.step_id = s.id
    JOIN pipelines p ON s.pipeline_id = p.id
    WHERE p.number = ? AND s.name = ?
    ORDER BY le.id
""", (pipeline_number, step_name))
for row in cursor:
    print(row[0])

The SQLite path is: the Woodpecker data volume mountpoint + /woodpecker.sqlite. This can be discovered via docker volume inspect disinto_woodpecker-data.

2. Mount Woodpecker SQLite into agents containers (read-only)

Add to docker-compose.yml (and bin/disinto generate_compose) for both agents and agents-llama services:

volumes:
  - woodpecker-data:/woodpecker-data:ro

This gives the agents read-only access to the Woodpecker SQLite database. The agents container already has Python 3 with the built-in sqlite3 module.

3. `lib/ci-helpers.sh` — add `ci_get_logs` function

ci_get_logs() {
    local pipeline_number="$1"
    local step_name="${2:-}"
    python3 "$FACTORY_ROOT/lib/ci-log-reader.py" "$pipeline_number" "$step_name"
}

Create lib/ci-log-reader.py (~30 lines) that:

Opens the SQLite database at /woodpecker-data/woodpecker.sqlite (container path)
Queries log_entries joined with steps and pipelines
If step_name given, filter to that step; otherwise return all failed steps
Outputs the log text to stdout
Truncates to last 200 lines if output is very large (avoid context bloat)

4. Update `lib/pr-lifecycle.sh` — inject CI logs on failure

In pr_walk_to_merge, when CI fails and the agent is re-invoked to fix it, include the actual CI log output in the prompt:

# When CI fails, get the log
PIPELINE_NUM=$(ci_pipeline_number "$HEAD_SHA")
CI_LOG=$(ci_get_logs "$PIPELINE_NUM" 2>/dev/null | tail -50)

# Include in the fix prompt
PROMPT="CI failed on pipeline #${PIPELINE_NUM}. Here is the log output:

\`\`\`
${CI_LOG}
\`\`\`

Fix the issue and push."

This gives the agent the exact error message instead of forcing it to guess.

Affected files

bin/disinto (add ci-logs subcommand)
lib/ci-log-reader.py (new — ~30 lines Python)
lib/ci-helpers.sh (add ci_get_logs wrapper)
lib/pr-lifecycle.sh (inject CI logs into fix prompt)
docker-compose.yml / bin/disinto generate_compose (mount woodpecker-data volume)
lib/AGENTS.md (document new function)

Acceptance criteria

disinto ci-logs <N> prints CI log output for pipeline N
disinto ci-logs <N> --step <name> filters to specific step
Agents containers have read-only access to Woodpecker SQLite
ci_get_logs function available in lib/ci-helpers.sh
When CI fails, pr_walk_to_merge includes actual log output in fix prompt
Log output truncated to last 200 lines to avoid context bloat
CI green

## Problem When CI fails on a dev-agent PR, the agent has no access to the actual CI log output. It only gets "CI failed" status and guesses what's wrong. This leads to repeated blind fix attempts (issue #124 took 8+ attempts because the agent couldn't see `USER: unbound variable` in the logs). The Woodpecker API log endpoints return HTML (SPA catch-all) instead of JSON — a v3 routing issue. But the logs are stored in the Woodpecker SQLite database and are accessible directly. ## What to do ### 1. `disinto ci-logs <pipeline-number>` CLI command Add a subcommand to `bin/disinto` that reads CI logs from the Woodpecker SQLite database: ```bash disinto ci-logs 346 # or with step filter: disinto ci-logs 346 --step smoke-init ``` Implementation: use Python's built-in `sqlite3` module (no new dependency): ```python import sqlite3 db = sqlite3.connect("/path/to/woodpecker.sqlite") cursor = db.execute(""" SELECT le.data FROM log_entries le JOIN steps s ON le.step_id = s.id JOIN pipelines p ON s.pipeline_id = p.id WHERE p.number = ? AND s.name = ? ORDER BY le.id """, (pipeline_number, step_name)) for row in cursor: print(row[0]) ``` The SQLite path is: the Woodpecker data volume mountpoint + `/woodpecker.sqlite`. This can be discovered via `docker volume inspect disinto_woodpecker-data`. ### 2. Mount Woodpecker SQLite into agents containers (read-only) Add to `docker-compose.yml` (and `bin/disinto` generate_compose) for both `agents` and `agents-llama` services: ```yaml volumes: - woodpecker-data:/woodpecker-data:ro ``` This gives the agents read-only access to the Woodpecker SQLite database. The agents container already has Python 3 with the built-in `sqlite3` module. ### 3. `lib/ci-helpers.sh` — add `ci_get_logs` function ```bash ci_get_logs() { local pipeline_number="$1" local step_name="${2:-}" python3 "$FACTORY_ROOT/lib/ci-log-reader.py" "$pipeline_number" "$step_name" } ``` Create `lib/ci-log-reader.py` (~30 lines) that: - Opens the SQLite database at `/woodpecker-data/woodpecker.sqlite` (container path) - Queries log_entries joined with steps and pipelines - If step_name given, filter to that step; otherwise return all failed steps - Outputs the log text to stdout - Truncates to last 200 lines if output is very large (avoid context bloat) ### 4. Update `lib/pr-lifecycle.sh` — inject CI logs on failure In `pr_walk_to_merge`, when CI fails and the agent is re-invoked to fix it, include the actual CI log output in the prompt: ```bash # When CI fails, get the log PIPELINE_NUM=$(ci_pipeline_number "$HEAD_SHA") CI_LOG=$(ci_get_logs "$PIPELINE_NUM" 2>/dev/null | tail -50) # Include in the fix prompt PROMPT="CI failed on pipeline #${PIPELINE_NUM}. Here is the log output: \`\`\` ${CI_LOG} \`\`\` Fix the issue and push." ``` This gives the agent the exact error message instead of forcing it to guess. ## Affected files - `bin/disinto` (add `ci-logs` subcommand) - `lib/ci-log-reader.py` (new — ~30 lines Python) - `lib/ci-helpers.sh` (add `ci_get_logs` wrapper) - `lib/pr-lifecycle.sh` (inject CI logs into fix prompt) - `docker-compose.yml` / `bin/disinto` generate_compose (mount woodpecker-data volume) - `lib/AGENTS.md` (document new function) ## Acceptance criteria - [ ] `disinto ci-logs <N>` prints CI log output for pipeline N - [ ] `disinto ci-logs <N> --step <name>` filters to specific step - [ ] Agents containers have read-only access to Woodpecker SQLite - [ ] `ci_get_logs` function available in lib/ci-helpers.sh - [ ] When CI fails, pr_walk_to_merge includes actual log output in fix prompt - [ ] Log output truncated to last 200 lines to avoid context bloat - [ ] CI green