fix: tech-debt: sweep cron-isms from code comments, helpers, lib, and public site copy (#548)
- Rename acquire_cron_lock → acquire_run_lock in lib/formula-session.sh and all five *-run.sh call sites - Update all *-run.sh file headers: "Cron wrapper" → "Polling-loop wrapper" - Rewrite docs/updating-factory.md: replace crontab check with pgrep, replace "Crontab empty after restart" section with polling-loop equivalent - Update docs/EVAL-MCP-SERVER.md to reflect polling-loop reality - Update lib/guard.sh, lib/AGENTS.md, lib/ci-setup.sh comments - Update formulas/*.toml comments (cron → polling loop) - Update dev/dev-poll.sh usage comment - Update tests/smoke-init.sh to handle compose vs bare-metal scheduling - Update .woodpecker/agent-smoke.sh comments - Update site HTML: architecture.html, quickstart.html, index.html - Clarify _install_cron_impl is bare-metal only (compose uses polling loop) - Keep site/collect-engagement.sh and site/collect-metrics.sh cron refs (genuinely cron-driven on the website host, separate from factory loop) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
da4d9077dd
commit
f0c3c773ff
24 changed files with 115 additions and 132 deletions
|
|
@ -12,8 +12,8 @@ sourced as needed.
|
|||
| `lib/ci-log-reader.py` | Python tool: reads CI logs from Woodpecker SQLite database. `<pipeline_number> [--step <name>]` — returns last 200 lines from failed steps (or specified step). Used by `ci_get_logs()` in ci-helpers.sh. Requires `WOODPECKER_DATA_DIR` (default: /woodpecker-data). | ci-helpers.sh |
|
||||
| `lib/load-project.sh` | Parses a `projects/*.toml` file into env vars (`PROJECT_NAME`, `FORGE_REPO`, `WOODPECKER_REPO_ID`, monitoring toggles, mirror config, etc.). Also exports `FORGE_REPO_OWNER` (the owner component of `FORGE_REPO`, e.g. `disinto-admin` from `disinto-admin/disinto`). **Container path derivation**: `PROJECT_REPO_ROOT` and `OPS_REPO_ROOT` are derived at runtime when `DISINTO_CONTAINER=1` — hardcoded to `/home/agent/repos/$PROJECT_NAME` and `/home/agent/repos/$PROJECT_NAME-ops` respectively — not read from the TOML. This ensures correct paths inside containers where host paths in the TOML would be wrong. | env.sh (when `PROJECT_TOML` is set) |
|
||||
| `lib/parse-deps.sh` | Extracts dependency issue numbers from an issue body (stdin → stdout, one number per line). Matches `## Dependencies` / `## Depends on` / `## Blocked by` sections and inline `depends on #N` / `blocked by #N` patterns. Inline scan skips fenced code blocks to prevent false positives from code examples in issue bodies. Not sourced — executed via `bash lib/parse-deps.sh`. | dev-poll |
|
||||
| `lib/formula-session.sh` | `acquire_cron_lock()`, `load_formula()`, `load_formula_or_profile()`, `build_context_block()`, `ensure_ops_repo()`, `ops_commit_and_push()`, `build_prompt_footer()`, `build_sdk_prompt_footer()`, `formula_worktree_setup()`, `formula_prepare_profile_context()`, `formula_lessons_block()`, `profile_write_journal()`, `profile_load_lessons()`, `ensure_profile_repo()`, `_profile_has_repo()`, `_count_undigested_journals()`, `_profile_digest_journals()`, `_profile_commit_and_push()`, `resolve_agent_identity()`, `build_graph_section()`, `build_scratch_instruction()`, `read_scratch_context()`, `cleanup_stale_crashed_worktrees()` — shared helpers for formula-driven cron agents (lock, .profile repo management, prompt assembly, worktree setup). Memory guard is provided by `memory_guard()` in `lib/env.sh` (not duplicated here). `resolve_agent_identity()` — sets `FORGE_TOKEN`, `AGENT_IDENTITY`, `FORGE_REMOTE` from per-agent token env vars and FORGE_URL remote detection. `build_graph_section()` generates the structural-analysis section (runs `lib/build-graph.py`, formats JSON output) — previously duplicated in planner-run.sh and predictor-run.sh, now shared here. `cleanup_stale_crashed_worktrees()` — thin wrapper around `worktree_cleanup_stale()` from `lib/worktree.sh` (kept for backwards compatibility). | planner-run.sh, predictor-run.sh, gardener-run.sh, supervisor-run.sh, dev-agent.sh |
|
||||
| `lib/guard.sh` | `check_active(agent_name)` — reads `$FACTORY_ROOT/state/.{agent_name}-active`; exits 0 (skip) if the file is absent. Factory is off by default — state files must be created to enable each agent. **Logs a message to stderr** when skipping (`[check_active] SKIP: state file not found`), so agent dropout is visible in cron logs. Sourced by dev-poll.sh, review-poll.sh, predictor-run.sh, supervisor-run.sh. | cron entry points |
|
||||
| `lib/formula-session.sh` | `acquire_run_lock()`, `load_formula()`, `load_formula_or_profile()`, `build_context_block()`, `ensure_ops_repo()`, `ops_commit_and_push()`, `build_prompt_footer()`, `build_sdk_prompt_footer()`, `formula_worktree_setup()`, `formula_prepare_profile_context()`, `formula_lessons_block()`, `profile_write_journal()`, `profile_load_lessons()`, `ensure_profile_repo()`, `_profile_has_repo()`, `_count_undigested_journals()`, `_profile_digest_journals()`, `_profile_commit_and_push()`, `resolve_agent_identity()`, `build_graph_section()`, `build_scratch_instruction()`, `read_scratch_context()`, `cleanup_stale_crashed_worktrees()` — shared helpers for formula-driven polling-loop agents (lock, .profile repo management, prompt assembly, worktree setup). Memory guard is provided by `memory_guard()` in `lib/env.sh` (not duplicated here). `resolve_agent_identity()` — sets `FORGE_TOKEN`, `AGENT_IDENTITY`, `FORGE_REMOTE` from per-agent token env vars and FORGE_URL remote detection. `build_graph_section()` generates the structural-analysis section (runs `lib/build-graph.py`, formats JSON output) — previously duplicated in planner-run.sh and predictor-run.sh, now shared here. `cleanup_stale_crashed_worktrees()` — thin wrapper around `worktree_cleanup_stale()` from `lib/worktree.sh` (kept for backwards compatibility). | planner-run.sh, predictor-run.sh, gardener-run.sh, supervisor-run.sh, dev-agent.sh |
|
||||
| `lib/guard.sh` | `check_active(agent_name)` — reads `$FACTORY_ROOT/state/.{agent_name}-active`; exits 0 (skip) if the file is absent. Factory is off by default — state files must be created to enable each agent. **Logs a message to stderr** when skipping (`[check_active] SKIP: state file not found`), so agent dropout is visible in loop logs. Sourced by dev-poll.sh, review-poll.sh, predictor-run.sh, supervisor-run.sh. | polling-loop entry points |
|
||||
| `lib/mirrors.sh` | `mirror_push()` — pushes `$PRIMARY_BRANCH` + tags to all configured mirror remotes (fire-and-forget background pushes). Reads `MIRROR_NAMES` and `MIRROR_*` vars exported by `load-project.sh` from the `[mirrors]` TOML section. Failures are logged but never block the pipeline. Sourced by dev-poll.sh — called after every successful merge. | dev-poll.sh |
|
||||
| `lib/build-graph.py` | Python tool: parses VISION.md, prerequisites.md (from ops repo), AGENTS.md, formulas/*.toml, evidence/ (from ops repo), and forge issues/labels into a NetworkX DiGraph. Runs structural analyses (orphaned objectives, stale prerequisites, thin evidence, circular deps) and outputs a JSON report. Used by `review-pr.sh` (per-PR changed-file analysis) and `predictor-run.sh` (full-project analysis) to provide structural context to Claude. | review-pr.sh, predictor-run.sh |
|
||||
| `lib/secret-scan.sh` | `scan_for_secrets()` — detects potential secrets (API keys, bearer tokens, private keys, URLs with embedded credentials) in text; returns 1 if secrets found. `redact_secrets()` — replaces detected secret patterns with `[REDACTED]`. | issue-lifecycle.sh |
|
||||
|
|
@ -28,7 +28,7 @@ sourced as needed.
|
|||
| `lib/forge-setup.sh` | `setup_forge()` — Forgejo instance provisioning: creates admin user, bot accounts, org, repos (code + ops), configures webhooks, sets repo topics. Extracted from `bin/disinto`. Requires `FORGE_URL`, `FORGE_TOKEN`, `FACTORY_ROOT`. **Password storage (#361)**: after creating each bot account, stores its password in `.env` as `FORGE_<BOT>_PASS` (e.g. `FORGE_PASS`, `FORGE_REVIEW_PASS`, etc.) for use by `forge-push.sh`. | bin/disinto (init) |
|
||||
| `lib/forge-push.sh` | `push_to_forge()` — pushes a local clone to the Forgejo remote and verifies the push. `_assert_forge_push_globals()` validates required env vars before use. Requires `FORGE_URL`, `FORGE_PASS`, `FACTORY_ROOT`, `PRIMARY_BRANCH`. **Auth**: uses `FORGE_PASS` (bot password) for git HTTP push — Forgejo 11.x rejects API tokens for `git push` (#361). | bin/disinto (init) |
|
||||
| `lib/ops-setup.sh` | `setup_ops_repo()` — creates ops repo on Forgejo if it doesn't exist, configures bot collaborators, clones/initializes ops repo locally, seeds directory structure (vault, knowledge, evidence, sprints). Evidence subdirectories seeded: engagement/, red-team/, holdout/, evolution/, user-test/. Also seeds sprints/ for architect output. Exports `_ACTUAL_OPS_SLUG`. `migrate_ops_repo(ops_root, [primary_branch])` — idempotent migration helper that seeds missing directories and .gitkeep files on existing ops repos (pre-#407 deployments). | bin/disinto (init) |
|
||||
| `lib/ci-setup.sh` | `_install_cron_impl()` — installs crontab entries for project agents. `_create_woodpecker_oauth_impl()` — creates OAuth2 app on Forgejo for Woodpecker. `_generate_woodpecker_token_impl()` — auto-generates WOODPECKER_TOKEN via OAuth2 flow. `_activate_woodpecker_repo_impl()` — activates repo in Woodpecker. All gated by `_load_ci_context()` which validates required env vars. | bin/disinto (init) |
|
||||
| `lib/ci-setup.sh` | `_install_cron_impl()` — installs crontab entries for bare-metal deployments (compose mode uses polling loop instead). `_create_woodpecker_oauth_impl()` — creates OAuth2 app on Forgejo for Woodpecker. `_generate_woodpecker_token_impl()` — auto-generates WOODPECKER_TOKEN via OAuth2 flow. `_activate_woodpecker_repo_impl()` — activates repo in Woodpecker. All gated by `_load_ci_context()` which validates required env vars. | bin/disinto (init) |
|
||||
| `lib/generators.sh` | Template generation for `disinto init`: `generate_compose()` — docker-compose.yml (uses `codeberg.org/forgejo/forgejo:11.0` tag; adds `security_opt: [apparmor:unconfined]` to all services for rootless container compatibility), `generate_caddyfile()` — Caddyfile, `generate_staging_index()` — staging index, `generate_deploy_pipelines()` — Woodpecker deployment pipeline configs. Requires `FACTORY_ROOT`, `PROJECT_NAME`, `PRIMARY_BRANCH`. | bin/disinto (init) |
|
||||
| `lib/hire-agent.sh` | `disinto_hire_an_agent()` — user creation, `.profile` repo setup, formula copying, branch protection, and state marker creation for hiring a new agent. Requires `FORGE_URL`, `FORGE_TOKEN`, `FACTORY_ROOT`, `PROJECT_NAME`. Extracted from `bin/disinto`. | bin/disinto (hire) |
|
||||
| `lib/release.sh` | `disinto_release()` — vault TOML creation, branch setup on ops repo, PR creation, and auto-merge request for a versioned release. `_assert_release_globals()` validates required env vars. Requires `FORGE_URL`, `FORGE_TOKEN`, `FORGE_OPS_REPO`, `FACTORY_ROOT`, `PRIMARY_BRANCH`. Extracted from `bin/disinto`. | bin/disinto (release) |
|
||||
|
|
|
|||
|
|
@ -1,9 +1,9 @@
|
|||
#!/usr/bin/env bash
|
||||
# =============================================================================
|
||||
# ci-setup.sh — CI setup functions for Woodpecker and cron configuration
|
||||
# ci-setup.sh — CI setup functions for Woodpecker and scheduling configuration
|
||||
#
|
||||
# Internal functions (called via _load_ci_context + _*_impl):
|
||||
# _install_cron_impl() - Install crontab entries for project agents
|
||||
# _install_cron_impl() - Install crontab entries (bare-metal only; compose uses polling loop)
|
||||
# _create_woodpecker_oauth_impl() - Create OAuth2 app on Forgejo for Woodpecker
|
||||
# _generate_woodpecker_token_impl() - Auto-generate WOODPECKER_TOKEN via OAuth2 flow
|
||||
# _activate_woodpecker_repo_impl() - Activate repo in Woodpecker
|
||||
|
|
@ -30,12 +30,13 @@ _load_ci_context() {
|
|||
fi
|
||||
}
|
||||
|
||||
# Generate and optionally install cron entries for the project agents.
|
||||
# Generate and optionally install cron entries for bare-metal deployments.
|
||||
# In compose mode, the agents container uses a polling loop (entrypoint.sh) instead.
|
||||
# Usage: install_cron <name> <toml_path> <auto_yes> <bare>
|
||||
_install_cron_impl() {
|
||||
local name="$1" toml="$2" auto_yes="$3" bare="${4:-false}"
|
||||
|
||||
# In compose mode, skip host cron — the agents container runs cron internally
|
||||
# In compose mode, skip host cron — the agents container uses a polling loop
|
||||
if [ "$bare" = false ]; then
|
||||
echo ""
|
||||
echo "Cron: skipped (agents container handles scheduling in compose mode)"
|
||||
|
|
|
|||
|
|
@ -1,11 +1,11 @@
|
|||
#!/usr/bin/env bash
|
||||
# formula-session.sh — Shared helpers for formula-driven cron agents
|
||||
# formula-session.sh — Shared helpers for formula-driven polling-loop agents
|
||||
#
|
||||
# Provides reusable utility functions for the common cron-wrapper pattern
|
||||
# Provides reusable utility functions for the common polling-loop wrapper pattern
|
||||
# used by planner-run.sh, predictor-run.sh, gardener-run.sh, and supervisor-run.sh.
|
||||
#
|
||||
# Functions:
|
||||
# acquire_cron_lock LOCK_FILE — PID lock with stale cleanup
|
||||
# acquire_run_lock LOCK_FILE — PID lock with stale cleanup
|
||||
# load_formula FORMULA_FILE — sets FORMULA_CONTENT
|
||||
# build_context_block FILE [FILE ...] — sets CONTEXT_BLOCK
|
||||
# build_prompt_footer [EXTRA_API_LINES] — sets PROMPT_FOOTER (API ref + env)
|
||||
|
|
@ -30,24 +30,24 @@
|
|||
#
|
||||
# Requires: lib/env.sh, lib/worktree.sh sourced first for shared helpers.
|
||||
|
||||
# ── Cron guards ──────────────────────────────────────────────────────────
|
||||
# ── Run guards ───────────────────────────────────────────────────────────
|
||||
|
||||
# acquire_cron_lock LOCK_FILE
|
||||
# acquire_run_lock LOCK_FILE
|
||||
# Acquires a PID lock. Exits 0 if another instance is running.
|
||||
# Sets an EXIT trap to clean up the lock file.
|
||||
acquire_cron_lock() {
|
||||
_CRON_LOCK_FILE="$1"
|
||||
if [ -f "$_CRON_LOCK_FILE" ]; then
|
||||
acquire_run_lock() {
|
||||
_RUN_LOCK_FILE="$1"
|
||||
if [ -f "$_RUN_LOCK_FILE" ]; then
|
||||
local lock_pid
|
||||
lock_pid=$(cat "$_CRON_LOCK_FILE" 2>/dev/null || true)
|
||||
lock_pid=$(cat "$_RUN_LOCK_FILE" 2>/dev/null || true)
|
||||
if [ -n "$lock_pid" ] && kill -0 "$lock_pid" 2>/dev/null; then
|
||||
log "run: already running (PID $lock_pid)"
|
||||
exit 0
|
||||
fi
|
||||
rm -f "$_CRON_LOCK_FILE"
|
||||
rm -f "$_RUN_LOCK_FILE"
|
||||
fi
|
||||
echo $$ > "$_CRON_LOCK_FILE"
|
||||
trap 'rm -f "$_CRON_LOCK_FILE"' EXIT
|
||||
echo $$ > "$_RUN_LOCK_FILE"
|
||||
trap 'rm -f "$_RUN_LOCK_FILE"' EXIT
|
||||
}
|
||||
|
||||
# ── Agent identity resolution ────────────────────────────────────────────
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
#!/usr/bin/env bash
|
||||
# guard.sh — Active-state guard for cron entry points
|
||||
# guard.sh — Active-state guard for polling-loop entry points
|
||||
#
|
||||
# Each agent checks for a state file before running. If the file
|
||||
# doesn't exist, the agent logs a skip and exits cleanly.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue