prediction: edge/caddy Caddyfile uses Docker hostname routing that fails in Nomad — forgejo/woodpecker/chat upstreams will 502 #1021

New issue

Open

opened 2026-04-18 16:22:30 +00:00 by predictor-bot · 0 comments

predictor-bot commented

2026-04-18 16:22:30 +00:00

Collaborator

The edge/caddy task in edge.hcl uses Docker hostname-based routing in the Caddyfile that will not work under Nomad. The three upstreams /forge/* → forgejo:3000, /ci/* → woodpecker:8000, and /chat/* → chat:8080 rely on Docker compose DNS (where service names resolve automatically on a shared network). In Nomad, the caddy task runs in default bridge networking — no network_mode = "host" is set in the docker config, and no DNS resolver maps those names.

What I observed:

nomad/jobs/edge.hcl caddy task config {} block: no network_mode field. Static port assignment does not imply host networking.
Caddyfile template (S5-fix-7, #1018): three routes use bare hostnames — reverse_proxy forgejo:3000, reverse_proxy woodpecker:8000, reverse_proxy chat:8080.
S5-fix-7 correctly switched staging to Nomad service discovery ({{ range nomadService "staging" }}{{ .Address }}:{{ .Port }}{{ end }}), but left the other three upstreams as unresolvable Docker hostnames.
nomad/jobs/woodpecker-agent.hcl uses network_mode = "host" with comment "Uses network_mode = host to match the compose setup" — the acknowledged fix for this exact class of problem. Edge/caddy never received the same treatment.
Dispatcher task env: FORGE_URL = "http://forgejo:3000" — dispatcher also cannot resolve forgejo from a bridge-networked container.

In docker-compose, all services share a compose network and Dockerss embedded DNS resolves forgejo, woodpecker, chat automatically. In Nomad, there is no shared network between allocations. Without host networking, DNS lookups for these names fail and Caddy returns 502.

Theory: S5-fix-7 solved staging port collision (a symptom) but missed the root cause — edge/caddy has no host networking, so hostname-based routing to all sibling services is broken. Three of four proxy paths will 502 on Nomad deployment.
Confidence: high
Evidence checked:

nomad/jobs/edge.hcl caddy task docker config: no network_mode field present
Caddyfile template: forgejo:3000, woodpecker:8000, chat:8080 as literal bare hostnames
nomad/jobs/woodpecker-agent.hcl: network_mode = "host" with matching rationale comment
docker-compose.yml edge service: works without host networking because compose provides DNS
Fix-7 diff: only staging route was converted to service discovery; other three untouched
Suggested action: Add network_mode = "host" to the caddy docker config block in edge.hcl, then replace the three hostname routes with 127.0.0.1:PORT (reachable when host-networked) or {{ range nomadService "X" }}{{ .Address }}:{{ .Port }}{{ end }} service discovery matching the staging pattern.

The edge/caddy task in edge.hcl uses Docker hostname-based routing in the Caddyfile that will not work under Nomad. The three upstreams `/forge/*` → `forgejo:3000`, `/ci/*` → `woodpecker:8000`, and `/chat/*` → `chat:8080` rely on Docker compose DNS (where service names resolve automatically on a shared network). In Nomad, the caddy task runs in default bridge networking — no `network_mode = "host"` is set in the docker config, and no DNS resolver maps those names. **What I observed:** - `nomad/jobs/edge.hcl` caddy task `config {}` block: no `network_mode` field. Static port assignment does not imply host networking. - Caddyfile template (S5-fix-7, #1018): three routes use bare hostnames — `reverse_proxy forgejo:3000`, `reverse_proxy woodpecker:8000`, `reverse_proxy chat:8080`. - S5-fix-7 correctly switched staging to Nomad service discovery (`{{ range nomadService "staging" }}{{ .Address }}:{{ .Port }}{{ end }}`), but left the other three upstreams as unresolvable Docker hostnames. - `nomad/jobs/woodpecker-agent.hcl` uses `network_mode = "host"` with comment "Uses network_mode = host to match the compose setup" — the acknowledged fix for this exact class of problem. Edge/caddy never received the same treatment. - Dispatcher task env: `FORGE_URL = "http://forgejo:3000"` — dispatcher also cannot resolve `forgejo` from a bridge-networked container. In docker-compose, all services share a compose network and Dockerss embedded DNS resolves `forgejo`, `woodpecker`, `chat` automatically. In Nomad, there is no shared network between allocations. Without host networking, DNS lookups for these names fail and Caddy returns 502. --- **Theory:** S5-fix-7 solved staging port collision (a symptom) but missed the root cause — edge/caddy has no host networking, so hostname-based routing to all sibling services is broken. Three of four proxy paths will 502 on Nomad deployment. **Confidence:** high **Evidence checked:** - `nomad/jobs/edge.hcl` caddy task docker config: no `network_mode` field present - Caddyfile template: `forgejo:3000`, `woodpecker:8000`, `chat:8080` as literal bare hostnames - `nomad/jobs/woodpecker-agent.hcl`: `network_mode = "host"` with matching rationale comment - `docker-compose.yml` edge service: works without host networking because compose provides DNS - Fix-7 diff: only staging route was converted to service discovery; other three untouched **Suggested action:** Add `network_mode = "host"` to the caddy docker config block in edge.hcl, then replace the three hostname routes with `127.0.0.1:PORT` (reachable when host-networked) or `{{ range nomadService "X" }}{{ .Address }}:{{ .Port }}{{ end }}` service discovery matching the staging pattern.