alexpires.me/a13labs.infra
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
Files
8724360fb249f683b4e46e7980355589e578a388
a13labs.infra/.opencode/plans/llamacpp-nginx-proxy.md
T
alexandre.pires 8724360fb2 Enhance infrastructure and application configurations 
- Updated AGENTS.md to include new directories for execution plans and implemented feature reports.
- Modified Ansible host variables for dev-02 and gpu-01 to change API endpoints and add new configurations for Scaleway.
- Adjusted firewall rules in gpu-01 to allow HTTPS traffic instead of the previous Ollama port.
- Added OAuth2 proxy configurations to gpu-01 for enhanced security.
- Removed deprecated open-webui module from Terraform app configurations.
- Updated Terraform configurations to reflect changes in local variables and removed unused secrets.
- Created a new sectool.json file for CloudNS integration with Bitwarden.
- Enhanced SELinux configurations for nginx to allow connections to any port and added oauth2-proxy port.
2026-07-12 16:28:45 -04:00

12 KiB
Raw Blame History

Plan: Replace OpenWebUI with llama.cpp behind nginx proxy

Current State

  • llama.cpp runs on gpu-01.lab.alexpires.me via Podman on port 11434 (bound to 127.0.0.1 with API key)
  • nginx proxy role runs on dev-02.lab.alexpires.me and gpu-01.lab.alexpires.me
  • OpenWebUI has been decommissioned (removed from K8s, DNS, inventory)
  • DNS for lab.alexpires.me is managed via Terraform (terraform/apps/dev-01/config.tf zones)
  • oauth2-proxy template uses OIDC provider (code.alexpires.me), set_xauthrequest = true

Goals

  • Deploy llama.cpp behind nginx proxy at llm.lab.alexpires.me ✅
  • API paths (/api/*) accessible without OAuth2 (token auth enforced via --api-key) ✅
  • All other paths (chat UI, etc.) protected with OAuth2 via oauth2-proxy ✅
  • Issue SSL certificate for llm.lab.alexpires.me via CloudNS DNS ✅
  • OpenWebUI decommissioned ✅

Important Constraint

Resolved. Phase 2 completed: llama.cpp now bound to 127.0.0.1 with API key. OpenWebUI decommissioned.

Changes Required

All changes have been implemented. See execution order below for completed steps.

1. Add DNS record for llm.lab.alexpires.me ✅

File: terraform/apps/dev-01/config.tf

Add A record to the lab.alexpires.me zone:

{
  name    = "llm"
  type    = "A"
  content = "10.19.4.106"
}

Run tofu plan/apply on terraform/apps/dev-01/ before Phase 1 so DNS resolves.

2. Add gpu-01 to [nginx] inventory group ✅

3. Update nginx-server.conf.j2 — per-site no_auth conditional ✅

4. Configure gpu-01 host vars for nginx proxy ✅

5. Add firewall rules for HTTPS ✅

6. Update llama_server role — support configurable bind address ✅

7. Two-phase transition ✅

Phase 1 — Deploy nginx proxy (llama.cpp still on 0.0.0.0, no API key)

  • Keep llama_server_bind: "0.0.0.0" (current behavior, default)
  • Do NOT set API key
  • nginx proxies to 127.0.0.1:11434 (container is bound to 0.0.0.0:11434, reachable from localhost)
  • OpenWebUI continues to work (still connects to gpu-01:11434 directly)
  • Test: llm.lab.alexpires.me/ requires OAuth2 login, llm.lab.alexpires.me/api/* is open

Phase 2 — Switch to localhost binding + enable API key (after validation)

  • Change llama_server_bind: "127.0.0.1" in host vars
  • Add --api-key to llama_server_argv_extra in host vars: 
    llama_server_argv_extra:
  - "--api-key"
  - "{{ lookup('env', 'LLAMA_SERVER_API_KEY') }}"
  # ... existing args ...
  • OpenWebUI will break (can no longer reach llama.cpp on gpu-01:11434) — decommission OpenWebUI
  • Verify all consumers migrate to llm.lab.alexpires.me/api/* with Authorization: Bearer <key>
  • Remove old firewall rules for port 11434 (no longer externally reachable)

Execution Order

Phase 1 — Deploy nginx proxy (llama.cpp unchanged, OpenWebUI still works)

Status: COMPLETE — All steps executed and verified.

  1. Update Terraform DNS — add llm A record in terraform/apps/dev-01/config.tf, run tofu apply ✅

  2. Update inventory — add gpu-01 to [nginx] group, clean up duplicate [certbot] group ✅

  3. Update nginx-server.conf.j2 — add no_auth per-site conditional (template stays generic) ✅

  4. Update llama_server role — add llama_server_bind variable (default 0.0.0.0) ✅

  5. Update host vars — add nginx sites (with extra_locations for /api/), OAuth2 creds, firewall rules ✅

  6. Run nginx proxy playbook — installs nginx + oauth2-proxy + certbot + deploys config ✅

  7. Fix SELinux — add nginx_connect_any boolean + port 4180 to http_port_t (manually applied, added to role for persistence) ✅

  8. Verify chat UI — llm.lab.alexpires.me/ returns oauth2-proxy login page (200) ✅

  9. Verify API — llm.lab.alexpires.me/api/v1/models returns 200 with models list ✅

  10. Verify OpenWebUI still works — llama.cpp still bound to 0.0.0.0:11434 ✅

  11. Update Terraform DNS — add llm A record in terraform/apps/dev-01/config.tf, run tofu apply

  12. Update inventory — add gpu-01 to [nginx] group, clean up duplicate [certbot] group

  13. Update nginx-server.conf.j2 — add no_auth per-site conditional (template stays generic)

  14. Update llama_server role — add llama_server_bind variable (default 0.0.0.0)

  15. Update host vars — add nginx sites (with extra_locations for /api/), OAuth2 creds, firewall rules

  16. Run nginx proxy playbook — installs nginx + oauth2-proxy + certbot + deploys config (all in one run)

  17. Verify chat UI — test llm.lab.alexpires.me/ with OAuth2 login

  18. Verify API — test llm.lab.alexpires.me/api/* (no auth yet, same as current direct access)

  19. Verify OpenWebUI still works — confirm dev-01 still connects to gpu-01:11434

Phase 2 — Switch to localhost + API key (after validation, decommission OpenWebUI)

Status: COMPLETE — All steps executed and verified.

  1. Update host vars — change llama_server_bind: "127.0.0.1", add --api-key to llama_server_argv_extra ✅
  2. Update firewall — remove port 11434 rules from fw_allowed_ports ✅
  3. Run llama_server playbook — restart container with new binding + API key ✅
  4. Run hardening playbook — playbook_hardening_cis.yml --limit gpu-01.lab.alexpires.me -t ufw to apply firewall changes ✅
  5. Verify API — llm.lab.alexpires.me/api/v1/chat/completions returns 401 without key, 200 with key ✅
  6. Decommission OpenWebUI — removed from terraform/apps/dev-01/apps.tf ✅
  7. Remove OpenWebUI DNS records — removed open-webui CNAME from terraform/apps/dev-01/config.tf zones ✅
  8. Remove OpenWebUI locals — removed open_webui_fqdn, open_webui_log_level, llm_proxy from config.tf ✅
  9. Remove webui_secret_key variable — removed from terraform/apps/dev-01/secrets.tf ✅
  10. Remove [openwebui] group — cleaned up inventory/hosts ✅
  11. Run tofu apply on terraform/apps/dev-01/ — destroyed 8 OpenWebUI K8s resources (namespace, deployment, service, ingress, secret, service accounts) ✅

Phase 2 — Switch to localhost + API key (after validation, decommission OpenWebUI)

  1. Update host vars — change llama_server_bind: "127.0.0.1", add --api-key to llama_server_argv_extra
  2. Update firewall — remove port 11434 rules from fw_allowed_ports
  3. Run llama_server playbook — restart container with new binding + API key
  4. Run hardening playbook — playbook_hardening_cis.yml --limit gpu-01.lab.alexpires.me -t ufw to apply firewall changes
  5. Verify API — test llm.lab.alexpires.me/api/* requires API key
  6. Decommission OpenWebUI — remove Terraform module from terraform/apps/dev-01/apps.tf and config.tf
  7. Remove OpenWebUI DNS records — clean up open-webui CNAME in terraform/apps/dev-01/config.tf zones
  8. Remove [openwebui] group — clean up inventory/hosts
  9. Run tofu apply on terraform/apps/dev-01/ — destroys OpenWebUI K8s resources

Playbooks to run

Phase 1

COMPLETE — All steps executed and verified.

cd ansible
sectool exec ansible-playbook -u $ANSIBLE_USER playbook_nginx_proxy.yml    # nginx + oauth2 + certbot (all-in-one)

Phase 2

COMPLETE — All steps executed and verified.

cd ansible
sectool exec ansible-playbook -u $ANSIBLE_USER playbook_llama_server.yml                # container binding + API key
sectool exec ansible-playbook -u $ANSIBLE_USER playbook_hardening_cis.yml -t ufw        # remove port 11434 rules

Then in terraform:

cd terraform/apps/dev-01
sectool exec tofu plan    # review OpenWebUI destruction
sectool exec tofu apply --auto-approve   # apply

Files to modify

File Phase Change
terraform/apps/dev-01/config.tf 1 Add llm A record to zones
inventory/hosts 1 Add gpu-01 to [nginx] group, clean up duplicate [certbot]
ansible/roles/nginx_proxy/templates/nginx-server.conf.j2 1 Add no_auth per-site conditional (3 blocks)
ansible/roles/llama_server/defaults/main.yml 1 Add llama_server_bind variable (default 0.0.0.0)
ansible/roles/llama_server/tasks/el/main.yml 1 Use llama_server_bind in podman port mapping
ansible/roles/nginx_proxy/tasks/nginx.yml 1 Add SELinux tasks (nginx_connect_any boolean + oauth2-proxy port to http_port_t)
ansible/host_vars/gpu-01.lab.alexpires.me.yml 1 Add nginx sites (with extra_locations for /api/), OAuth2 creds, firewall rules
ansible/host_vars/gpu-01.lab.alexpires.me.yml 2 Change llama_server_bind to 127.0.0.1, add --api-key to llama_server_argv_extra, remove port 11434 fw rules ✅
terraform/apps/dev-01/apps.tf 2 Remove module "open-webui" block ✅
terraform/apps/dev-01/config.tf 2 Remove open_webui_fqdn, open_webui_log_level, llm_proxy locals; remove open-webui DNS record from zones ✅
terraform/apps/dev-01/secrets.tf 2 Remove webui_secret_key variable ✅
inventory/hosts 2 Remove [openwebui] group ✅

Rollback plan

Phase 1 rollback (nginx proxy broken)

  • Remove gpu-01 from [nginx] group in inventory/hosts
  • Re-run hardening playbook on gpu-01 to clean up nginx/oauth2-proxy packages
  • Revert nginx-server.conf.j2 template changes
  • llama.cpp continues running on 0.0.0.0:11434 — no impact
  • OpenWebUI continues to work

Issues Encountered & Resolved

1. nginx copy module missing remote_src: true

File: ansible/roles/nginx_proxy/tasks/oauth2_proxy.yml:16 The copy module was trying to copy from the controller instead of the remote host. Added remote_src: true.

2. SELinux blocking nginx → oauth2-proxy / llama.cpp connections

Symptom: connect() to 127.0.0.1:4180 failed (13: Permission denied) in nginx error log Fix:

  • Set SELinux boolean nginx_connect_any to true (persistent)
  • Added port 4180 to http_port_t via semanage port
  • Added both as Ansible tasks in nginx.yml for persistence

3. Template no_auth attribute error

Symptom: object of type 'dict' has no attribute 'no_auth' Fix: Changed {% if not item.no_auth %} to {% if not (item.no_auth | default(false) | bool) %} to handle missing attribute on dicts.

4. Playbook failed on model query after container restart (Phase 2)

Symptom: urlopen error [Errno 111] Connection refused when querying http://127.0.0.1:11434/models?reload=1 Cause: The playbook's "Query llama router model list" task ran before the container was fully started. The container did start correctly with 127.0.0.1:11434 binding and API key active. Impact: Non-blocking — the container was running correctly; only the model list query task failed.

Known Warnings (non-blocking)

  • types_hash_bucket_size: 64 — nginx warning, non-blocking
  • oauth2-proxy PKCE warning (--code-challenge-method) — non-blocking, S256 is auto-negotiated

Rollback plan

Phase 1 rollback (nginx proxy broken)

  • Revert llama_server_bind to 0.0.0.0 in host vars
  • Remove --api-key from llama_server_argv_extra
  • Re-add port 11434 firewall rules
  • Re-run playbook_llama_server.yml + playbook_hardening_cis.yml -t ufw — container restarts with old settings
  • OpenWebUI still works (llama.cpp is back on 0.0.0.0 without API key)
  • Fix issues and re-attempt Phase 2

Phase 2 rollback (after OpenWebUI decommission)

  • Restore llama_server_bind: "0.0.0.0" in host vars
  • Remove --api-key from llama_server_argv_extra
  • Re-add port 11434 firewall rules
  • Restore module "open-webui" in terraform/apps/dev-01/apps.tf
  • Restore open_webui_fqdn, open_webui_log_level, llm_proxy locals in config.tf
  • Restore open-webui DNS CNAME record in config.tf zones
  • Restore webui_secret_key variable in secrets.tf
  • Re-add [openwebui] group in inventory/hosts
  • Run tofu apply to recreate OpenWebUI K8s resources
Reference in New Issue View Git Blame Copy Permalink