diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 0000000..206828f --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,130 @@ +# Copilot Instructions for a13labs.infra + +This repository manages infrastructure mostly with Ansible and OpenTofu/Terraform. +These instructions define how Terraform app modules should be developed, deployed, and debugged. + +## Scope and Structure + +- App roots: + - terraform/apps/dev-01 + - terraform/apps/prod-01 +- Reusable modules: + - terraform/modules/apps + - terraform/modules/db + - terraform/modules/utils +- New app modules should be created under terraform/modules/apps/. +- Use terraform/modules/apps/template as the base shape when creating a new app module. + +## Cluster References + +Always cover both clusters in planning and reviews: + +- dev-01 + - App root: terraform/apps/dev-01 + - Kubernetes context: microk8s-dev-01 +- pro-01 + - App root: terraform/apps/prod-01 + - Kubernetes context: microk8s-prod-01 + +Note: The production folder name is prod-01, while discussions may refer to it as pro-01. + +## Module Development Standards + +When adding a new app module in terraform/modules/apps/: + +1. Create the standard files: + - provider.tf + - variables.tf + - config.tf + - main.tf + - ingress.tf (if exposed over HTTP/HTTPS) +2. Provider constraints should match repository conventions: + - required_version ~>1.8 + - kubernetes provider version 2.36.0 +3. Keep names consistent across namespace, service account, deployment labels, and service selectors. +4. Use locals in config.tf for derived paths and normalized values. +5. Treat secrets as Terraform inputs (sensitive variables), not hardcoded literals. +6. Prefer explicit image tags instead of latest for predictable rollouts. + +## Security and Hardening Defaults + +For Kubernetes deployments, prefer hardened defaults unless an app requires exceptions: + +- Disable service account token automount at both SA and pod level. +- Use pod security context with non-root UID/GID and RuntimeDefault seccomp. +- Use container security context: + - allow_privilege_escalation = false + - privileged = false + - drop all capabilities + - read_only_root_filesystem = true +- Provide writable mounts only where required (for example cache path and /tmp via emptyDir). + +## Integrating a Module into an App Root + +For each cluster root (dev-01 and pro-01): + +1. Register module in apps.tf. +2. Add needed locals in config.tf (FQDN, path, issuer, feature toggles). +3. Add DNS record entries when the app is externally reachable. +4. Add sensitive input variables in secrets.tf. +5. Map TF_VAR entries in sectool.env. + +## Secrets Pattern (sectool + TF_VAR) + +Secret flow is: + +1. Declare sensitive variables in secrets.tf. +2. Map TF_VAR_ in terraform/apps//sectool.env. +3. Keep value source in vault/environment, not in repository. +4. Run OpenTofu through sectool so variables are injected. + +Example workflow from an app root: + +- sectool -f ../../../sectool.json exec tofu init +- sectool -f ../../../sectool.json exec tofu validate +- sectool -f ../../../sectool.json exec tofu plan +- sectool -f ../../../sectool.json exec tofu apply -auto-approve + +## OpenTofu and Deployment Workflow + +Preferred command flow per cluster root: + +1. Format and validate first: + - tofu fmt + - sectool -f ../../../sectool.json exec tofu validate +2. Plan before apply: + - sectool -f ../../../sectool.json exec tofu plan +3. Apply: + - sectool -f ../../../sectool.json exec tofu apply -auto-approve + +Targeted apply can be used for emergency recovery, then follow with a full plan to detect drift. + +## Runtime Debugging Playbook (kubectl) + +When a deployment fails (CrashLoopBackOff): + +1. Inspect pod state and events: + - kubectl -n get pods -o wide + - kubectl -n describe pod +2. Check logs: + - kubectl -n logs --all-containers=true --tail=200 + - kubectl -n logs --previous --all-containers=true --tail=200 +3. Validate rollout: + - kubectl -n rollout status deployment/ --timeout=120s +4. If needed, test temporary env override with kubectl set env, verify recovery, then persist in Terraform module. + +## Known SearXNG Behavior in This Repo + +- SearXNG may fail on startup with IPv6 bind on hosts without IPv6 socket support. +- Set GRANIAN_HOST to 0.0.0.0 in module-managed configuration. +- Keep image tag pinned in app roots to avoid unexpected upstream behavior changes. + +## Review Checklist for Copilot Changes + +Before proposing Terraform changes: + +1. Confirm changes cover both dev-01 and pro-01 impact. +2. Confirm secrets are wired via sensitive variables + sectool mapping. +3. Confirm image tags are pinned. +4. Confirm deployment hardening defaults are retained. +5. Confirm commands use sectool + tofu in examples and runbooks. diff --git a/.opencode/skills/terraform-develop-app/SKILL.md b/.opencode/skills/terraform-develop-app/SKILL.md new file mode 100644 index 0000000..c13b6c8 --- /dev/null +++ b/.opencode/skills/terraform-develop-app/SKILL.md @@ -0,0 +1,301 @@ +--- +name: terraform-develop-app +description: > + Use when developing, deploying, or debugging Terraform app modules in this + repository. Covers module scaffolding, secrets injection with sectool and + TF_VAR, OpenTofu command workflow, Kubernetes runtime debugging with kubectl, + hardening defaults for pod and container security contexts, and known quirks + such as the SearXNG IPv6 Granian bind issue. Also covers integrating modules + into dev-01 and prod-01 app roots including DNS records, sectool.env mapping, + and targeted apply recovery patterns. +--- + +# Terraform App Module Skill + +## Purpose + +Use this skill when developing, deploying, or debugging Terraform app modules in this repository. It covers module authoring conventions, secrets injection with sectool, OpenTofu command workflow, Kubernetes runtime debugging, and known behavioral quirks discovered in production. + +--- + +## Repository Layout + +``` +terraform/ + apps/ + dev-01/ # dev cluster root + prod-01/ # prod cluster root + modules/ + apps/ # reusable app modules + db/ # database modules + utils/ # utility modules +``` + +App roots are the deployment entry points. Reusable logic lives in modules and is consumed by app roots. + +Use `terraform/modules/apps/template` as the baseline shape when creating a new module. + +--- + +## Cluster References + +| Name | Folder | Kubernetes context | +|--------|--------------------------|----------------------| +| dev-01 | terraform/apps/dev-01 | microk8s-dev-01 | +| pro-01 | terraform/apps/prod-01 | microk8s-prod-01 | + +When planning or reviewing changes, always consider both clusters. The production folder is named `prod-01`; discussions may call it `pro-01`. + +--- + +## Creating a New App Module + +Create `terraform/modules/apps//` with these files: + +### provider.tf + +```hcl +terraform { + required_version = "~>1.8" + required_providers { + kubernetes = { + source = "hashicorp/kubernetes" + version = "2.36.0" + } + } +} +``` + +### variables.tf + +Declare at minimum: + +```hcl +variable "persistent_folder" { + description = "Path for persistent data on the host" + type = string +} + +variable "fqdn" { + description = "FQDN for the service" + type = string +} + +variable "tag" { + description = "Image tag to deploy" + type = string + default = "x.y.z" # always pin, never use latest +} +``` + +Add sensitive variables at module level only when the module itself needs to inject them into a Kubernetes secret. If the secret is passed from the app root, declare it sensitive there. + +### config.tf + +Use locals to normalize inputs. Do not append the app name to `persistent_folder` if the caller already passes an app-specific path: + +```hcl +locals { + app_folder = var.persistent_folder + app_version = var.tag + app_fqdn = var.fqdn +} +``` + +### main.tf resource order + +1. `kubernetes_namespace` +2. `kubernetes_service_account` — with `automount_service_account_token = false` +3. `kubernetes_config_map` (non-sensitive config) +4. `kubernetes_secret` (sensitive data) +5. `kubernetes_deployment` +6. `kubernetes_service` + +### ingress.tf (when HTTP/HTTPS exposed) + +```hcl +resource "kubernetes_ingress_v1" "" { + metadata { + annotations = { + "kubernetes.io/ingress.class" = "public" + "cert-manager.io/cluster-issuer" = var.issuer + "kubernetes.io/tls-acme" = "true" + "nginx.ingress.kubernetes.io/ssl-redirect" = "true" + } + } + ... +} +``` + +Accept `issuer` as a variable (default `"letsencrypt-prod"`) so the same module works for both clusters and internal CA scenarios. + +--- + +## Hardening Defaults + +Apply these to every new deployment unless there is a documented exception: + +```hcl +spec { + automount_service_account_token = false + + security_context { + run_as_non_root = true + run_as_user = + run_as_group = + fs_group = + seccomp_profile { + type = "RuntimeDefault" + } + } + + container { + security_context { + allow_privilege_escalation = false + privileged = false + read_only_root_filesystem = true + capabilities { + drop = ["ALL"] + } + } + + # provide writable scratch via emptyDir, not root filesystem + volume_mount { + name = "-tmp" + mount_path = "/tmp" + } + } + + volume { + name = "-tmp" + empty_dir {} + } +} +``` + +--- + +## Integrating a Module into an App Root + +Do this for each cluster root: + +1. Add a `module` block in `apps.tf`: + ```hcl + module "myapp" { + source = "../../modules/apps/myapp" + persistent_folder = local.myapp_folder + fqdn = local.myapp_fqdn + tag = "x.y.z" + secret_key = var.myapp_secret_key + } + ``` + +2. Add locals in `config.tf`: + ```hcl + myapp_folder = "/mnt/md0/myapp" + myapp_fqdn = "myapp.${local.lab_domain}" + ``` + +3. Add a DNS CNAME record inside `local.zones[0].records`: + ```hcl + { name = "myapp", type = "CNAME", content = "dev-01.${local.lab_domain}." } + ``` + +4. Add sensitive variable in `secrets.tf`: + ```hcl + variable "myapp_secret_key" { + description = "Secret key for MyApp" + type = string + sensitive = true + } + ``` + +5. Map the env variable in `sectool.env`: + ``` + TF_VAR_myapp_secret_key=$MYAPP_SECRET_KEY + ``` + +--- + +## Secrets Pattern (sectool + TF_VAR) + +Secrets are sourced from Bitwarden via `sectool` using `sectool.json` at the repository root. + +Flow: +1. Store secret value in Bitwarden under the project referenced in `sectool.json`. +2. Declare the variable in `secrets.tf` with `sensitive = true`. +3. Map `TF_VAR_=$ENV_VAR_NAME` in `terraform/apps//sectool.env`. +4. Run all OpenTofu commands via `sectool exec` so variables are injected automatically. + +--- + +## OpenTofu Workflow + +Run from the relevant app root (e.g., `terraform/apps/dev-01`): + +```bash +# Initialise +sectool -f ../../../sectool.json exec tofu init + +# Format and validate +tofu fmt +sectool -f ../../../sectool.json exec tofu validate + +# Plan +sectool -f ../../../sectool.json exec tofu plan + +# Apply +sectool -f ../../../sectool.json exec tofu apply -auto-approve + +# Targeted apply (emergency recovery only — always follow with a full plan) +sectool -f ../../../sectool.json exec tofu apply -auto-approve -target=module. +``` + +--- + +## Runtime Debugging Playbook + +When a pod is in CrashLoopBackOff: + +```bash +# 1. Inspect state and events +kubectl -n get pods -o wide +kubectl -n describe pod + +# 2. Check current and previous logs +kubectl -n logs --all-containers=true --tail=200 +kubectl -n logs --previous --all-containers=true --tail=200 + +# 3. Validate rollout after a fix +kubectl -n rollout status deployment/ --timeout=120s + +# 4. Test a runtime env override before persisting in Terraform +kubectl -n set env deployment/ KEY=value +# verify recovery, then add KEY to the module ConfigMap and re-apply +``` + +--- + +## Known Quirks + +### SearXNG — IPv6 bind failure + +The `searxng/searxng` image defaults to `GRANIAN_HOST=::` (IPv6). On nodes without IPv6 socket support this causes an immediate crash: + +``` +RuntimeError: Address family not supported by protocol (os error 97) +``` + +Fix: set `GRANIAN_HOST = "0.0.0.0"` in the module ConfigMap. + +--- + +## Review Checklist + +Before proposing or applying any Terraform change: + +- [ ] Changes scoped to both dev-01 and prod-01 where relevant. +- [ ] Sensitive inputs declared in `secrets.tf` + mapped in `sectool.env`. +- [ ] Image tag is pinned (not `latest`). +- [ ] Hardening defaults present (non-root, drop-all capabilities, read-only root fs). +- [ ] Commands in examples use `sectool -f ../../../sectool.json exec tofu`. diff --git a/.opencode/skills/terraform-update-app-version/SKILL.md b/.opencode/skills/terraform-update-app-version/SKILL.md new file mode 100644 index 0000000..65ed6db --- /dev/null +++ b/.opencode/skills/terraform-update-app-version/SKILL.md @@ -0,0 +1,115 @@ +--- +name: terraform-update-app-version +description: > + Use when updating a container image version for an app managed by Terraform in + this repository. Covers how to find where the tag is set, update dev-01 and + prod-01 app roots safely, run sectool-wrapped OpenTofu validation and plan, + apply changes, and verify rollout with kubectl. +--- + +# Terraform Update App Version Skill + +## Purpose + +Use this skill when you need to update the deployed container image version for an app module. + +This repository usually sets the image tag in the app root module call (`terraform/apps//apps.tf`) and passes it to the module variable `tag`, then to `local.app_version` inside the module. + +Always use pinned tags. Do not use `latest`. + +--- + +## Scope + +Update both clusters unless the request explicitly says otherwise: + +- dev-01: `terraform/apps/dev-01` +- pro-01 folder: `terraform/apps/prod-01` + +Note: production is often called "pro-01" in discussion, but the folder name is `prod-01`. + +--- + +## Where to Change the Version + +Common location: + +- `terraform/apps//apps.tf` + +Pattern: + +```hcl +module "" { + source = "../../modules/apps/" + ... + tag = "" +} +``` + +Module wiring pattern: + +- `terraform/modules/apps//variables.tf` defines `variable "tag"` +- `terraform/modules/apps//config.tf` typically sets `local.app_version = var.tag` +- `terraform/modules/apps//main.tf` uses `${local.app_version}` in image reference + +If tag is not in the app root, check the module defaults and root overrides before editing. + +--- + +## Update Workflow + +Run from each app root directory (`terraform/apps/dev-01` and `terraform/apps/prod-01`) as needed. + +1. Edit `apps.tf` and set the new pinned tag. +2. Validate configuration: + +```bash +tofu fmt +sectool -f ../../../sectool.json exec tofu validate +``` + +3. Review plan: + +```bash +sectool -f ../../../sectool.json exec tofu plan +``` + +4. Apply: + +```bash +sectool -f ../../../sectool.json exec tofu apply -auto-approve +``` + +5. Verify rollout: + +```bash +kubectl --context microk8s- -n rollout status deployment/ --timeout=120s +kubectl --context microk8s- -n get pods -o wide +kubectl --context microk8s- -n logs deployment/ --all-containers=true --tail=200 +``` + +--- + +## Emergency Recovery + +If immediate recovery is required, a targeted apply can be used temporarily: + +```bash +sectool -f ../../../sectool.json exec tofu apply -auto-approve -target=module. +``` + +Afterward, always run a full plan to detect drift: + +```bash +sectool -f ../../../sectool.json exec tofu plan +``` + +--- + +## Review Checklist + +- [ ] New image tag is pinned and explicit. +- [ ] Both cluster roots were considered (dev-01 and prod-01). +- [ ] Commands were run with `sectool -f ../../../sectool.json exec tofu`. +- [ ] Rollout and pod health were verified with kubectl. +- [ ] Any runtime hotfix used during debugging was persisted back into Terraform. diff --git a/ansible/files/dockerfiles/Dockerfile.llamacpp.rocm b/ansible/files/dockerfiles/Dockerfile.llamacpp.rocm new file mode 100644 index 0000000..55939ab --- /dev/null +++ b/ansible/files/dockerfiles/Dockerfile.llamacpp.rocm @@ -0,0 +1,14 @@ +FROM ghcr.io/ggml-org/llama.cpp:server-rocm + +ARG UID=1000 +ARG GID=1000 +RUN groupadd --system --gid ${GID} worker && \ + useradd --system --gid ${GID} --uid ${UID} --home /home/worker worker && \ + mkdir -p /home/worker && chown worker:worker /home/worker && \ + cp /app/llama-server /home/worker/ && \ + chmod +x /home/worker/llama-server && \ + chown worker:worker /home/worker/llama-server + +WORKDIR /home/worker +USER worker +ENTRYPOINT ["/home/worker/llama-server"] \ No newline at end of file diff --git a/ansible/files/dockerfiles/Dockerfile.ollama b/ansible/files/dockerfiles/Dockerfile.ollama index c306c4e..6b8707e 100644 --- a/ansible/files/dockerfiles/Dockerfile.ollama +++ b/ansible/files/dockerfiles/Dockerfile.ollama @@ -1,4 +1,4 @@ -FROM docker.io/ollama/ollama:0.13.4 +FROM docker.io/ollama/ollama:latest ARG UID=1000 ARG GID=1000 diff --git a/ansible/files/dockerfiles/Dockerfile.ollama.rocm b/ansible/files/dockerfiles/Dockerfile.ollama.rocm index 5473dad..0640da6 100644 --- a/ansible/files/dockerfiles/Dockerfile.ollama.rocm +++ b/ansible/files/dockerfiles/Dockerfile.ollama.rocm @@ -1,4 +1,4 @@ -FROM docker.io/ollama/ollama:0.13.4-rocm +FROM docker.io/ollama/ollama:rocm ARG UID=1000 ARG GID=1000 diff --git a/ansible/host_vars/gpu-01.lab.alexpires.me.yml b/ansible/host_vars/gpu-01.lab.alexpires.me.yml index bbc8d30..d94e843 100644 --- a/ansible/host_vars/gpu-01.lab.alexpires.me.yml +++ b/ansible/host_vars/gpu-01.lab.alexpires.me.yml @@ -88,6 +88,7 @@ duo_api_hostname: api-7cda1654.duosecurity.com # podman services settings ollama_home: "/mnt/data/ollama" +llama_home: "/mnt/data/llamacpp" podman_user_pubkey: "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHHOPBR9p9kq5Cqzpe4cr3jHnweaYrHPXv5sXNzt+sCXP54uc5rWUBhxW2OQVvQzJ47dEVhEKi4WSC7LcuKS2G5AQDzWXNiasHvYIYQU3F/EknVCZnsiXYqXphYkJA6rJCNRnISZCIC1mocq6PB7J08ONdRFCvjfUBuVRT8BNGXNmQ/zQ==" # ComfyUI settings diff --git a/ansible/playbook_podman_llama_rocm.yml b/ansible/playbook_podman_llama_rocm.yml new file mode 100644 index 0000000..575786a --- /dev/null +++ b/ansible/playbook_podman_llama_rocm.yml @@ -0,0 +1,30 @@ +- name: Setup llama.cpp - ROCm + hosts: llamacpp + vars: + podman_user: "{{ llama_user | default('llama') }}" + podman_group: "{{ llama_group | default('llama') }}" + podman_extra_groups: "users,video" + podman_user_home: "{{ llama_home | default('/home/llama') }}" + podman_user_folders: + - data + pods: + - name: llamacpp-rocm + build: + dockerfile: "{{ lookup('file', 'dockerfiles/Dockerfile.llamacpp.rocm') }}" + env: + LLAMA_ARG_MODELS_DIR: "/home/worker/.llamacpp" + LLAMA_ARG_THREADS: 6 + LLAMA_ARG_PARALLEL: 1 + LLAMA_ARG_CACHE: "true" + ports: + - "0.0.0.0:{{ llama_port | default(11435) }}:11434/tcp" + volumes: + - "{{ llama_home }}/data:/home/worker/.llamacpp:Z" + device: + - "/dev/kfd:/dev/kfd:rw" + - "/dev/dri:/dev/dri:rw" + + tasks: + - name: Setup Pods + ansible.builtin.include_tasks: + file: tasks/podman.yml diff --git a/ansible/playbook_podman_ollama.yml b/ansible/playbook_podman_ollama.yml index 1be5010..f1f5637 100644 --- a/ansible/playbook_podman_ollama.yml +++ b/ansible/playbook_podman_ollama.yml @@ -16,13 +16,17 @@ OLLAMA_NUM_THREADS: 6 OLLAMA_NUM_PARALLEL: 1 OLLAMA_KV_CACHE: "true" + OLLAMA_VULKAN: 1 + GGML_VK_VISIBLE_DEVICES: 0,1,2 + OLLAMA_NUM_GPU: 99 ports: - "0.0.0.0:{{ ollama_port | default(11434) }}:11434/tcp" volumes: - "{{ ollama_home }}/data:/home/worker/.ollama:Z" device: - "nvidia.com/gpu=all" - + - "/dev/kfd:/dev/kfd:rw" + - "/dev/dri:/dev/dri:rw" tasks: - name: Setup Pods ansible.builtin.include_tasks: diff --git a/ansible/playbook_podman_ollama_rocm.yml b/ansible/playbook_podman_ollama_rocm.yml index 21f002b..a4cce6a 100644 --- a/ansible/playbook_podman_ollama_rocm.yml +++ b/ansible/playbook_podman_ollama_rocm.yml @@ -12,10 +12,12 @@ build: dockerfile: "{{ lookup('file', 'dockerfiles/Dockerfile.ollama.rocm') }}" env: - OLLAMA_CONTEXT_SIZE: 65536 - OLLAMA_NUM_THREADS: 6 - OLLAMA_NUM_PARALLEL: 1 + OLLAMA_CONTEXT_LENGTH: 4096 + OLLAMA_NUM_THREADS: 10 + OLLAMA_NUM_PARALLEL: 2 + OLLAMA_FLASH_ATTENTION: 1 OLLAMA_KV_CACHE: "true" + HSA_OVERRIDE_GFX_VERSION: "11.0.0" ports: - "0.0.0.0:{{ ollama_port | default(11434) }}:11434/tcp" volumes: diff --git a/inventory/hosts b/inventory/hosts index d48fdcd..00bfcae 100644 --- a/inventory/hosts +++ b/inventory/hosts @@ -119,3 +119,6 @@ hpz440.lab.alexpires.me [game_station] steambox.local + +[llamacpp] +gpu-01.lab.alexpires.me diff --git a/terraform/apps/dev-01/apps.tf b/terraform/apps/dev-01/apps.tf index d4c6da7..a6c4915 100644 --- a/terraform/apps/dev-01/apps.tf +++ b/terraform/apps/dev-01/apps.tf @@ -5,7 +5,7 @@ module "open-webui" { fqdn = local.open_webui_fqdn ollama_proxy = local.ollama_proxy secret_key = var.webui_secret_key - tag = "0.6-slim" + tag = "v0.9.2-slim" log_level = local.open_webui_log_level } @@ -82,3 +82,13 @@ module "mstream" { music_folder = local.mstream_music_folder fqdn = local.mstream_fqdn } + +module "searxng" { + source = "../../modules/apps/searxng" + + persistent_folder = local.searxng_folder + fqdn = local.searxng_fqdn + issuer = local.searxng_issuer + tag = "2026.5.2-aefc3c316" + secret_key = var.searxng_secret_key +} diff --git a/terraform/apps/dev-01/config.tf b/terraform/apps/dev-01/config.tf index febf817..8b6f58c 100644 --- a/terraform/apps/dev-01/config.tf +++ b/terraform/apps/dev-01/config.tf @@ -111,6 +111,11 @@ locals { name = "music" type = "CNAME" content = "dev-01.${local.lab_domain}." + }, + { + name = "search" + type = "CNAME" + content = "dev-01.${local.lab_domain}." } ] } @@ -234,6 +239,11 @@ locals { # mstream mstream_music_folder = "/mnt/storage/Music" mstream_fqdn = "music.${local.lab_domain}" + + # searxng + searxng_folder = "/mnt/md0/searxng" + searxng_fqdn = "search.${local.lab_domain}" + searxng_issuer = "letsencrypt-prod" } data "terraform_remote_state" "scaleway" { diff --git a/terraform/apps/dev-01/provider.tf b/terraform/apps/dev-01/provider.tf index a9a1a17..a1e1624 100644 --- a/terraform/apps/dev-01/provider.tf +++ b/terraform/apps/dev-01/provider.tf @@ -16,5 +16,8 @@ provider "kubernetes" { config_path = "~/.kube/config" config_context = "microk8s-dev-01" insecure = true + ignore_annotations = [ + "kubectl.kubernetes.io.*", + ] } diff --git a/terraform/apps/dev-01/secrets.tf b/terraform/apps/dev-01/secrets.tf index b99d048..eea8a33 100644 --- a/terraform/apps/dev-01/secrets.tf +++ b/terraform/apps/dev-01/secrets.tf @@ -27,3 +27,9 @@ variable "webui_secret_key" { type = string sensitive = true } + +variable "searxng_secret_key" { + description = "The secret key for SearXNG" + type = string + sensitive = true +} diff --git a/terraform/apps/dev-01/sectool.env b/terraform/apps/dev-01/sectool.env index b82f712..6c088bb 100644 --- a/terraform/apps/dev-01/sectool.env +++ b/terraform/apps/dev-01/sectool.env @@ -6,3 +6,4 @@ TF_VAR_nextcloud_admin_password=$NEXTCLOUD_ADMIN_PASSWORD TF_VAR_scaleway_project_id=$SCALEWAY_PROJECT_ID TF_VAR_telegram_bot_token=$TELEGRAM_BOT_TOKEN TF_VAR_webui_secret_key=$WEBUI_SECRET_KEY +TF_VAR_searxng_secret_key=$SEARXNG_SECRET_KEY diff --git a/terraform/apps/prod-01/apps.tf b/terraform/apps/prod-01/apps.tf index 4603cd7..8f2be08 100644 --- a/terraform/apps/prod-01/apps.tf +++ b/terraform/apps/prod-01/apps.tf @@ -82,7 +82,7 @@ module "gitea" { mail_from = "gitea@${local.primary_domain}" no_reply_address = "no-reply@${local.primary_domain}" smtp_user = var.scaleway_project_id - db_password = var.gitea_mysql_password + db_passwd = var.gitea_mysql_password smtp_host = local.scaleway_smtp_host smtp_port = local.scaleway_smtp_port smtp_user = var.scaleway_project_id diff --git a/terraform/apps/prod-01/resources/alexpires.me.conf b/terraform/apps/prod-01/resources/alexpires.me.conf index 9cce716..57e9e62 100644 --- a/terraform/apps/prod-01/resources/alexpires.me.conf +++ b/terraform/apps/prod-01/resources/alexpires.me.conf @@ -1,7 +1,7 @@ resolver kube-dns.kube-system.svc.cluster.local valid=10s; location ^~ /api/ssh_locker/ { - rewrite ^/api/ssh_locker/(.*)$ /$1 break; + rewrite ^/api/ssh_locker/(?.*)$ /$path break; proxy_pass https://backend.ssh-locker-web.svc.cluster.local; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; diff --git a/terraform/modules/apps/open-webui/variables.tf b/terraform/modules/apps/open-webui/variables.tf index 3ab7e92..1f1daae 100644 --- a/terraform/modules/apps/open-webui/variables.tf +++ b/terraform/modules/apps/open-webui/variables.tf @@ -11,7 +11,7 @@ variable "fqdn" { variable "tag" { description = "The version of app to install" type = string - default = "main" + default = "v0.8.12" } variable "issuer" { @@ -32,7 +32,6 @@ variable "ollama_proxy" { default = { enabled = false } - } variable "secret_key" { diff --git a/terraform/modules/apps/searxng/config.tf b/terraform/modules/apps/searxng/config.tf new file mode 100644 index 0000000..5719da8 --- /dev/null +++ b/terraform/modules/apps/searxng/config.tf @@ -0,0 +1,6 @@ +locals { + app_folder = var.persistent_folder + + app_version = var.tag + app_fqdn = var.fqdn +} diff --git a/terraform/modules/apps/searxng/ingress.tf b/terraform/modules/apps/searxng/ingress.tf new file mode 100644 index 0000000..1687cf7 --- /dev/null +++ b/terraform/modules/apps/searxng/ingress.tf @@ -0,0 +1,39 @@ +resource "kubernetes_ingress_v1" "searxng" { + metadata { + name = "ingress" + namespace = kubernetes_namespace.searxng.metadata[0].name + + annotations = { + "kubernetes.io/ingress.class" = "public" + "cert-manager.io/cluster-issuer" = var.issuer + "kubernetes.io/tls-acme" = "true" + "nginx.ingress.kubernetes.io/ssl-redirect" = "true" + } + } + + spec { + dynamic "tls" { + for_each = var.tls_enabled ? [1] : [] + content { + hosts = [var.fqdn] + secret_name = "searxng-tls" + } + } + + rule { + host = var.fqdn + http { + path { + backend { + service { + name = kubernetes_service.searxng.metadata[0].name + port { + number = 8080 + } + } + } + } + } + } + } +} diff --git a/terraform/modules/apps/searxng/main.tf b/terraform/modules/apps/searxng/main.tf new file mode 100644 index 0000000..1998fad --- /dev/null +++ b/terraform/modules/apps/searxng/main.tf @@ -0,0 +1,191 @@ +resource "kubernetes_namespace" "searxng" { + metadata { + name = "searxng" + } +} + +resource "kubernetes_service_account" "searxng" { + + metadata { + name = "searxng-sa" + namespace = kubernetes_namespace.searxng.metadata[0].name + } + + automount_service_account_token = false +} + +resource "kubernetes_config_map" "searxng_config" { + metadata { + name = "searxng-config" + namespace = kubernetes_namespace.searxng.metadata[0].name + } + + data = { + BASE_URL = "https://${local.app_fqdn}/" + GRANIAN_HOST = "0.0.0.0" + } +} + +resource "kubernetes_secret" "searxng_secrets" { + metadata { + name = "searxng-keys" + namespace = kubernetes_namespace.searxng.metadata[0].name + } + + data = { + SEARXNG_SECRET = var.secret_key + } +} + +resource "kubernetes_deployment" "searxng" { + metadata { + name = "searxng" + namespace = kubernetes_namespace.searxng.metadata[0].name + labels = { + app = "searxng" + } + } + + spec { + replicas = 1 + + selector { + match_labels = { + app = "searxng" + } + } + + strategy { + type = "Recreate" + } + + template { + metadata { + labels = { + app = "searxng" + } + } + + spec { + service_account_name = kubernetes_service_account.searxng.metadata[0].name + automount_service_account_token = false + + security_context { + run_as_non_root = true + run_as_user = 977 + run_as_group = 977 + fs_group = 977 + + seccomp_profile { + type = "RuntimeDefault" + } + } + + container { + name = "searxng" + image = "searxng/searxng:${local.app_version}" + + env_from { + config_map_ref { + name = kubernetes_config_map.searxng_config.metadata[0].name + } + } + + env { + name = "SEARXNG_SECRET" + value_from { + secret_key_ref { + name = kubernetes_secret.searxng_secrets.metadata[0].name + key = "SEARXNG_SECRET" + } + } + } + + security_context { + allow_privilege_escalation = false + privileged = false + + capabilities { + drop = ["ALL"] + } + + read_only_root_filesystem = true + } + + port { + name = "http" + container_port = 8080 + protocol = "TCP" + } + + liveness_probe { + http_get { + path = "/" + port = "http" + } + initial_delay_seconds = 30 + period_seconds = 15 + } + + readiness_probe { + http_get { + path = "/" + port = "http" + } + initial_delay_seconds = 10 + period_seconds = 10 + } + + volume_mount { + name = "searxng-data" + mount_path = "/var/cache/searxng" + } + + volume_mount { + name = "searxng-tmp" + mount_path = "/tmp" + } + } + + volume { + name = "searxng-data" + host_path { + path = local.app_folder + } + } + + volume { + name = "searxng-tmp" + empty_dir {} + } + } + } + } +} + +resource "kubernetes_service" "searxng" { + metadata { + name = "searxng-service" + namespace = kubernetes_namespace.searxng.metadata[0].name + } + + spec { + selector = { + app = "searxng" + } + + port { + name = "http" + port = 8080 + target_port = 8080 + } + + type = "ClusterIP" + } + + lifecycle { + ignore_changes = [ + metadata[0].annotations + ] + } +} diff --git a/terraform/modules/apps/searxng/provider.tf b/terraform/modules/apps/searxng/provider.tf new file mode 100644 index 0000000..2eaa508 --- /dev/null +++ b/terraform/modules/apps/searxng/provider.tf @@ -0,0 +1,9 @@ +terraform { + required_version = "~>1.8" + required_providers { + kubernetes = { + source = "hashicorp/kubernetes" + version = "2.36.0" + } + } +} diff --git a/terraform/modules/apps/searxng/variables.tf b/terraform/modules/apps/searxng/variables.tf new file mode 100644 index 0000000..46c0863 --- /dev/null +++ b/terraform/modules/apps/searxng/variables.tf @@ -0,0 +1,33 @@ +variable "persistent_folder" { + description = "The path to the persistent folder" + type = string +} + +variable "fqdn" { + description = "FQDN name of the service" + type = string +} + +variable "tag" { + description = "The version of searxng to install" + type = string + default = "latest" +} + +variable "issuer" { + description = "The cert-manager cluster issuer" + type = string + default = "letsencrypt-prod" +} + +variable "tls_enabled" { + description = "Enable TLS in ingress" + type = bool + default = true +} + +variable "secret_key" { + description = "Secret key for SearXNG" + type = string + sensitive = true +}