alexpires.me/a13labs.infra
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Enhance infrastructure and application configurations

Browse Source 
- Updated AGENTS.md to include new directories for execution plans and implemented feature reports.
- Modified Ansible host variables for dev-02 and gpu-01 to change API endpoints and add new configurations for Scaleway.
- Adjusted firewall rules in gpu-01 to allow HTTPS traffic instead of the previous Ollama port.
- Added OAuth2 proxy configurations to gpu-01 for enhanced security.
- Removed deprecated open-webui module from Terraform app configurations.
- Updated Terraform configurations to reflect changes in local variables and removed unused secrets.
- Created a new sectool.json file for CloudNS integration with Bitwarden.
- Enhanced SELinux configurations for nginx to allow connections to any port and added oauth2-proxy port.
This commit is contained in:
Alexandre Pires
2026-07-12 16:28:45 -04:00
parent a4a8554fb6
commit 8724360fb2
23 changed files with 2106 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.opencode/plans/llama_server_template_command.md
+271
View File
@@ -0,0 +1,271 @@
# Plan: Refactor llama_server role — template-based container command
## Current State
The `llama_server` role deploys `llama.cpp` server via Podman (EL) or NSSB (Windows) in router mode.
**Existing structure:**
| Component | Location |
|---|---|
| `defaults/main.yml` | User/group, image, tag, port, bind, models list, `argv_extra`, exporter config, Windows config |
| `tasks/el/main.yml` | User/group creation → podman image pull → **container created inline** with hardcoded flags → systemd wrapper → HF model sync → exporter |
| `tasks/windows/main.yml` | Binary download → NSSM install with same hardcoded flags → HF sync → firewall → exporter |
| `templates/podman.service.j2` | Thin systemd wrapper: `podman start/stop llamacpp` |
| `templates/llama_exporter.service.j2` | Exporter systemd service |
| `vars/main.yml` | Internal non-overridable vars |
| `files/scripts/` | `llama_hf_sync.py` (Linux + Windows), `llama_exporter.py`, tests |
**Current container command** (`tasks/el/main.yml:142-152`):
```yaml
command: >-
{{
[
'--metrics',
'--models-preset',
'/models/.router/models.ini',
'--models-max',
(llama_server_models_max | string)
]
+ (llama_server_argv_extra | default([]))
}}
```
**Current Windows command** (`tasks/windows/main.yml:220-238`):
```yaml
argv: >-
{{
[
'nssm.exe',
'install',
'llama-server',
(llama_server_windows_install_dir ~ '\\' ~ llama_server_windows_bin_name),
'--port',
(llama_server_port | default(11434) | string),
'--host',
(llama_server_host | default('127.0.0.1') | string),
'--metrics',
'--models-preset',
(__llama_preset_file | string),
'--models-max',
(llama_server_models_max | string)
]
+ (llama_server_argv_extra | default([]))
}}
```
**Problem:** Both commands are hardcoded inline. The only extensibility is `llama_server_argv_extra` (append-only list), which has no validation, no conditional logic, and no self-documentation.
## Goals
- Replace hardcoded command construction with Jinja2 templates for both EL and Windows paths.
- Add role variables for all major llama.cpp server flags with sensible defaults and conditional rendering.
- Maintain backward compatibility: `llama_server_argv_extra` still works as a suffix.
- Enable future configuration of MCP servers, SSL/TLS, API keys, chat templates, and performance tuning without manual playbook edits.
## Changes Required
### 1. Add new role variables to `defaults/main.yml`
Add a new section after the existing defaults:
```yaml
# --- Server configuration ---
llama_server_bind: "0.0.0.0"
llama_server_host: "127.0.0.1"
llama_server_port: 11434
# SSL/TLS
llama_server_ssl_enabled: false
llama_server_ssl_cert: ""
llama_server_ssl_key: ""
# Authentication
llama_server_api_key: ""
# Chat
llama_server_chat_template: ""
# Performance
llama_server_ctx_size: 4096
llama_server_n_gpu_layers: 35
llama_server_batch_size: 512
llama_server_ubatch_size: 512
llama_server_threads: 0
llama_server_threads_batch: 0
# Features
llama_server_embeddings: false
llama_server_cors: false
llama_server_log_timestamps: false
llama_server_log_level: "warn"
llama_server_log_print: false
llama_server_slot_save_context: false
# Web UI
llama_server_webui: ""
llama_server_webui_author: ""
# MCP
llama_server_mcp_servers: []
# KV cache quantization
llama_server_cache_type_q4: false
# Speculative decoding
llama_server_speculative_model: ""
llama_server_draft: 0
# RoPE scaling
llama_server_rope_freq_scale: 0.0
# GPU offload
llama_server_tensor_split: 0
llama_server_split_mode: "layer"
llama_server_main_gpu: 0
```
### 2. Create EL container command template
**File:** `templates/llama_server_command_el.j2`
```jinja2
{{
[
'--metrics'
]
+ (llama_server_models_preset_global_path | default('/models/.router/models.ini') | ternary(['--models-preset', llama_server_models_preset_global_path], []))
+ (llama_server_models_max | default(1) | string | ternary(['--models-max', llama_server_models_max | string], []))
+ (llama_server_host | default('127.0.0.1') | ternary(['--host', llama_server_host], []))
+ (llama_server_ssl_enabled | default(false) | bool | ternary(
['--ssl', '--ssl-cert', llama_server_ssl_cert, '--ssl-key', llama_server_ssl_key] if llama_server_ssl_cert and llama_server_ssl_key else [], []))
+ (llama_server_api_key | default('') | ternary(['--api-key', llama_server_api_key], []))
+ (llama_server_chat_template | default('') | ternary(['--chat-template', llama_server_chat_template], []))
+ (llama_server_ctx_size | default(4096) | ternary(['--ctx-size', llama_server_ctx_size | string], []))
+ (llama_server_n_gpu_layers | default(35) | ternary(['--n-gpu-layers', llama_server_n_gpu_layers | string], []))
+ (llama_server_batch_size | default(512) | ternary(['--batch-size', llama_server_batch_size | string], []))
+ (llama_server_ubatch_size | default(512) | ternary(['--ubatch-size', llama_server_ubatch_size | string], []))
+ (llama_server_threads | default(0) | ternary(['--threads', llama_server_threads | string], []))
+ (llama_server_threads_batch | default(0) | ternary(['--threads-batch', llama_server_threads_batch | string], []))
+ (llama_server_embeddings | default(false) | bool | ternary(['--embedding'], []))
+ (llama_server_cors | default(false) | bool | ternary(['--cors'], []))
+ (llama_server_log_timestamps | default(false) | bool | ternary(['--log-timestamps'], []))
+ (llama_server_log_level | default('warn') | ternary(['--log-level', llama_server_log_level], []))
+ (llama_server_log_print | default(false) | bool | ternary(['--log-print'], []))
+ (llama_server_slot_save_context | default(false) | bool | ternary(['--slot-save-context'], []))
+ (llama_server_webui | default('') | ternary(['--webui-name', llama_server_webui], []))
+ (llama_server_webui_author | default('') | ternary(['--webui-author', llama_server_webui_author], []))
+ (llama_server_mcp_servers | default([]) | length | ternary(['--mcp-servers', (llama_server_mcp_servers | to_json)], []))
+ (llama_server_cache_type_q4 | default(false) | bool | ternary(['--cache-type-q4'], []))
+ (llama_server_speculative_model | default('') | ternary(['--speculative-model', llama_server_speculative_model], []))
+ (llama_server_draft | default(0) | ternary(['--draft', llama_server_draft | string], []))
+ (llama_server_rope_freq_scale | default(0.0) | ternary(['--rope-freq-scale', llama_server_rope_freq_scale | string], []))
+ (llama_server_tensor_split | default(0) | ternary(['--tensor-split', llama_server_tensor_split | string], []))
+ (llama_server_split_mode | default('layer') | ternary(['--split-mode', llama_server_split_mode], []))
+ (llama_server_main_gpu | default(0) | ternary(['--main-gpu', llama_server_main_gpu | string], []))
+ (llama_server_argv_extra | default([]))
}}
```
**Note:** The above uses a Jinja2 expression that produces a JSON array string. The `ternary()` calls handle conditional inclusion. The `llama_server_mcp_servers` serializes to JSON for the `--mcp-servers` flag.
### 3. Create Windows service command template
**File:** `templates/llama_server_command_windows.j2`
Same logic as EL but output as a flat list suitable for NSSM `nssm.exe install` argv (each flag and value as separate list elements).
### 4. Refactor `tasks/el/main.yml`
Replace the hardcoded command block (lines 142-152) in the `podman_container` task:
```yaml
- name: Set fact llama_server_el_command
ansible.builtin.set_fact:
llama_server_el_command: "{{ lookup('template', 'templates/llama_server_command_el.j2') }}"
- name: Create pods
containers.podman.podman_container:
name: "llamacpp"
image: "{{ llama_server_image }}:{{ llama_server_tag }}"
state: started
device: "{{ llama_server_devices | default(omit) }}"
env: "{{ llama_server_env | default(omit) }}"
ports: "{{ llama_server_bind }}:{{ llama_server_port }}:8080/tcp"
volumes:
- "{{ llama_server_home }}/models:/models:Z"
- "{{ llama_server_home }}/.cache:/app/.cache:Z"
command: "{{ llama_server_el_command }}"
```
### 5. Refactor `tasks/windows/main.yml`
Replace the hardcoded `nssm.exe install` argv block (lines 220-238):
```yaml
- name: Set fact llama_server_windows_argv
ansible.builtin.set_fact:
llama_server_windows_argv: "{{ lookup('template', 'templates/llama_server_command_windows.j2') }}"
- name: Install llama-server service via NSSM
ansible.windows.win_command:
argv: "{{ llama_server_windows_argv }}"
args:
chdir: "{{ llama_server_windows_install_dir }}"
register: __llama_nssm_install
changed_when: __llama_nssm_install.rc == 0
```
### 6. Create MCP servers preset file template (optional, for future)
**File:** `templates/mcp_servers.json.j2`
```json
{{ llama_server_mcp_servers | to_nice_json }}
```
Used when MCP servers need to be written to a file and passed via `--mcp-servers` flag pointing to a path.
### 7. Add tasks to write SSL/TLS and API key files (conditional)
When `llama_server_ssl_cert` or `llama_server_api_key` are set, ensure the referenced files exist on the remote host. This may require:
- A task to create the API key file if provided as a variable (not file path)
- A task to validate SSL cert/key paths exist
## Files to modify
| File | Change |
|------|--------|
| `ansible/roles/llama_server/defaults/main.yml` | Add new role variables section |
| `ansible/roles/llama_server/templates/llama_server_command_el.j2` | **New** — EL container command template |
| `ansible/roles/llama_server/templates/llama_server_command_windows.j2` | **New** — Windows NSSM command template |
| `ansible/roles/llama_server/templates/mcp_servers.json.j2` | **New** — MCP servers JSON preset (optional) |
| `ansible/roles/llama_server/tasks/el/main.yml` | Replace hardcoded command with `lookup('template')` + `set_fact` |
| `ansible/roles/llama_server/tasks/windows/main.yml` | Replace hardcoded nssm argv with `lookup('template')` + `set_fact` |
| `ansible/roles/llama_server/tasks/el/llama_exporter.yml` | No change |
| `ansible/roles/llama_server/files/scripts/` | No change |
## Rollback plan
- The `llama_server_argv_extra` variable continues to work as a suffix in both templates.
- If template rendering fails, revert `tasks/el/main.yml` and `tasks/windows/main.yml` to the hardcoded command blocks.
- Delete the new template files.
- No data loss — templates only affect command construction, not state.
## Testing
1. **Baseline test:** Run role with existing defaults — should produce identical command to current hardcoded version.
2. **Variable test:** Set individual variables (e.g., `llama_server_ctx_size: 8192`, `llama_server_embeddings: true`) and verify they appear in the rendered command.
3. **Conditional test:** Set `llama_server_ssl_enabled: true` with cert/key paths — verify `--ssl`, `--ssl-cert`, `--ssl-key` flags are included.
4. **MCP test:** Set `llama_server_mcp_servers` to a list of dicts — verify JSON serialization in `--mcp-servers` flag.
5. **Backward compat test:** Use `llama_server_argv_extra` alongside new variables — verify both are present.
6. **Windows test:** Same scenarios on Windows target with NSSM.
## Open questions
1. **Should `--models-preset` and `--models-max` remain in the template or stay hardcoded?** They're core to router mode and unlikely to change. Keeping them in the template makes the command self-contained but adds complexity.
2. **For `mcp_servers`** — does llama.cpp accept the flag as `--mcp-servers '{"name":"..."}'` (JSON string) or `--mcp-servers /path/to/config.json` (file path)? This affects whether we need a separate file template.
3. **Should defaults be opinionated** (e.g., `llama_server_ctx_size: 4096`) or minimal (only required flags)? Opinionated defaults reduce host var boilerplate but may not suit all use cases.
4. **Should we add assertions** for mutually exclusive flags (e.g., `--split-mode` values)?