a13labs.infra/.opencode/plans/llama_server_template_command.md

# Plan: Refactor llama_server role — template-based container command

## Current State

The `llama_server` role deploys `llama.cpp` server via Podman (EL) or NSSB (Windows) in router mode.

**Existing structure:**

| Component | Location |
|---|---|
| `defaults/main.yml` | User/group, image, tag, port, bind, models list, `argv_extra`, exporter config, Windows config |
| `tasks/el/main.yml` | User/group creation → podman image pull → **container created inline** with hardcoded flags → systemd wrapper → HF model sync → exporter |
| `tasks/windows/main.yml` | Binary download → NSSM install with same hardcoded flags → HF sync → firewall → exporter |
| `templates/podman.service.j2` | Thin systemd wrapper: `podman start/stop llamacpp` |
| `templates/llama_exporter.service.j2` | Exporter systemd service |
| `vars/main.yml` | Internal non-overridable vars |
| `files/scripts/` | `llama_hf_sync.py` (Linux + Windows), `llama_exporter.py`, tests |

**Current container command** (`tasks/el/main.yml:142-152`):

```yaml
command: >-
  {{
    [
      '--metrics',
      '--models-preset',
      '/models/.router/models.ini',
      '--models-max',
      (llama_server_models_max | string)
    ]
    + (llama_server_argv_extra | default([]))
  }}
```

**Current Windows command** (`tasks/windows/main.yml:220-238`):

```yaml
argv: >-
  {{
    [
      'nssm.exe',
      'install',
      'llama-server',
      (llama_server_windows_install_dir ~ '\\' ~ llama_server_windows_bin_name),
      '--port',
      (llama_server_port | default(11434) | string),
      '--host',
      (llama_server_host | default('127.0.0.1') | string),
      '--metrics',
      '--models-preset',
      (__llama_preset_file | string),
      '--models-max',
      (llama_server_models_max | string)
    ]
    + (llama_server_argv_extra | default([]))
  }}
```

**Problem:** Both commands are hardcoded inline. The only extensibility is `llama_server_argv_extra` (append-only list), which has no validation, no conditional logic, and no self-documentation.

## Goals

- Replace hardcoded command construction with Jinja2 templates for both EL and Windows paths.
- Add role variables for all major llama.cpp server flags with sensible defaults and conditional rendering.
- Maintain backward compatibility: `llama_server_argv_extra` still works as a suffix.
- Enable future configuration of MCP servers, SSL/TLS, API keys, chat templates, and performance tuning without manual playbook edits.

## Changes Required

### 1. Add new role variables to `defaults/main.yml`

Add a new section after the existing defaults:

```yaml
# --- Server configuration ---
llama_server_bind: "0.0.0.0"
llama_server_host: "127.0.0.1"
llama_server_port: 11434

# SSL/TLS
llama_server_ssl_enabled: false
llama_server_ssl_cert: ""
llama_server_ssl_key: ""

# Authentication
llama_server_api_key: ""

# Chat
llama_server_chat_template: ""

# Performance
llama_server_ctx_size: 4096
llama_server_n_gpu_layers: 35
llama_server_batch_size: 512
llama_server_ubatch_size: 512
llama_server_threads: 0
llama_server_threads_batch: 0

# Features
llama_server_embeddings: false
llama_server_cors: false
llama_server_log_timestamps: false
llama_server_log_level: "warn"
llama_server_log_print: false
llama_server_slot_save_context: false

# Web UI
llama_server_webui: ""
llama_server_webui_author: ""

# MCP
llama_server_mcp_servers: []

# KV cache quantization
llama_server_cache_type_q4: false

# Speculative decoding
llama_server_speculative_model: ""
llama_server_draft: 0

# RoPE scaling
llama_server_rope_freq_scale: 0.0

# GPU offload
llama_server_tensor_split: 0
llama_server_split_mode: "layer"
llama_server_main_gpu: 0
```

### 2. Create EL container command template

**File:** `templates/llama_server_command_el.j2`

```jinja2
{{
  [
    '--metrics'
  ]
  + (llama_server_models_preset_global_path | default('/models/.router/models.ini') | ternary(['--models-preset', llama_server_models_preset_global_path], []))
  + (llama_server_models_max | default(1) | string | ternary(['--models-max', llama_server_models_max | string], []))
  + (llama_server_host | default('127.0.0.1') | ternary(['--host', llama_server_host], []))
  + (llama_server_ssl_enabled | default(false) | bool | ternary(
      ['--ssl', '--ssl-cert', llama_server_ssl_cert, '--ssl-key', llama_server_ssl_key] if llama_server_ssl_cert and llama_server_ssl_key else [], []))
  + (llama_server_api_key | default('') | ternary(['--api-key', llama_server_api_key], []))
  + (llama_server_chat_template | default('') | ternary(['--chat-template', llama_server_chat_template], []))
  + (llama_server_ctx_size | default(4096) | ternary(['--ctx-size', llama_server_ctx_size | string], []))
  + (llama_server_n_gpu_layers | default(35) | ternary(['--n-gpu-layers', llama_server_n_gpu_layers | string], []))
  + (llama_server_batch_size | default(512) | ternary(['--batch-size', llama_server_batch_size | string], []))
  + (llama_server_ubatch_size | default(512) | ternary(['--ubatch-size', llama_server_ubatch_size | string], []))
  + (llama_server_threads | default(0) | ternary(['--threads', llama_server_threads | string], []))
  + (llama_server_threads_batch | default(0) | ternary(['--threads-batch', llama_server_threads_batch | string], []))
  + (llama_server_embeddings | default(false) | bool | ternary(['--embedding'], []))
  + (llama_server_cors | default(false) | bool | ternary(['--cors'], []))
  + (llama_server_log_timestamps | default(false) | bool | ternary(['--log-timestamps'], []))
  + (llama_server_log_level | default('warn') | ternary(['--log-level', llama_server_log_level], []))
  + (llama_server_log_print | default(false) | bool | ternary(['--log-print'], []))
  + (llama_server_slot_save_context | default(false) | bool | ternary(['--slot-save-context'], []))
  + (llama_server_webui | default('') | ternary(['--webui-name', llama_server_webui], []))
  + (llama_server_webui_author | default('') | ternary(['--webui-author', llama_server_webui_author], []))
  + (llama_server_mcp_servers | default([]) | length | ternary(['--mcp-servers', (llama_server_mcp_servers | to_json)], []))
  + (llama_server_cache_type_q4 | default(false) | bool | ternary(['--cache-type-q4'], []))
  + (llama_server_speculative_model | default('') | ternary(['--speculative-model', llama_server_speculative_model], []))
  + (llama_server_draft | default(0) | ternary(['--draft', llama_server_draft | string], []))
  + (llama_server_rope_freq_scale | default(0.0) | ternary(['--rope-freq-scale', llama_server_rope_freq_scale | string], []))
  + (llama_server_tensor_split | default(0) | ternary(['--tensor-split', llama_server_tensor_split | string], []))
  + (llama_server_split_mode | default('layer') | ternary(['--split-mode', llama_server_split_mode], []))
  + (llama_server_main_gpu | default(0) | ternary(['--main-gpu', llama_server_main_gpu | string], []))
  + (llama_server_argv_extra | default([]))
}}
```

**Note:** The above uses a Jinja2 expression that produces a JSON array string. The `ternary()` calls handle conditional inclusion. The `llama_server_mcp_servers` serializes to JSON for the `--mcp-servers` flag.

### 3. Create Windows service command template

**File:** `templates/llama_server_command_windows.j2`

Same logic as EL but output as a flat list suitable for NSSM `nssm.exe install` argv (each flag and value as separate list elements).

### 4. Refactor `tasks/el/main.yml`

Replace the hardcoded command block (lines 142-152) in the `podman_container` task:

```yaml
- name: Set fact llama_server_el_command
  ansible.builtin.set_fact:
    llama_server_el_command: "{{ lookup('template', 'templates/llama_server_command_el.j2') }}"

- name: Create pods
  containers.podman.podman_container:
    name: "llamacpp"
    image: "{{ llama_server_image }}:{{ llama_server_tag }}"
    state: started
    device: "{{ llama_server_devices | default(omit) }}"
    env: "{{ llama_server_env | default(omit) }}"
    ports: "{{ llama_server_bind }}:{{ llama_server_port }}:8080/tcp"
    volumes:
      - "{{ llama_server_home }}/models:/models:Z"
      - "{{ llama_server_home }}/.cache:/app/.cache:Z"
    command: "{{ llama_server_el_command }}"
```

### 5. Refactor `tasks/windows/main.yml`

Replace the hardcoded `nssm.exe install` argv block (lines 220-238):

```yaml
- name: Set fact llama_server_windows_argv
  ansible.builtin.set_fact:
    llama_server_windows_argv: "{{ lookup('template', 'templates/llama_server_command_windows.j2') }}"

- name: Install llama-server service via NSSM
  ansible.windows.win_command:
    argv: "{{ llama_server_windows_argv }}"
  args:
    chdir: "{{ llama_server_windows_install_dir }}"
  register: __llama_nssm_install
  changed_when: __llama_nssm_install.rc == 0
```

### 6. Create MCP servers preset file template (optional, for future)

**File:** `templates/mcp_servers.json.j2`

```json
{{ llama_server_mcp_servers | to_nice_json }}
```

Used when MCP servers need to be written to a file and passed via `--mcp-servers` flag pointing to a path.

### 7. Add tasks to write SSL/TLS and API key files (conditional)

When `llama_server_ssl_cert` or `llama_server_api_key` are set, ensure the referenced files exist on the remote host. This may require:
- A task to create the API key file if provided as a variable (not file path)
- A task to validate SSL cert/key paths exist

## Files to modify

| File | Change |
|------|--------|
| `ansible/roles/llama_server/defaults/main.yml` | Add new role variables section |
| `ansible/roles/llama_server/templates/llama_server_command_el.j2` | **New** — EL container command template |
| `ansible/roles/llama_server/templates/llama_server_command_windows.j2` | **New** — Windows NSSM command template |
| `ansible/roles/llama_server/templates/mcp_servers.json.j2` | **New** — MCP servers JSON preset (optional) |
| `ansible/roles/llama_server/tasks/el/main.yml` | Replace hardcoded command with `lookup('template')` + `set_fact` |
| `ansible/roles/llama_server/tasks/windows/main.yml` | Replace hardcoded nssm argv with `lookup('template')` + `set_fact` |
| `ansible/roles/llama_server/tasks/el/llama_exporter.yml` | No change |
| `ansible/roles/llama_server/files/scripts/` | No change |

## Rollback plan

- The `llama_server_argv_extra` variable continues to work as a suffix in both templates.
- If template rendering fails, revert `tasks/el/main.yml` and `tasks/windows/main.yml` to the hardcoded command blocks.
- Delete the new template files.
- No data loss — templates only affect command construction, not state.

## Testing

1. **Baseline test:** Run role with existing defaults — should produce identical command to current hardcoded version.
2. **Variable test:** Set individual variables (e.g., `llama_server_ctx_size: 8192`, `llama_server_embeddings: true`) and verify they appear in the rendered command.
3. **Conditional test:** Set `llama_server_ssl_enabled: true` with cert/key paths — verify `--ssl`, `--ssl-cert`, `--ssl-key` flags are included.
4. **MCP test:** Set `llama_server_mcp_servers` to a list of dicts — verify JSON serialization in `--mcp-servers` flag.
5. **Backward compat test:** Use `llama_server_argv_extra` alongside new variables — verify both are present.
6. **Windows test:** Same scenarios on Windows target with NSSM.

## Open questions

1. **Should `--models-preset` and `--models-max` remain in the template or stay hardcoded?** They're core to router mode and unlikely to change. Keeping them in the template makes the command self-contained but adds complexity.
2. **For `mcp_servers`** — does llama.cpp accept the flag as `--mcp-servers '{"name":"..."}'` (JSON string) or `--mcp-servers /path/to/config.json` (file path)? This affects whether we need a separate file template.
3. **Should defaults be opinionated** (e.g., `llama_server_ctx_size: 4096`) or minimal (only required flags)? Opinionated defaults reduce host var boilerplate but may not suit all use cases.
4. **Should we add assertions** for mutually exclusive flags (e.g., `--split-mode` values)?