a13labs.infra/.opencode/plans/llama_server_template_command.md

Plan: Refactor llama_server role — template-based container command

Current State

The llama_server role deploys llama.cpp server via Podman (EL) or NSSB (Windows) in router mode.

Existing structure:

Component	Location
defaults/main.yml	User/group, image, tag, port, bind, models list, argv_extra, exporter config, Windows config
tasks/el/main.yml	User/group creation → podman image pull → container created inline with hardcoded flags → systemd wrapper → HF model sync → exporter
tasks/windows/main.yml	Binary download → NSSM install with same hardcoded flags → HF sync → firewall → exporter
templates/podman.service.j2	Thin systemd wrapper: podman start/stop llamacpp
templates/llama_exporter.service.j2	Exporter systemd service
vars/main.yml	Internal non-overridable vars
files/scripts/	llama_hf_sync.py (Linux + Windows), llama_exporter.py, tests

Current container command (tasks/el/main.yml:142-152):

command: >-
  {{
    [
      '--metrics',
      '--models-preset',
      '/models/.router/models.ini',
      '--models-max',
      (llama_server_models_max | string)
    ]
    + (llama_server_argv_extra | default([]))
  }}

Current Windows command (tasks/windows/main.yml:220-238):

argv: >-
  {{
    [
      'nssm.exe',
      'install',
      'llama-server',
      (llama_server_windows_install_dir ~ '\\' ~ llama_server_windows_bin_name),
      '--port',
      (llama_server_port | default(11434) | string),
      '--host',
      (llama_server_host | default('127.0.0.1') | string),
      '--metrics',
      '--models-preset',
      (__llama_preset_file | string),
      '--models-max',
      (llama_server_models_max | string)
    ]
    + (llama_server_argv_extra | default([]))
  }}

Problem: Both commands are hardcoded inline. The only extensibility is llama_server_argv_extra (append-only list), which has no validation, no conditional logic, and no self-documentation.

Goals

• Replace hardcoded command construction with Jinja2 templates for both EL and Windows paths.
• Add role variables for all major llama.cpp server flags with sensible defaults and conditional rendering.
• Maintain backward compatibility: llama_server_argv_extra still works as a suffix.
• Enable future configuration of MCP servers, SSL/TLS, API keys, chat templates, and performance tuning without manual playbook edits.

Changes Required

1. Add new role variables to defaults/main.yml

Add a new section after the existing defaults:

# --- Server configuration ---
llama_server_bind: "0.0.0.0"
llama_server_host: "127.0.0.1"
llama_server_port: 11434

# SSL/TLS
llama_server_ssl_enabled: false
llama_server_ssl_cert: ""
llama_server_ssl_key: ""

# Authentication
llama_server_api_key: ""

# Chat
llama_server_chat_template: ""

# Performance
llama_server_ctx_size: 4096
llama_server_n_gpu_layers: 35
llama_server_batch_size: 512
llama_server_ubatch_size: 512
llama_server_threads: 0
llama_server_threads_batch: 0

# Features
llama_server_embeddings: false
llama_server_cors: false
llama_server_log_timestamps: false
llama_server_log_level: "warn"
llama_server_log_print: false
llama_server_slot_save_context: false

# Web UI
llama_server_webui: ""
llama_server_webui_author: ""

# MCP
llama_server_mcp_servers: []

# KV cache quantization
llama_server_cache_type_q4: false

# Speculative decoding
llama_server_speculative_model: ""
llama_server_draft: 0

# RoPE scaling
llama_server_rope_freq_scale: 0.0

# GPU offload
llama_server_tensor_split: 0
llama_server_split_mode: "layer"
llama_server_main_gpu: 0

2. Create EL container command template

File: templates/llama_server_command_el.j2

{{
  [
    '--metrics'
  ]
  + (llama_server_models_preset_global_path | default('/models/.router/models.ini') | ternary(['--models-preset', llama_server_models_preset_global_path], []))
  + (llama_server_models_max | default(1) | string | ternary(['--models-max', llama_server_models_max | string], []))
  + (llama_server_host | default('127.0.0.1') | ternary(['--host', llama_server_host], []))
  + (llama_server_ssl_enabled | default(false) | bool | ternary(
      ['--ssl', '--ssl-cert', llama_server_ssl_cert, '--ssl-key', llama_server_ssl_key] if llama_server_ssl_cert and llama_server_ssl_key else [], []))
  + (llama_server_api_key | default('') | ternary(['--api-key', llama_server_api_key], []))
  + (llama_server_chat_template | default('') | ternary(['--chat-template', llama_server_chat_template], []))
  + (llama_server_ctx_size | default(4096) | ternary(['--ctx-size', llama_server_ctx_size | string], []))
  + (llama_server_n_gpu_layers | default(35) | ternary(['--n-gpu-layers', llama_server_n_gpu_layers | string], []))
  + (llama_server_batch_size | default(512) | ternary(['--batch-size', llama_server_batch_size | string], []))
  + (llama_server_ubatch_size | default(512) | ternary(['--ubatch-size', llama_server_ubatch_size | string], []))
  + (llama_server_threads | default(0) | ternary(['--threads', llama_server_threads | string], []))
  + (llama_server_threads_batch | default(0) | ternary(['--threads-batch', llama_server_threads_batch | string], []))
  + (llama_server_embeddings | default(false) | bool | ternary(['--embedding'], []))
  + (llama_server_cors | default(false) | bool | ternary(['--cors'], []))
  + (llama_server_log_timestamps | default(false) | bool | ternary(['--log-timestamps'], []))
  + (llama_server_log_level | default('warn') | ternary(['--log-level', llama_server_log_level], []))
  + (llama_server_log_print | default(false) | bool | ternary(['--log-print'], []))
  + (llama_server_slot_save_context | default(false) | bool | ternary(['--slot-save-context'], []))
  + (llama_server_webui | default('') | ternary(['--webui-name', llama_server_webui], []))
  + (llama_server_webui_author | default('') | ternary(['--webui-author', llama_server_webui_author], []))
  + (llama_server_mcp_servers | default([]) | length | ternary(['--mcp-servers', (llama_server_mcp_servers | to_json)], []))
  + (llama_server_cache_type_q4 | default(false) | bool | ternary(['--cache-type-q4'], []))
  + (llama_server_speculative_model | default('') | ternary(['--speculative-model', llama_server_speculative_model], []))
  + (llama_server_draft | default(0) | ternary(['--draft', llama_server_draft | string], []))
  + (llama_server_rope_freq_scale | default(0.0) | ternary(['--rope-freq-scale', llama_server_rope_freq_scale | string], []))
  + (llama_server_tensor_split | default(0) | ternary(['--tensor-split', llama_server_tensor_split | string], []))
  + (llama_server_split_mode | default('layer') | ternary(['--split-mode', llama_server_split_mode], []))
  + (llama_server_main_gpu | default(0) | ternary(['--main-gpu', llama_server_main_gpu | string], []))
  + (llama_server_argv_extra | default([]))
}}

Note: The above uses a Jinja2 expression that produces a JSON array string. The ternary() calls handle conditional inclusion. The llama_server_mcp_servers serializes to JSON for the --mcp-servers flag.

3. Create Windows service command template

File: templates/llama_server_command_windows.j2

Same logic as EL but output as a flat list suitable for NSSM nssm.exe install argv (each flag and value as separate list elements).

4. Refactor tasks/el/main.yml

Replace the hardcoded command block (lines 142-152) in the podman_container task:

- name: Set fact llama_server_el_command
  ansible.builtin.set_fact:
    llama_server_el_command: "{{ lookup('template', 'templates/llama_server_command_el.j2') }}"

- name: Create pods
  containers.podman.podman_container:
    name: "llamacpp"
    image: "{{ llama_server_image }}:{{ llama_server_tag }}"
    state: started
    device: "{{ llama_server_devices | default(omit) }}"
    env: "{{ llama_server_env | default(omit) }}"
    ports: "{{ llama_server_bind }}:{{ llama_server_port }}:8080/tcp"
    volumes:
      - "{{ llama_server_home }}/models:/models:Z"
      - "{{ llama_server_home }}/.cache:/app/.cache:Z"
    command: "{{ llama_server_el_command }}"

5. Refactor tasks/windows/main.yml

Replace the hardcoded nssm.exe install argv block (lines 220-238):

- name: Set fact llama_server_windows_argv
  ansible.builtin.set_fact:
    llama_server_windows_argv: "{{ lookup('template', 'templates/llama_server_command_windows.j2') }}"

- name: Install llama-server service via NSSM
  ansible.windows.win_command:
    argv: "{{ llama_server_windows_argv }}"
  args:
    chdir: "{{ llama_server_windows_install_dir }}"
  register: __llama_nssm_install
  changed_when: __llama_nssm_install.rc == 0

6. Create MCP servers preset file template (optional, for future)

File: templates/mcp_servers.json.j2

{{ llama_server_mcp_servers | to_nice_json }}

Used when MCP servers need to be written to a file and passed via --mcp-servers flag pointing to a path.

7. Add tasks to write SSL/TLS and API key files (conditional)

When llama_server_ssl_cert or llama_server_api_key are set, ensure the referenced files exist on the remote host. This may require:

• A task to create the API key file if provided as a variable (not file path)
• A task to validate SSL cert/key paths exist

Files to modify

File	Change
ansible/roles/llama_server/defaults/main.yml	Add new role variables section
ansible/roles/llama_server/templates/llama_server_command_el.j2	New — EL container command template
ansible/roles/llama_server/templates/llama_server_command_windows.j2	New — Windows NSSM command template
ansible/roles/llama_server/templates/mcp_servers.json.j2	New — MCP servers JSON preset (optional)
ansible/roles/llama_server/tasks/el/main.yml	Replace hardcoded command with lookup('template') + set_fact
ansible/roles/llama_server/tasks/windows/main.yml	Replace hardcoded nssm argv with lookup('template') + set_fact
ansible/roles/llama_server/tasks/el/llama_exporter.yml	No change
ansible/roles/llama_server/files/scripts/	No change

Rollback plan

• The llama_server_argv_extra variable continues to work as a suffix in both templates.
• If template rendering fails, revert tasks/el/main.yml and tasks/windows/main.yml to the hardcoded command blocks.
• Delete the new template files.
• No data loss — templates only affect command construction, not state.

Testing

1. Baseline test: Run role with existing defaults — should produce identical command to current hardcoded version.
2. Variable test: Set individual variables (e.g., llama_server_ctx_size: 8192, llama_server_embeddings: true) and verify they appear in the rendered command.
3. Conditional test: Set llama_server_ssl_enabled: true with cert/key paths — verify --ssl, --ssl-cert, --ssl-key flags are included.
4. MCP test: Set llama_server_mcp_servers to a list of dicts — verify JSON serialization in --mcp-servers flag.
5. Backward compat test: Use llama_server_argv_extra alongside new variables — verify both are present.
6. Windows test: Same scenarios on Windows target with NSSM.

Open questions

1. Should --models-preset and --models-max remain in the template or stay hardcoded? They're core to router mode and unlikely to change. Keeping them in the template makes the command self-contained but adds complexity.
2. For mcp_servers — does llama.cpp accept the flag as --mcp-servers '{"name":"..."}' (JSON string) or --mcp-servers /path/to/config.json (file path)? This affects whether we need a separate file template.
3. Should defaults be opinionated (e.g., llama_server_ctx_size: 4096) or minimal (only required flags)? Opinionated defaults reduce host var boilerplate but may not suit all use cases.
4. Should we add assertions for mutually exclusive flags (e.g., --split-mode values)?