Refactor llama_server role for Windows support and EL-specific setup

- Added Windows-specific tasks for llama_server role, including installation of dependencies, setup of services using NSSM, and configuration of firewall rules.
- Introduced a new task file for Windows Exporter setup.
- Consolidated EL-specific setup tasks into a separate include file.
- Updated inventory to include additional host for llama server.
- Adjusted error handling in Prometheus service removal task to prevent failures.
This commit is contained in:
2026-06-01 14:59:30 +02:00
parent bb8b14dcca
commit 3775fa4b6e
12 changed files with 1306 additions and 355 deletions
@@ -168,6 +168,37 @@ llama_server_models:
quant: "IQ4_XS"
revision: "main"
llama_server_args_extra:
[
"-t",
10,
"-np",
1,
"-b",
1024,
"-ub",
512,
"-fa",
"on",
"-ctk",
"q4_0",
"-ctv",
"q8_0",
"-cram",
-1,
"-sm",
"layer",
"-dev",
"Vulkan1,Vulkan2",
"-ts",
"8,12",
"-fit",
"off",
"-mg",
1,
"--mmap",
]
# # Optional global model preset options applied under [*] in models.ini.
# # Keys map to llama.cpp CLI args without leading dashes, short args, or LLAMA_ARG_* env names.
# llama_preset_global: