Practical notes from running local LLM inference on an Intel Arc Pro B70 32 GB system.
| Guide | Scope | Current takeaway |
|---|---|---|
| Intel Arc Pro B70 + llama.cpp SYCL setup | llama.cpp SYCL builds, runtime environment, GGUF serving, release checks | Best validated Qwen3.6 GGUF path uses f16 KV, flash attention, --parallel 1, -b 8192, -ub 4096. |
| Intel Arc Pro B70 + vLLM XPU + Qwen3.8-27B | vLLM XPU Docker serving, Qwen3.8 GPTQ INT4, FP8 KV, MTP4 | Current preferred Qwen3.8 path uses vLLM v0.30, float16, FP8 KV, MTP4, 180K context, and medium reasoning defaults. |
- Intel Arc Pro B70 / Battlemage / Xe2 inference tuning
- llama.cpp SYCL release tracking and long-context validation
- vLLM XPU Docker serving for Qwen-class models
- Practical measurements from real local agent workloads
# vLLM health and model list
curl http://127.0.0.1:11444/health
curl http://127.0.0.1:11444/v1/models
# vLLM metrics
curl -s http://127.0.0.1:11444/metrics | grep -E 'vllm:(prompt_tokens_total|generation_tokens_total|time_to_first_token_seconds_sum|e2e_request_latency_seconds_sum|spec_decode_num_draft_tokens_total|spec_decode_num_accepted_tokens_total|kv_cache_usage_perc)'
# B70 memory and power
xpu-smi --query-gpu=memory.used,memory.total,memory.free,utilization.memory,power.draw --id=0 --format=csv