Native Windows vLLM: stable 0.27.1; 0.29.0 prereleases for Python 3.13/CUDA 13.0 and Python 3.14/CUDA 13.2 (CPU TorchAudio). PyTorch 2.13, FlashAttention/Rust, OpenAI-compatible serving, 10 KV formats, Multi-TurboQuant and experimental CPU/NVMe KV offload. No WSL or Docker.
windows gpu cuda pytorch nvidia triton msvc quantization kv-cache awq llm llm-serving vllm llm-inference flash-attention qwen kv-cache-compression turboquant vllm-windows multi-turboquant
-
Updated
Sep 20, 2026 - Python