ninfer

Star

Here are 5 public repositories matching this topic...

BenGamliel / NInfer-Manager

Star

Lightweight Windows GUI, tray controller and model manager for NInfer

windows winforms nvidia openai-api llm local-ai ninfer

Updated Sep 3, 2026
C#

reinwu / ninfer-L20

Star

Port NInfer, a single-GPU CUDA inference engine, to the NVIDIA L20 (Ada sm_89, 92 SMs, 48 GB): patch set, build tooling, and measured results

cuda ada-lovelace wsl2 llama-cpp llm-inference qwen3 sm89 ninfer nvidia-l20 gpu-port

Updated Sep 20, 2026
PowerShell

reinwu / llamacpp-stats-dashboard

Star

Measuring proxy + throughput dashboard for local LLM engines (llama.cpp, NInfer): live tok/s, cache-hit rate, TTFT, and history charts. Single-file panel, stdlib-only proxy, MIT.

python dashboard metrics prometheus throughput observability llamacpp llm-inference tokens-per-second ninfer

Updated Sep 20, 2026
HTML

lsh123 / ninfer-monitor

Star

A cross-platform desktop app that tails the NInfer server request log in real time and displays the core operational details of a running NInfer inference server in a live GUI dashboard:

monitor llm ninfer

Updated Sep 17, 2026
Rust

Azhu9701 / ninfer-4090d

Star

Qwen3.8-27B on RTX 4090 D (48GB): production deployment of NInfer with MTP7 + E8 KV + NVMe disk cache, 195 tok/s decode, crash forensics for WDDM desktop GPUs

windows cuda mtp kv-cache directstorage llm-inference speculative-decoding qwen3 ninfer rtx-4090d

Updated Sep 3, 2026
C++

Add this topic to your repo

To associate your repository with the ninfer topic, visit your repo's landing page and select "manage topics."

Learn more