BuildPrivateAI
Thread

vllm

4 runs · 1 note
Signed runsThe measured evidence
Model Tok/s Peak VRAM TTFT Verdict
Qwen3.6-35B-A3B-FP8 · FP8
NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition (96GB) · Single GPU
192.4 63.8 GiB 127.8 ms Verdict Great for DGX Sparks
gemma-4-26B-A4B-it-FP8-Dynamic · FP8
NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition (96GB) · Single GPU
185.5 62.8 GiB 320.1 ms Verdict Great for DGX Sparks
Nex-N2-mini · BF16
NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition (96GB) · Single GPU
134.5 81.5 GiB 118.4 ms Verdict Best Small MoE
Qwen3.6-27B-Text-NVFP4-MTP · NVFP4
NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition (96GB) · Single GPU
118.9 64.3 GiB 137.9 ms Verdict Great local coder
NotesThe analysis

Private AI Is Not Just About Privacy: It’s About Cost, Consistency, and Control

Private AI is not just about keeping data private. For serious AI users, local LLMs matter because of cost, consistency, control, open-weight models, and the limits of hosted AI subscriptions.

Mark Chang · Jun 27, 2026