Nex AGI · Benchmark run
Nex-N2-mini BF16 on RTX PRO 6000 Max-Q
Nex-N2-mini in BF16 served by vLLM reached 134.515 tok/s decode at a 4K shared repository prefix and stayed at 89.148 tok/s at the 261,504-token 256K-class bucket. This is a no-speculation vLLM result as MTP is not yet available for this model as of the time of the benchmark.
Conditions
- Model
- Nex-N2-mini
- Family
- Nex AGI
- Parameters
- 35B
- Quantization
- BF16
- Rig
- windows-blackwell-max-q
- Topology
- Single GPU
- Nodes
- 1
- GPUs
- 1
- Tensor parallelism
- N/A
- Speculative decoding
- N/A
- Context
- 4,096 tokens
- Latency
- 118 ms
- Peak GPU temp
- 83 °C
- Engine
- vLLM
- Runtime version
- 0.23.0
- Operating system
- Windows (WSL2)
- Driver
- NVIDIA 596.72; host CUDA 13.2; vLLM Docker image vllm/vllm-openai:latest
- Model source
- https://huggingface.co/nex-agi/Nex-N2-mini
Topology notes
Single RTX PRO 6000 Blackwell Max-Q GPU exposed to vLLM through Docker Desktop on Windows with WSL2 GPU passthrough.
Recipe
This is a vLLM no-speculation result. The local model card recommends Nex AGI's customized SGLang fork for best local deployment, so this row should not be read as a best possible Nex-N2-mini runtime result across engines. vLLM also warned that prefix caching in Mamba cache align mode is experimental for this architecture.
Operator notes
Nex-N2-mini is published by Nex AGI which is built on Qwen3.5-35B-A3B-Base. The same model card recommends Nex AGI's customized SGLang fork for best local serving, but for this benchmark we utilized vLLM. The initial compatibility smoke used BF16 weights, vLLM 0.23.0, no quantization flag, prefix caching, KV12G, max_model_len 16384, max_num_batched_tokens 4096, and 128 generated tokens. It completed zero-failure at 153.844 tok/s for 4K and 150.960 tok/s for 8K, proving the base Qwen3_5MoeForConditionalGeneration path served correctly. The winning no-spec ladder used max_model_len 262144, KV12G, max_num_batched_tokens 8192, max_num_seqs 1, and 512 generated tokens. vLLM reported a 611,669-token GPU KV cache and 2.33x maximum concurrency for 262,144-token requests. The zero-failure decode series was 134.515 tok/s at 4K, 105.374 at 8K, 95.716 at 16K, 91.349 at 32K, 77.414 at 64K, 72.732 at 128K, and 89.148 at the 261,504-token 256K-class bucket. The deepest bucket's maximum input plus 512 output fit under the 262,144-token service window. Peak telemetry for the full ladder was 81.5 GB VRAM, 306 W, and 83 C.
Text generation sweep
Decode throughput at each measured context and concurrency, not just the headline number. Hover a point for its value, or toggle a series in the legend.