Workstation gpu with Qwen3.8-Flash-Next

gabrielolympie

@gabrielolympie · Aug 31

SGLang patches and launchers delivering very high single-card throughput on a workstation GPU.

tpurtell

@tpurtell · Sep 7

Reproducible serving recipe for two Blackwell-class setups.

mratsim

@mratsim · Sep 2

Optimised serving stack for the Blackwell workstation card.

SSHdotCodes

@SSHdotCodes · Aug 26

262K context with exact sampled MTP on a single workstation card.

yepapa-nest

@yepapa-nest · Aug 28

CUDA-graph build for the workstation Blackwell.

SirTificate

@SirTificate · Aug 27

FP8 and NVFP4 with MTP speculation on a dual-Blackwell box.

u/flynth92

@flynth92 · Oct 7

Posted a llama.cpp multi-GPU fork with binaries and Docker images after running five 262k-token sessions on six PCIe-only cards. Shows big-MoE long-context serving is possible on plain consumer rigs.

A fork of the NInfer engine reaches 400 tg/s under ideal conditions with NVFP4 MoE experts on a 96GB card and the n-gram table on disk.

u/sdfprwggv

@sdfprwggv · Oct 5

Posted full flags for an NVFP4 Strata fork with FP8 PLE, INT8 KV cache and MTP on a single 32GB Blackwell card with 64GB DDR5.

Forked Strata with Opus 5.5 to run Flash-Next on a 2018 POWER9 machine with V100 GPUs over NVLink, hitting 113 tok/s decode.

Hardware notes, vLLM work and benchmarks from an unusual inference machine built around cheap Ascend cards, which are not CUDA drop-ins.