Repos made with Qwen3.8-Flash-Next

Qwen3.8-Flash-Next 180B MoE on one DGX Spark — stable recipe with KV-loop fixes, KDA decode kernel, EAGLE speculation.

Qwen 3.8 Flash NVFP4 with PLE NVFP4 quantized to run on a single DGX Spark

• NEW: Qwen3.8 Flash: Over 7x Faster First Token | llama.cpp vs SGLang vs FreeToken. Benchmarked! https://youtu.be/RlsxXB5q-cA
• Why Qwen3 8 Flash Is So Cheap to Run 0 to 96GB VRAM Explained and Tested! https://youtu.be/L4XyWM6IrBc
• DeepSeek V4 Flash 284B on a single RTX 6000…

Qwen3.8-Flash-Next on 4x RTX 3090 with vLLM: 924,993 kv pool, 160-175tok/s single decode, 3 x 262K sessions resident, MTP, host-mapped PLE, pinned build and container recipes.

SGLang 0.5.21 for Qwen3.8-Flash-Next on one RTX PRO 6000: 264.5 tok/s decode, W4A16 routed experts (KLD 0.0111 vs BF16), 262K context, exact speculative rejection sampling, reproducible benchmarks.

Qwen3.8-Flash-Next (176B MoE, 6B active) at 2.5 bpw on one 24 GB GPU with 32 GB RAM: SGLang serving patch, kernels, measurement tools and engineering log

Qwen3.8-Flash-Next on a single DGX Spark (GB10): a 126.0 GiB checkpoint served in 121.63 GiB of unified memory. Two SGLang patches, reproducible recipe, GSM8K-verified.

llama.cpp fork optimized for serving qwen3.8 flash next on v100s

Reproducible 6-bit Qwen3.8 Flash Next tuning for M3 Ultra: attributed oMLX/MLX patches, setup scripts and raw benchmarks.

What I run on my inference server: Qwen3.8-Flash-Next on two RTX 3090s, and how I got here

First vLLM deployment of Qwen3.8-Flash-Next (RadixArk NVFP4) on 2× DGX Spark (GB10/SM121) — TP2+EP, MTP speculative decode, CUDA graphs, 262K context. Includes the 3-line PLE FP8 quant-resolver patch the official day-0 image needs, launcher, benchmarks, and gotchas for…

A 125B-parameter model, fully local, on a MacBook: Qwen3.8-Flash-Next at ~45 tok/s on Apple silicon.

Qwen3.8-Flash-Next on 4x RTX 3090: vLLM recipe, FP8 QSA KV reader for Ampere, benchmark receipts

A 180B model that officially needs two DGX Sparks, running on one at full 256k context - by compressing the one part of the model no quantizer can touch.

Production recipe for serving Qwen3.8-Flash-Next (180B Hybrid MoE) on a single 128GB NVIDIA DGX Spark with SGLang, NVMe PLE offloading & MTP speculative decoding.

exllamav3 on AMD Strix Halo (gfx1151 / RDNA 3.5): WMMA GEMV port, grouped MoE, LDS fix, int8 mixers, MTP — 41 tok/s Qwen3.8-Flash-Next. Based on sdougbrown/exllamav3 integration.

Vitronia — Qwen3.8-Flash-Next on AMD Strix Halo (gfx1151): 30 to 58 tok/s on Vulkan. Draft-head speculation, measurements, pitfalls, reproducible stack

Serving Qwen3.8-Flash-Next on a single AMD Strix Halo (Ryzen AI MAX+ 395, gfx1151, 128 GB unified memory)

Qwen3.8-Flash-Next (176B MoE) on a single RTX PRO 6000 Blackwell 96GB — NVMe-streamed PLE table, CUDA graphs with NEXTN speculation, measured against a dense 27B on the same card.

Qwen3.8-Flash-Next (104 GB MoE) on a hybrid iGPU + eGPU machine: a llama.cpp tree with device-resident speculative rollback, host-cached backend hand-offs, exact sparse-attention gather, co-batched verification and long-context indexing. 22 -> 50 tok/s single-stream, 84 tok/s on…

Qwen3.8-Flash-Next on ONE DGX Spark - 262K context, llama.cpp + unsloth GGUF

Qwen3.8-Flash-Next on Windows / Strix Halo: build recipe, Lemonade deployment, admission shim, benchmarks

Qwen3.8-Flash-Next (qwen4exp) on 4x RTX 3090 in upstream vLLM: 193 tok/s single stream, full 262K context, W4A16 experts + FP8 n-gram table on the NVMe via our patch, MTP draft, vision on or off. llama.cpp lane 2.

Run Qwen3.8-Flash-Next EXL3 as a local Hermes Agent with TabbyAPI, ExLlamaV3 and native OpenAI tool calling on consumer NVIDIA GPUs.