Six RTX 3090/4090 cards without NVLink: Flash-Next prefill 8-10x faster than stock…

Posted a llama.cpp multi-GPU fork with binaries and Docker images after running five 262k-token sessions on six PCIe-only cards. Shows big-MoE long-context serving is possible on plain consumer rigs.

Open on Reddit

Launch: Qwen3.8-Flash-Next
Use case: Workstation gpu
Format: Build
Source: reddit.com

flynth92

See all

Niko1221

@Niko1221 · Sep 24

Open-source inference engine with an OpenAI/Anthropic-style local API that fits the 125B MoE onto a 12GB consumer GPU plus system RAM. The project behind the model's local-running surge.

MiaAI-Lab

@MiaAI-Lab · Sep 4

Step-by-step recipe to squeeze the 125B model onto one DGX Spark instead of two, with config and measured throughput.

peonist-ai

@peonist-ai · Aug 26

Container-based server tuned for the gfx1151 Strix Halo APU, aimed at squeezing maximum tokens per second from the 125B model on a mini-PC.

slee-persis

@slee-persis · Aug 26

Ensemble harness claiming a group of small Qwen3.8 models reach frontier-level coding scores at a fraction of the cost. Results are self-reported and worth verifying.

Open on Reddit

Six RTX 3090/4090 cards without NVLink: Flash-Next prefill 8-10x faster than stock…

More built with Qwen3.8-Flash-Next