Alibaba Cloud· Released Aug 26
Open-weight 125B-parameter multimodal MoE (about 6B active, plus a 51B n-gram embedding table) from Alibaba's Qwen team, an early preview of the Qwen4 architecture. Its n-gram table can sit in system RAM or on SSD, which let community engines such as Strata run it on gaming PCs and Macs.
Open-source inference engine with an OpenAI/Anthropic-style local API that fits the 125B MoE onto a 12GB consumer GPU plus system RAM. The project behind the model's local-running surge.
Tool · 16.8K stars
Step-by-step recipe to squeeze the 125B model onto one DGX Spark instead of two, with config and measured throughput.
Repo · 594 stars
Container-based server tuned for the gfx1151 Strix Halo APU, aimed at squeezing maximum tokens per second from the 125B model on a mini-PC.
Repo · 876 stars
Ensemble harness claiming a group of small Qwen3.8 models reach frontier-level coding scores at a fraction of the cost. Results are self-reported and worth verifying.
Research · 736 stars
Tells the story of the 12GB RTX 5070 user who wrote Strata after finding llama.cpp too slow.
Post · 1.6K likes
Follow-up engine that speeds single-Spark decoding of the 125B model via a custom tensor layout.
Repo · 199 stars
Posts a video of the 125B MoE running at about 21 tok/s decode with a quarter-million-token window on one gaming card, no quantised cache.
Build · 4.7K likes
Streams expert weights from SSD so a Mac that cannot hold the model in memory still runs it at roughly 12 tok/s. A neat trick for memory-poor Apple machines.
Tool · 422 stars
Early write-up of how to serve the model on a single DGX Spark, published within a day of the weights landing.
Repo · 409 stars
Dual-node vLLM deployment of the NVFP4 build across a pair of DGX Sparks, for higher context and throughput.
Repo · 404 stars
Reports prefill and decode speeds for a single 24-32GB GPU plus NVMe, including an Intel Arc B70 result.
Post · 685 likes
Launch-day announcement of GGUF quants and a guide for CPU and unified-memory setups.
Post · 2.5K likes
Local app that spins up a GPU in your own Colab account and serves Qwen3.8, letting people without a big card try the model.
Tool · 96 stars
A llama.cpp variant optimised for Ampere cards, reporting very high token rates on 3090 and 3090 Ti, relevant for Flash-Next hosts.
Tool · 148 stars
Japanese explainer of the lazy dequantisation and expert cache design, with about 65 tok/s.
Post · 346 likes
Companion to the solo recipe for a pair of Sparks.
Repo · 121 stars
Adds NVIDIA's NVFP4 quantisation to Strata so a single 12GB+ RTX 20 to 50-series card can host the MoE.
Repo · 54 stars
An inference engine written for Ryzen AI MAX 395 on Linux and Windows that deploys only Qwen-family MoE models. The author is an automation engineer, not a full-time programmer, who built it to get more out of his mini PC.
Repo · 64 stars
VLLM patch that keeps the huge n-gram table on disk and reaches about 44 tok/s on one Spark.
Repo · 128 stars
Shows how paging the embedding table to NVMe lets the 180B-class checkpoint run on a single DGX Spark.
Repo · 127 stars
Dual-3090 build with 128GB RAM reporting about 4,200 tok/s prefill and 111 tok/s decode on long prompts.
Repo · 108 stars
Optimised AMD toolchain by a llama.cpp contributor.
Repo · 73 stars
Serves turboderp's EXL3 quant of the model on one Spark using exllamav3.
Repo · 61 stars