Qwen3.8-Flash-Next

German-language write-up of benchmark numbers, test harness and results for running the model on three second-hand 3090s.

A runtime focused on Qwen models that squeezes Flash-Next onto 16GB Macs. From the developer of the MacBat app.

Gave the same simple prompt for a widget-rich Android weather app to Flash-Next and other Qwen models to see which produced a working app.

Investigated why Strata 0.1.40.1 left VRAM free on a 5060 Ti, 3090 and two 3060s running Flash-Next IQ3 at 262k context, and found it was by design. A useful look at how the engine sizes its caches.

Posted a llama.cpp multi-GPU fork with binaries and Docker images after running five 262k-token sessions on six PCIe-only cards. Shows big-MoE long-context serving is possible on plain consumer rigs.

A developer who now codes mostly locally gave both models the same complex feature in the LlamaStash Rust project and compared the results.

Made a custom MLX quant sized for a 96GB machine after finding existing models lacked the right balance of speed and headroom.

A fork of the NInfer engine reaches 400 tg/s under ideal conditions with NVFP4 MoE experts on a 96GB card and the n-gram table on disk.

Compared Flash-Next, DeepSeek V4 Flash, GLM-5.3-Flash and other Qwen models using a private benchmark built from real coding, agentic and game tasks.

The Strata author announced experimental Linux support for AMD Strix Halo with the best numbers on long-context decode and prompt processing using Unsloth Q4 and GSQ-RCO weights.

Went from sceptic to convert after trying Strata on a low-end 5060 Ti with 32GB RAM and showed what the speed feels like in real demos.

Describes async input uploads and other ROCm and llama-server fixes written with Claude that sped up Flash-Next decode on a Lenovo P620 with four 7900 XTX. Offers to publish configs for other Radeon owners.

Show-and-tell post for a fork with CPU decode optimisations.

Showed that UkisAI's reasoning-efficient fine-tune finishes agent tasks at xhigh effort sooner than base Flash-Next at medium, using a Rust/CUDA engine for GB10.

Posted full flags for an NVFP4 Strata fork with FP8 PLE, INT8 KV cache and MTP on a single 32GB Blackwell card with 64GB DDR5.

The team released 95GB EXL3 weights and a new version of their ExLlamaV3-based Kyojin engine, using speculative decoding for about 1,400 tok/s prefill.

Got 17.5 tok/s decode and 360 tok/s prefill using a carousel buffer for experts and a second external SSD for parallel reads, both new tricks.

A 64MB resource dashboard with rich logs that collects metrics across multiple inference engines on the network, better than the default API stats.

Forked the DwarfStar engine, removed everything except Qwen3.8 Flash Next and ported extra optimizations for Mac, with a few upstream PRs in the queue.

Tested four ways of placing Strata's n-gram table on a 4090 48GB with 128GB RAM and found the default is right while parking conversations is what really helps.

Forked Strata with Opus 5.5 to run Flash-Next on a 2018 POWER9 machine with V100 GPUs over NVLink, hitting 113 tok/s decode.

Ran two local models against Opus 4.6 on the same tasks to see how much real coding work can stay local, and reports that one of them tied. Posted as numbers, not a fight.

Shows an open-air frame of three 3060s hitting 38-40 tok/s on Strata where llama.cpp managed 13, and asks if others reuse mining hardware.

Ran the 176B MoE on an ordinary gaming laptop with 32GB RAM using his own open-source TensorSharp engine, to see how far a normal machine can go.