Qwen3.8-Flash-Next

Documents a tuning ladder with measured throughput across context lengths.

Pure-CPU C implementation aimed at laptops with no discrete GPU.

Optimised serving stack for the Blackwell workstation card.

Targets old V100 server cards, supporting Flash-Next and GLM-5.3-Flash.

Tool to add knowledge through the PLE n-gram table.

Custom 4-bit llama.cpp build for AMD Strix Halo mini PCs.

An experiment that pushes the Qwen3.8 models onto the Neural Engine of Apple Silicon rather than the GPU. A long-shot attempt to cut power use for local inference.

Early notes on fitting the 126GiB checkpoint onto one DGX Spark.

Build and run notes for the AtomicChat 4.27-bpw quant with multi-token prediction on a mid-range gaming PC.

OMLX setup using prompt-lookup MTP drafts that speed edit turns by 1.3 to 2.2x with exact output. By a privacy educator who wants coding models that stay on his own machine.

Runs the 177B build across RTX 5060 Ti and 4070 Super cards.

Experimental one-box DGX Spark deployment.

Verified guide for running the model on a high-memory Mac.

262K context with vision and FP8 KV cache on four used GPUs.

Extends Perplexity's open Lily inference engine so it can run Qwen3.8-Flash-Next on M5 Macs. Shows how a community member adopted a corporate engine for the new model.

A stability-focused recipe for a single DGX Spark with a KDA decode kernel and EAGLE speculation, fixing KV-loop failures others hit.

Fits the model by quantising the embedding table too.

Community measurements of memory needs at different quants and contexts.

Showcases long-context concurrent serving on used cards.

262K context with exact sampled MTP on a single workstation card.

SGLang patch and kernels for a single mid-range card.

Day-one single Spark build.

A fork tuned for Volta-generation Tesla V100 cards, giving used data-centre GPUs a path to run the 125B MoE. Author is a hobbyist maker.

A reproducible recipe with attributed oMLX and MLX patches, setup scripts and raw benchmarks for the 6-bit quant on an M3 Ultra. Written by an Austin AI consultant for people running big models on Macs.