Qwen

Quantisation and launch recipe focused purely on decode speed at 4-bit.

Intel GPU route with experts streamed from NVMe and 32GB host RAM.

Experiment that edits the model's n-gram embedding table to inject removable facts without fine-tuning.

A bespoke inference engine for the Qwen4-style architecture that runs Flash-Next on a 16GB card.

Hand-tuned ROCm inference path for AMD RDNA4 cards running the 125B model.

Packaged run recipe for a single AMD APU.

Tested llama.cpp GGUF route for a single Spark or Asus GX10 box.

VLLM patch and guide for running Flash-Next on a CMP 170HX with streamed embeddings.

Restart-resumable local agent stack that runs Qwen3.8 on one RTX 3090/4090/5090.

Serving the model through ExLlamaV3 quantisation.

Video of a generated web scene compared against a rival model, with a live demo link.

Convenience script that boots the model on a single DGX Spark or variant.

Community vLLM build that runs the model on cheap ex-mining SM80 hardware.

Single Go binary exposing the web chat as an API.

Cheap dual-card build that also scales to 800 tok/s at 23 concurrent users.

SGLang patches and launchers delivering very high single-card throughput on a workstation GPU.

An LLM engine written from scratch in HIP for AMD Strix Halo and Qwen3.8-Flash-Next, reaching about 50 tokens per second out. Built to see how far a single-model engine can go.

A llama.cpp fork that speeds ROCm prefill and fits multi-token prediction for GSQ quantizations on AMD gfx1151 hardware.

IQ3_XXS 177B build served from a 24GB laptop GPU with 64GB RAM.

Measured run using 6.6GiB of VRAM and about 48GiB of RAM on a laptop-class card.

Reproducible serving recipe for two Blackwell-class setups.

Reproducible Strix Halo setup with receipts for speed and long context.

Another route to fit the NVFP4 checkpoint on one Spark by offloading its embedding table.

Shrinks long-context memory use.