6-bit Flash-Next tuned for the Mac Studio M3 Ultra

A reproducible recipe with attributed oMLX and MLX patches, setup scripts and raw benchmarks for the 6-bit quant on an M3 Ultra. Written by an Austin AI consultant for people running big models on Macs.

Open on GitHub

Preview of 6-bit Flash-Next tuned for the Mac Studio M3 Ultra

Launch: Qwen3.8-Flash-Next
Use case: Apple silicon
Format: Repo
Source: github.com

Thanh Pham

Obsessed with AI, transforming businesses with AI solutions in Austin, TX.

See all

carloslfu

@carloslfu · Aug 28

Streams expert weights from SSD so a Mac that cannot hold the model in memory still runs it at roughly 12 tok/s. A neat trick for memory-poor Apple machines.

Niko1221

@Niko1221 · Sep 24

Open-source inference engine with an OpenAI/Anthropic-style local API that fits the 125B MoE onto a 12GB consumer GPU plus system RAM. The project behind the model's local-running surge.

MiaAI-Lab

@MiaAI-Lab · Sep 4

Step-by-step recipe to squeeze the 125B model onto one DGX Spark instead of two, with config and measured throughput.

peonist-ai

@peonist-ai · Aug 26

Container-based server tuned for the gfx1151 Strix Halo APU, aimed at squeezing maximum tokens per second from the 125B model on a mini-PC.

Open on GitHub

6-bit Flash-Next tuned for the Mac Studio M3 Ultra

More built with Qwen3.8-Flash-Next