Serving Flash-Next on a single Strix Halo with 128GB unified memory

abliter8Aug 2810 stars

A step-by-step serving recipe for running the 125B MoE on one Ryzen AI MAX+ 395 box, written by a hobbyist. Useful as a replicable baseline for that hardware.

Open on GitHub

Preview of Serving Flash-Next on a single Strix Halo with 128GB unified memory

Launch: Qwen3.8-Flash-Next
Use case: Amd local
Format: Repo
Source: github.com

abliter8

Having fun with AI. Always experimenting and going deeper. Not a tech pro like some but working hard with AI to create n

See all

peonist-ai

@peonist-ai · Aug 26

Container-based server tuned for the gfx1151 Strix Halo APU, aimed at squeezing maximum tokens per second from the 125B model on a mini-PC.

IIIIIllllIIIIIlllll

@IIIIIllllIIIIIlllll · Sep 19

An inference engine written for Ryzen AI MAX 395 on Linux and Windows that deploys only Qwen-family MoE models. The author is an automation engineer, not a full-time programmer, who built it to get more out of his mini PC.

pwilkin

@pwilkin · Sep 5

Optimised AMD toolchain by a llama.cpp contributor.

Niko1221

@Niko1221 · Sep 24

Open-source inference engine with an OpenAI/Anthropic-style local API that fits the 125B MoE onto a 12GB consumer GPU plus system RAM. The project behind the model's local-running surge.

Open on GitHub

Serving Flash-Next on a single Strix Halo with 128GB unified memory

More built with Qwen3.8-Flash-Next