The Strata author announced experimental Linux support for AMD Strix Halo with the best numbers on long-context decode and prompt processing using Unsloth Q4 and GSQ-RCO weights.
u/KnownAd4832
@KnownAd4832 · Sep 24
Went from 15 tok/s in llama.cpp to about 65 tok/s by writing an inference engine for this one model and this class of PC, with 2-bit RCO-GSQ quants running faster still.
