Strata fork for an IBM AC922 POWER9 server: 7,357 tok/s prefill
Forked Strata with Opus 5.5 to run Flash-Next on a 2018 POWER9 machine with V100 GPUs over NVLink, hitting 113 tok/s decode.
Forked Strata with Opus 5.5 to run Flash-Next on a 2018 POWER9 machine with V100 GPUs over NVLink, hitting 113 tok/s decode.
Niko1221
@Niko1221 · Sep 24
Open-source inference engine with an OpenAI/Anthropic-style local API that fits the 125B MoE onto a 12GB consumer GPU plus system RAM. The project behind the model's local-running surge.
MiaAI-Lab
@MiaAI-Lab · Sep 4
Step-by-step recipe to squeeze the 125B model onto one DGX Spark instead of two, with config and measured throughput.
