Flash-Next on a 24GB RTX 4090 with 250K context

Open-source inference engine with an OpenAI/Anthropic-style local API that fits the 125B MoE onto a 12GB consumer GPU plus system RAM. The project behind the model's local-running surge.

Tells the story of the 12GB RTX 5070 user who wrote Strata after finding llama.cpp too slow.

Launch-day announcement of GGUF quants and a guide for CPU and unified-memory setups.

A llama.cpp variant optimised for Ampere cards, reporting very high token rates on 3090 and 3090 Ti, relevant for Flash-Next hosts.