Flash-Next at the full 262K context on 31GB RAM: 130 tok/s decode
Shows three Strata patches that let a 5090 plus a 4070 Ti Super on a PCIe x1 slot run the 76GB IQ3_XXS quant at 1,800 tok/s prompt processing.
Shows three Strata patches that let a 5090 plus a 4070 Ti Super on a PCIe x1 slot run the 76GB IQ3_XXS quant at 1,800 tok/s prompt processing.
sergqwer
@sergqwer · Sep 29
Adds NVIDIA's NVFP4 quantisation to Strata so a single 12GB+ RTX 20 to 50-series card can host the MoE.

DominikBucko
@DominikBucko · Aug 30
Dual-3090 build with 128GB RAM reporting about 4,200 tok/s prefill and 111 tok/s decode on long prompts.
