vLLM recipe with FP8 KV reader for four 3090s
Ampere-compatible reader plus benchmark receipts.

Ampere-compatible reader plus benchmark receipts.

sergqwer
@sergqwer · Sep 29
Adds NVIDIA's NVFP4 quantisation to Strata so a single 12GB+ RTX 20 to 50-series card can host the MoE.

DominikBucko
@DominikBucko · Aug 30
Dual-3090 build with 128GB RAM reporting about 4,200 tok/s prefill and 111 tok/s decode on long prompts.
