Strata benchmark: keeping the n-gram table in RAM gains ~1%, conversation parking 35x

Shot-Ad-4147Oct 4

Tested four ways of placing Strata's n-gram table on a 4090 48GB with 128GB RAM and found the default is right while parking conversations is what really helps.

Open on Reddit

Launch: Qwen3.8-Flash-Next
Use case: Benchmarks
Format: Build
Source: reddit.com

Shot-Ad-4147

See all

0xSero

@0xSero · Sep 29

Reports prefill and decode speeds for a single 24-32GB GPU plus NVMe, including an Intel Arc B70 result.

Niko1221

@Niko1221 · Sep 24

Open-source inference engine with an OpenAI/Anthropic-style local API that fits the 125B MoE onto a 12GB consumer GPU plus system RAM. The project behind the model's local-running surge.

MiaAI-Lab

@MiaAI-Lab · Sep 4

Step-by-step recipe to squeeze the 125B model onto one DGX Spark instead of two, with config and measured throughput.

peonist-ai

@peonist-ai · Aug 26

Container-based server tuned for the gfx1151 Strix Halo APU, aimed at squeezing maximum tokens per second from the 125B model on a mini-PC.

Open on Reddit

Strata benchmark: keeping the n-gram table in RAM gains ~1%, conversation parking 35x

More built with Qwen3.8-Flash-Next