u/Beamsters@Beamsters · Sep 9A co-creator of the MLX-serve engine support shows Flash-Next staying coherent at about 760k tokens of context with an 8-bit KV cache, demonstrated in a video.
