Max Bitzer

FLE: Fast Local Experts — an 89 GB Mixture-of-Experts model at ~90 tokens/s on one 32 GB GPU

We run Qwen3.8-Flash-Next, a 512-expert Mixture-of-Experts (MoE) language model, on a single workstation with one 32 GB GPU, 120 GB of system RAM and 16 CPU cores. Two pieces make this practical. First, FLE-IQ3_M (internally G3), a hybrid quantization that combines the Unsloth UD-IQ4_XS release with expert layers from...