A family of OptIQ-4bit quants. Ready to load.

Diffusion specifics DiffusionGemma is not loadable by stock mlx-lm/mlx-vlm; OptIQ ships a vendored, dependency-free decoder for it (mlx-optiq ≥ 0.2.3). Text + image generation and LoRA fine-tuning (a diffusion-native denoising loss) both work; optiq serve runs it with the fast confidence-threshold sampler by default (4.6–5× faster than the model's default). MTP / speculative drafting and KV-cache quant don't apply — the model is non-autoregressive and the parallel canvas un-masking is the native analog.

dhara-250m specifics A tri-mode tiny model built as a base to fine-tune, like Gemma-270M. Its mlx-native port registers with mlx-lm, so the whole OptIQ pipeline — convert, LoRA, eval, optiq serve, KV-quant — works. The recommended decode mode is self-speculation (--mtp): it drafts a block and AR-verifies it (two forwards per round), so output is identical to autoregressive decode but faster (~1.4× on M3 Max decoding greedily, several tokens per round). The model is overhead-bound, so 4-bit and bf16 decode at the same speed — the quant's win is size, not speed. Benchmark scores sit at the 250M floor and are preserved intact by quantization.

Hybrid KV cache Only the four full-attention layers carry a KV cache; the Mamba2 blocks keep recurrent state instead. Each repo ships a kv_config.json covering just those attention layers (three at 4-bit, one at 8-bit). Point optiq serve --kv-config kv_config.json at it. optiq kv-cache gained NemotronH support in v0.1.5 — earlier versions raised ZeroDivisionError on this architecture.

Heads up MiniCPM5 ships a hybrid <think> reasoning mode. Pass chat_template_kwargs={"enable_thinking": true} to wake it up; expect substantially higher math/tool scores in that mode. OptIQ's benchmark recipe forces thinking off for cross-family comparability, so the table reflects fast-assistant performance.

Mixed-precision KV now works on Gemma-4 (v0.1.3+) Each Gemma-4 repo above ships a recommended kv_config.json from a real sensitivity-analysis pass. Point optiq serve --kv-config kv_config.json at it. The runtime fills in for upstream mlx-lm's RotatingKVCache.to_quantized (which raises NotImplementedError in v0.1.2 and earlier) via optiq.runtime.kv.RotatingQuantizedKVCache, plus a small SDPA dispatch patch for Gemma-4's KV-sharing layers. The model still loads fine without the config (stock fp16 KV).

Recommended starting point For most users on a 36 GB Mac, the Qwen3.5-9B quant is the default. Strongest Capability-per-GB and runs at full 64 k context with mixed-precision KV. Bundled MTP head delivers ~1.4× decode via optiq serve --mtp. Drop to 4 B for laptops with less RAM, step up to 27 B if you have headroom.

from mlx_lm import load, generate

# Pick any of the 12. Stock mlx-lm, no special loader needed.
model, tok = load("mlx-community/Qwen3.6-27B-OptiQ-4bit")

prompt = tok.apply_chat_template(
    [{"role": "user", "content": "Explain mixed-precision quantization in 3 sentences."}],
    tokenize=False,
    add_generation_prompt=True,
)
out = generate(model, tok, prompt=prompt, max_tokens=300)
print(out)

Per-family notes Each model family has slightly different recommended sampling defaults and chat templates. See the MiniCPM5, Qwen3.5, Qwen3.6 and Gemma-4 getting-started guides.