Skip to content

FP8 block-quantized MoE models have no usable kernel under moe_strategy=offload (0.1.3) #534

Description

@tielemao

Before you start

  • I have read the FAQ and my problem is not answered there.
  • I have read the Roadmap and this is not already planned there.
  • I have searched existing issues and found no duplicate.
  • I have restarted the Desktop app to pick up the latest update and the problem still happens.

What happened

After FreeToken Desktop updated to v0.2.0-beta.21 (bundling engine freetoken 0.1.3+gb332d3a97), serving an FP8 block-quantized MoE model fails at startup with:

KernelSelectionError: kernel 'triton' was requested but cannot run here:
has no CPU executor format; decode must run on the GPU
The scheduler process dies and the API server exits with code 15.

Environment: Windows 11, RTX 5060 Ti 16 GB (14.8 GiB free), Python 3.12, engine 0.1.3+gb332d3a97 (win_amd64 wheel from the beta channel).

Model: Qwen3.6-35B-A3B-FP8 — FP8 block quantization with weight_block_size: [128, 128] and bf16 scales.

Command:

ft serve --model F:\models\Qwen3.6-35B-A3B-FP8 --port 1919 --moe-backend auto --max-running-requests 4 --memory-ratio 0.85
Root cause analysis: Since free VRAM (14.8 GiB) cannot hold the model, the engine auto-selects moe_strategy='offload', which sets the decode target to CPU. However, for FP8-block-quantized MoE layers,

freetoken/layers/quantization/moe/fp8_block.py
declares only one candidate kernel, TritonFp8BlockMoEKernel, which has cpu_format = None and rejects CPU decode (cpu_ok=False: "has no CPU executor format; decode must run on the GPU"). The alternative dsv4 kernel is skipped because it only serves e8m0 block scales (this checkpoint uses bf16 scales). With zero usable kernels, the new kernel-selection check introduced in 0.1.3 raises KernelSelectionError.

Notably, fp8_block appears to be the only quantization format without a CPU executor format — nvfp4 (cpu_format="nvfp4"), mxfp4 (cpu_format="ds_fp4"), and unquantized (cpu_format="bf16") all have one.

Regression: The same model and command worked on engine 0.1.2 (which has no kernel-selection validation). Rolling back to 0.1.2 restores service.

Request: Please add a CPU executor format for fp8_block, or allow offload-strategy decode of fp8_block experts to run on the GPU.

Desktop app version

0.2.0-beta.21

OS

Windows 11

OS details

No response

GPU and driver

RTX 5060Ti 16GB

CPU and system RAM

i7-12700 64GB

Checkpoint

Qwen3.6-35B-A3B-FP8

Model settings

No response

Engine log


Anything else

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Desktopproblem related to FreeToken DesktopbugSomething isn't workingwindows

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions