FrozenInstanceError: cannot assign to field 'audio_config'

[TheFausap]

Before you start

• I have read the FAQ and my problem is not answered there.
• I have read the Roadmap and this is not already planned there.
• I have searched existing issues and found no duplicate.
• I am on the latest release, or on a freshly rebuilt main when building from source.

What happened

Hello

I was loading Gemma 4 26B A4B with default arguments, and after 10 seconds of loading, I got the error in the title.

How did you install FreeToken

pip / uv wheel

FreeToken version

freetoken version 0.1.3

OS

Other Linux

OS details

Oracle Linux 10

GPU and driver

GTX 1660 - 6GB VRAM - CUDA VER 13.3

CPU and system RAM

AMD Ryzen 5 3600 6-Core Processor, 32GB RAM

Checkpoint

zerofata/G4-MeroMero-26B-A4B-gguf

Command

ft serve --model ../JCN3690/LLM/G4-MeroMero-26B-A4B-Q4_K_M.gguf

Full log

[2026-09-17|19:40:28] INFO     Parsed arguments:
ServerArgs(model_path='../JCN3690/LLM/G4-MeroMero-26B-A4B-Q4_K_M.gguf', tp_info=DistributedInfo(rank=0, size=1), dtype=torch.bfloat16, max_running_req=4, attention_backend='auto', moe_strategy='auto', quant_backend=None, ple_backend='disk', expert_load='auto', moe_cache_size=0, moe_cache_rate=None, moe_cache_auto=False, kv_reserve_tokens=8192, moe_cache_policy='lru', moe_prefill_overlap=True, moe_prefill_hit_d2d=False, moe_collect_stats=False, moe_cpu_threads=0, moe_cpu_layers=None, moe_hybrid_max_fetch=-1, cuda_graph_bs=None, cuda_graph_max_bs=None, page_size=1, memory_ratio=0.9, linear_state_cache_ratio=2.0, swa_full_tokens_ratio=0.2, swa_num_pages_override=None, distributed_timeout=60.0, use_dummy_weight=False, use_pynccl=True, max_seq_len_override=None, num_page_override=None, num_token_override=None, mm=MultimodalConfig(disabled_encoders=frozenset(), embed_cache_device='cpu', encoder_weights='host', image_min_tokens=None, image_max_tokens=None, processor_kwargs={}), max_extend_tokens=8192, cache_type='radix', offline_mode=False, decode_log_interval=40, special_token_ckpt=False, _unique_suffix='.pid=1789721', server_host='127.0.0.1', server_port=1919, num_tokenizer=0, silent_output=False, shell_mode=False, served_model_name='G4-MeroMero-26B-A4B-Q4_K_M.gguf', tool_call_parser='gemma4', reasoning_parser='gemma4', sampling_defaults='model', max_output_tokens=None, enable_cache_report=False, allowed_media_domains='', allowed_local_media_path='', cors_origins='tauri://localhost,http://tauri.localhost,http://localhost:1420', gpu=(), gpu_assigned=None)
[2026-09-17|19:40:28|FrontendAPI] INFO     Default sampling config (source=model): temperature=1.0, top_k=64, top_p=0.949999988079071
INFO:     Started server process [1789721]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:1919 (Press CTRL+C to quit)
/home/fausap/Documents/FreeToken/python/freetoken/engine/engine.py:1135: FutureWarning: torch.cuda._set_allocator_settings is deprecated. Use torch._C._accelerator_setAllocatorSettings instead.
  torch.cuda.memory._set_allocator_settings("expandable_segments:True")
[2026-09-17|19:40:31|core|rank=0] INFO     Enabled expandable_segments (override via PYTORCH_ALLOC_CONF)
Process freetoken-TP0-scheduler:
[2026-09-17|19:40:51|FrontendAPI] ERROR    Backend supervisor: FrozenInstanceError: cannot assign to field 'audio_config'
Traceback (most recent call last):
  File "/home/fausap/miniconda/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/fausap/miniconda/lib/python3.13/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fausap/Documents/FreeToken/python/freetoken/server/launch.py", line 81, in _run_scheduler
    scheduler = Scheduler(args)
  File "/home/fausap/Documents/FreeToken/python/freetoken/scheduler/scheduler.py", line 68, in __init__
    self.engine = Engine(config)
                  ~~~~~~^^^^^^^^
  File "/home/fausap/Documents/FreeToken/python/freetoken/engine/engine.py", line 310, in __init__
    _adjust_config(config)
    ~~~~~~~~~~~~~~^^^^^^^^
  File "/home/fausap/Documents/FreeToken/python/freetoken/engine/engine.py", line 1424, in _adjust_config
    model_config = config.model_config
                   ^^^^^^^^^^^^^^^^^^^
  File "/home/fausap/miniconda/lib/python3.13/functools.py", line 1025, in __get__
    val = self.func(instance)
  File "/home/fausap/Documents/FreeToken/python/freetoken/engine/config.py", line 134, in model_config
    setattr(hf_config, key, None)
    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 20, in __setattr__
dataclasses.FrozenInstanceError: cannot assign to field 'audio_config'
[2026-09-17|19:41:01|FrontendAPI] ERROR    Backend worker is gone and cannot be restarted; stopping the API server
INFO:     Shutting down
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
INFO:     Finished server process [1789721]
Terminated
(FreeToken) (base) fausap@localhost:~/Documents/FreeToken$ /home/fausap/miniconda/lib/python3.13/multiprocessing/resource_tracker.py:400: UserWarning: resource_tracker: There appear to be 3 leaked semaphore objects to clean up at shutdown: {'/mp-i_860heo', '/mp-mvjqfkvr', '/mp-mythgrfo'}
  warnings.warn(

Anything else

No response

[jason-fxz]

gguf not supported yet

[TheFausap]

In the code gemma4 seems supported. the model I am using is a fine-tuning of Gemma 4 26B