Expose exllamav3's recurrent checkpoint intervals in the model config

[matthematics1137]

Is your pull request related to a problem? Please describe.
exllamav3 checkpoints the recurrent state of hybrid models so a request with a matching prefix resumes instead of re-prefilling, and its Generator takes two interval arguments for that. TabbyAPI passes neither, so a long prompt is checkpointed only near its end and any edit earlier in a cached prompt (a system header, an early tool result) costs a full re-prefill.

Why should this feature be added?
Two optional model: keys (also on /v1/model/load) pass the intervals through only when set, so the engine defaults apply otherwise. With a 1,024-token ingestion grid the cost of an edit becomes proportional to its distance from the end of the prompt instead of all-or-nothing, for +2-3% cold prefill and one recurrent state of host RAM per checkpoint.

Examples
Edit at token 1,321 of a 5,266-token cached prompt: TTFT 10.2 s today, 8.6 s at interval 1024; edit at token 3,769: 10.1 s today, 5.0 s at 1024 (full table below, measured on a 27B Qwen3.5-class hybrid on a 12 GB RTX 4080 Laptop).

Additional context
Details, measurements, testing and risks follow. Companion PR: #467 (slot-cost log); the two touch the same docs row.

---

Motivation

exllamav3 keeps checkpoints of the recurrent state of hybrid models (Gated DeltaNet, short-conv, SWA
layers) in host RAM so a request with a matching prefix resumes instead of re-prefilling. Generator takes
recurrent_checkpoint_interval (default 2048; the last 2 x chunk_size tokens of the prompt, and
generation) and recurrent_checkpoint_interval_pp (default 32768; the rest of the prompt). TabbyAPI passes
neither (create_generator), so a long prompt is checkpointed only near its end; recurrent states cannot
roll back, so an edit before that (a system header, an early tool result) costs a full re-prefill.

Measured on a 27B Qwen3.5-class hybrid (48 recurrent layers, EXL3 2.0 bpw, MTP drafter, 12 GB RTX 4080
Laptop at 80 W): 5,266-token prompt, one word edited at the given token position after the unedited
prompt was served once, TTFT in seconds, median of two repeats. The interval was set with a lab-only
engine patch (env var), since TabbyAPI has no knob:

ingestion interval	cold	unedited	edit @276	@1,321	@2,024	@2,639	@3,769	@4,373
32768 (today)	10.0	1.2	10.2	10.2	10.1	10.1	10.1	3.0
2048	10.2	1.2	10.4	10.4	6.8	6.8	6.9	3.1
1024	10.3	1.2	10.5	8.6	6.8	6.9	5.0	3.1

Every cell is "replay from the last checkpoint at or below the edit" at ~470 tok/s: the edit cost becomes
proportional to the distance from the edit to the end of the prompt instead of all-or-nothing. Price:
+2-3% cold prefill; one host-RAM checkpoint per extra grid point (148 MiB for this model: +147 MiB RSS at
2048, +447 MiB at 1024, from the 4 GB sysmem_recurrent_cache budget); VRAM identical (8,756 MiB).
(Evidence: the author's lab notes, see the footer.)

Change

• common/config_models.py: optional model: keys recurrent_checkpoint_interval and
  recurrent_checkpoint_interval_pp (int, multiple_of=256, gt=0, default None = engine defaults).
• backends/exllamav3/model.py: create() reads both (warning if set for a model without recurrent
  layers); create_generator() passes each to AsyncGenerator only when set, so the engine defaults
  stay in force otherwise, and logs the effective values (the engine rounds _pp up to a chunk multiple).
• endpoints/core/types/model.py: the same fields on ModelLoadRequest (/v1/model/load), like
  chunk_size; a separable hunk if config-only is preferred.
• config_sample.yml, docs/02.-Server-options.md: documentation with the trade-off above; the sample
  entries are what generate_config_file() produces from the new descriptions.

Testing

On the author's machine: the engine-side effect (table) was measured with the lab env-var patch, not with this PR, which was
checked statically only (no server run): it applies cleanly to the pristine files; py_compile; pydantic
accepts 1024/2048 and rejects 1000 and 0 for both keys on ModelConfig and ModelLoadRequest;
generate_config_file() reproduces the sample entries. Owner re-test: load Qwen3.8-27B EXL3 with
recurrent_checkpoint_interval_pp: 1024 (then 2048), check the "Using recurrent checkpoint intervals"
line, rerun tools/edit_probe.py and expect the rows above and +447 / +147 MiB RSS; a non-recurrent
model with the keys set (warning only); one API load with the keys; use_as_default with one key.

Risks and compatibility

• Defaults unchanged: with both keys unset the generator call is the old one. exllamav3 1.4.3 and 1.4.7
  accept both arguments (checked); Generator.__init__ takes **kwargs, so older engines ignore them.
• Pydantic validates multiple_of=256 for both keys; the engine's own assert only checks the generation
  interval (twice, an upstream typo) and rounds _pp up to a chunk multiple.
• A larger recurrent_checkpoint_interval also widens the allocation boundary in TabbyAPI's
  validate_context_requirements and the engine's max_rq_tokens alignment (job.py); _pp has no such
  coupling. Denser grids fill sysmem_recurrent_cache sooner (4 GB = 27 checkpoints here); overflow only
  evicts older checkpoints (LRU).
• With the companion PR for the recurrent slot-cost log applied too, the docs/ hunk needs a trivial
  manual merge (both touch the max_batch_size row); the other files stack in either order.

---

Measurements and the pre-registered experiment log are in the author's lab notes (private for now; the relevant tables are reproduced above, and the full write-up is available on request).

🤖 Generated with Claude Code

https://claude.ai/code/session_01FWY9bG6iQAswvYgYFch6ex

[Downtown-Case]

+1

In fact, I came here looking for precisely these knobs. When (say) running Deepseek V4 or a similar model using most of system RAM, they're quite useful for tuning caching behavior.