API: Add llama.cpp-compatible timings to completion responses

[midagedev]

Is your pull request related to a problem? Please describe.
Part of #454. TabbyAPI has no top-level timings object, so clients that read llama-server's timings (llama-swap, benchmark tools) get no speed figures from it.

Why should this feature be added?
The numbers are already in the finish chunk that usage reads. This exposes them in llama.cpp's shape without changing usage or the backend.

Examples
Test fixture: 1000 prompt tokens (900 cached), 50 generated in 1.2 s, draft 40 accepted / 8 rejected. Output of get_timings:

"timings": {
  "cache_n": 900,
  "prompt_n": 100,
  "prompt_ms": 60.0,
  "prompt_per_token_ms": 0.6,
  "prompt_per_second": 1666.6666666666667,
  "predicted_n": 50,
  "predicted_ms": 1200.0,
  "predicted_per_token_ms": 24.0,
  "predicted_per_second": 41.66666666666667,
  "draft_n": 48,
  "draft_n_accepted": 40
}

Where it goes:

• Non-streaming /v1/chat/completions and /v1/completions: top level.
• Streams: the last chunk. That is the usage chunk with include_usage, and the finish_reason chunk without it.
• Requests with more than one generation get none. usage still aggregates them.

Additional context

Keys. They follow llama.cpp's server_slot_stats::to_json. prompt_n excludes cached tokens, as in llama.cpp, and usage.prompt_tokens is unchanged. draft_n and draft_n_accepted appear only when draft tokens were produced, matching llama.cpp's n_draft_tokens > 0 guard.

Generation rate divides by predicted_n, not n - 1. llama.cpp uses n_gen - 1 because its first token comes from the last prompt batch, outside the generation time. exllamav3 works differently:

• It prefills up to the last prompt token (job.py#L1227, #L367-L368).
• It stamps time_first_token before the decode pass that produces the first token (generator.py#L638-L640).

So time_generate (job.py#L761) already covers every token.

Measured directly on exllamav3 1.5.0 (no draft model, greedy, no stop conditions, fixed token counts, two runs each):

tokens	time_generate (s)	÷ n	÷ (n − 1)
1	0.0628 / 0.0618	0.062	—
2	0.1047 / 0.1048	0.052	0.105
3	0.1566 / 0.1573	0.052	0.078
8	0.4138 / 0.4163	0.052	0.059

One token costs one pass rather than zero, and ÷ n is flat. Current main agrees: a max_tokens: 1 chat request returns "completion_tokens": 1, "completion_time": 0.11. With ÷ n, predicted_per_second also matches usage.completion_tokens_per_sec (the backend's own gen_tokens / gen_time), up to rounding.

Resolution. Times keep the finish chunk's 10 ms rounding. Changing it would touch handle_finish_chunk, which #443 also edits.

Live check. Run against a server on main 53da791 and on this branch, with GLM-5.3-Flash EXL3 on exllamav3 1.5.0, torch 2.10.0+cu128, and an MTP draft:

request	main	this branch
chat, stream, include_usage	no timings	last of 52 chunks, with usage
chat, stream	no timings	last of 52 chunks, with finish_reason
chat, non-stream	no timings	top level, draft_n 51 / accepted 45
chat, non-stream, n: 2	no timings	timings: null
text, stream, include_usage	no timings	last of 34 chunks, with usage
text, non-stream	no timings	top level

That run used the n - 1 rate I started with. The switch to ÷ n came after the measurement above and is covered by unit tests.

Tests. tests/test_timings.py has 24 tests. It fails to import on main (cannot import name 'Timings') and passes on this branch. The async stream drivers need a loaded model, so their compose and serialize functions are tested instead.

$ python -m unittest tests.test_timings tests.test_usage_stats
Ran 35 tests ... OK
$ ruff format --diff && ruff check   # 0.11.10
101 files already formatted
All checks passed!

🤖 Generated with Claude Code