Repository navigation
[memory] layout: fixed-point leaf values take 8 bytes - #68
Merged
Merged
Conversation
This was referenced Nov 6, 2026
Merged
victor-ab
force-pushed
the
victor-ab/leaf-i64
branch
from
November 6, 2026 14:26
6788466 to
a4abfcb
Compare
victor-ab
force-pushed
the
victor-ab/leaf-i64
branch
from
November 6, 2026 23:40
a4abfcb to
8cd6e7b
Compare
ukaratay
approved these changes
Nov 7, 2026
ukaratay
left a comment
Collaborator
There was a problem hiding this comment.
Approving. The packing is exact: the odd significand is below 2^53, and odd << shift fits an i128 by construction, since it came from one.
victor-ab
force-pushed
the
victor-ab/leaf-i64
branch
from
November 7, 2026 18:29
8cd6e7b to
9ac23f5
Compare
The leaf table held each Fixed leaf as an i128, though it holds an original f64, not a sum. Pack the leaf as a signed odd mantissa with eight low bits for its shift, halving the Fixed table without changing the sum's scale or rounding. A finite f64's odd mantissa is below 2^53 in magnitude, so it fits with the low bits in an i64. The fixed-scale rule bounds the shift to 0 through 123 for a nonempty model, or 124 with zero trees. Arithmetic right shift recovers the signed mantissa; both signed zeros pack as zero. Every Fixed plan fits, for either threshold type, so no further plan or fallback is needed. Wide leaves keep their digits, and sums and difference arrays still use i128 or wide accumulators: their significant bits need not fit a leaf's encoding. Both the constant walk's sums and the grouped leaf pass reconstruct the fixed integer before adding it. The leaf pass takes the stored value type apart from the sink's value type, so Wide leaves still pass through unchanged. The conversion is always inlined to keep production independent of research. Test all shifts, signed zeros, integers, the floating-point extremes and random finite values at exact scales. The existing layout, oracle and reference-walk tests check the decoded leaves and both prediction paths; introspection now counts eight bytes per Fixed leaf.
victor-ab
force-pushed
the
victor-ab/leaf-i64
branch
from
November 7, 2026 20:49
9ac23f5 to
47d5790
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Under a fixed-point plan, each leaf value is an
i128: 16 bytes, the next largest part of a loaded model after its nodes and rank tables. A finite f64's fixed-point value is its odd significand, below 2^53, shifted left, so 8 bytes hold it exactly.Design
layout: store fixed leaves in eight bytesLeafValues::Fixedholds aFixedLeaf, ani64per leaf: the signed odd significand shifted left 8 bits, and the shift in the low byte; zero is canonical. Itsi128value is rebuilt exactly where a leaf is added, single-row and grouped.Plan::Fixedmodel qualifies, float32 models included: significands are below 2^53 and shifts are 0 to 123 for nonempty models (src/exact/mod.rs). Wide plans keep their limbs, and the accumulators stay 128-bit or wide.Evidence
predict_fixedper request on the generated requests, one thread, Apple M4 Pro: p50 change from [memory] layout: exceptions take 6 bytes, with 16-bit feature-relative slots and a 32-bit fallback #67, median of four paired rounds (A B B A, 1 s per cell), with the rounds' range:On AVX-512 (GCP c4-standard-8, Intel Xeon Platinum 8581C, SMT off, all-core turbo, one pinned core; six paired rounds), for the served requests and for groups of 128 to 1,024 rows (
max_group_width1024): like any group of 65 to 1,024 rows, these takeHandlecarriers and reachlist_avx512:Where the rounds disagree in sign, only their range is given.
cargo test --release --features researchpasses; clippy (workspace, research, library alone) and fmt.