Repository navigation
Reimplement Shamir secret sharing in verified Rust - #8614
Amaury Chamayou (achamayou) wants to merge 22 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
The FFI safety contracts omit initialized-buffer requirements, and the README incorrectly conflates submitter authentication with validation of share contents.
2 open issues
What changed in this PR
Adds an experimental, opt-in Rust backend for CCF recovery-share secret sharing, with Lean proofs over an Aeneas-extracted arithmetic core. The C++ backend remains the default, and the proofs explicitly exclude the foreign-function interface and C++ adapter.
Changes:
- Implements the Rust core, C interface, C++ adapter, and backend selection while preserving share compatibility.
- Adds refinement and safety proofs, reproducible extraction checks, Miri checks, and cross-backend compatibility tests.
- Documents the proof boundary and experimental status, and adds two input guards to the existing C++ implementation.
Custom instructions used
.github/copilot-instructions.md.github/instructions/reviewing.instructions.md
| File | Description |
|---|---|
src/rust/src/shamir.rs |
Adds the checked C interface and interface tests. |
src/rust/src/lib.rs |
Includes the Shamir interface module. |
src/rust/shamir/tests/compatibility.rs |
Tests arithmetic, recovery, and invalid inputs. |
src/rust/shamir/src/lib.rs |
Implements the safe, allocation-free arithmetic core. |
src/rust/shamir/Cargo.toml |
Defines the standalone core crate. |
src/rust/shamir/Cargo.lock |
Records the core crate’s dependency state. |
src/rust/shamir/.gitignore |
Excludes the extraction intermediate. |
src/rust/cbindgen.toml |
Configures C header generation. |
src/rust/Cargo.toml |
Links the core and enables release overflow checks. |
src/rust/Cargo.lock |
Records the new core dependency. |
src/crypto/test/secret_sharing.cpp |
Tests both backends and serialized-share compatibility. |
src/crypto/sharing.h |
Documents supported thresholds and input errors. |
src/crypto/sharing.cpp |
Rejects zero recovery thresholds and excessive share counts. |
src/crypto/sharing_rust.cpp |
Adapts the C++ API to Rust with cleansed scratch buffers. |
src/crypto/sharing_ffi.h |
Provides the generated C interface declarations. |
scripts/lean-format-checks.sh |
Supports Shamir while excluding generated model files. |
scripts/generate-shamir-header.sh |
Generates or checks the C header. |
scripts/extract-shamir.sh |
Extracts the Lean model using pinned tools. |
scripts/check-shamir.sh |
Builds proofs and audits declarations and axioms. |
lean/shamir/Shamir/Properties/Encoding.lean |
Defines the machine-value interpretation used by theorem statements. |
lean/shamir/Shamir/Properties.lean |
States public correctness and safety guarantees. |
lean/shamir/Shamir/Proofs/Witnesses.lean |
Supplies witnesses that theorem premises can hold. |
lean/shamir/Shamir/Proofs/Sharing.lean |
Proves share generation behavior. |
lean/shamir/Shamir/Proofs/Refinement.lean |
Connects the extracted implementation to CSLib. |
lean/shamir/Shamir/Proofs/Recovery.lean |
Proves reconstruction and output preservation on errors. |
lean/shamir/Shamir/Proofs/Outcome.lean |
Proves termination and success conditions for arbitrary inputs. |
lean/shamir/Shamir/Proofs/Inverse.lean |
Proves exponentiation and field inversion. |
lean/shamir/Shamir/Proofs/Interpolation.lean |
Proves interpolation-weight computation. |
lean/shamir/Shamir/Proofs/Evaluation.lean |
Proves Horner polynomial evaluation. |
lean/shamir/Shamir/Proofs/Definitions.lean |
Defines mathematical helpers for proof invariants. |
lean/shamir/Shamir/Proofs/Arithmetic.lean |
Proves reduction and bounded field arithmetic. |
lean/shamir/Shamir/Model/Field.lean |
Defines the sharing field and proves primality. |
lean/shamir/Shamir/Model.lean |
Combines the extracted model with its field interpretation. |
lean/shamir/Shamir.lean |
Collects handwritten model and proof modules. |
lean/shamir/README.md |
Documents compatibility, proof scope, and reproduction steps. |
lean/shamir/ListDeclarations.lean |
Enumerates project declarations for kernel checking. |
lean/shamir/lakefile.toml |
Configures Lean libraries, dependencies, and checks. |
lean/shamir/lake-manifest.json |
Pins Lean dependency revisions. |
lean/shamir/Generated/CcfShamir.lean |
Contains the extracted executable model. |
lean/shamir/Generated.lean |
Exposes the generated model library. |
lean/shamir/.gitignore |
Excludes local Lake artifacts. |
lean/common/README.md |
Documents Shamir’s separate dependency and workflow requirements. |
lean/AGENT.md |
Documents the extraction-based exception to trace validation. |
doc/contribute/build_ccf.rst |
Describes the experimental backend build option. |
doc/architecture/verification.rst |
Adds Shamir to the verification architecture documentation. |
CMakeLists.txt |
Links both backends into compatibility tests. |
cmake/verify_ccf_rs_exports.cmake |
Validates the added Shamir exports. |
cmake/crypto.cmake |
Selects the production and reference backends. |
cmake/ccf_rs.cmake |
Preserves Shamir exports during Rust library integration. |
.github/workflows/README.md |
Documents the dedicated Shamir workflow. |
.github/workflows/lean.yml |
Excludes Shamir from the shared Lean workflow. |
.github/workflows/lean-shamir.yml |
Adds extraction, interface, proof, and consistency checks. |
.github/skills/formatting-and-linting/SKILL.md |
Documents Shamir-specific formatting exclusions and routing. |
.changes/rust-shamir-backend.added.md |
Announces the experimental backend and compatibility intent. |
Review effort: Balanced
There was a problem hiding this comment.
🔵 Needs a closer look
Human review is needed for the security-critical recovery proof/extraction boundary and unverified unsafe C ABI/C++ integration, with Rust-backend network recovery not exercised by CI.
No open issues 🎉
2 resolved since last review
Review effort: Balanced
There was a problem hiding this comment.
🔵 Needs a closer look
The new cryptographic extraction/proof trust boundary and unsafe cross-language bridge require human review of theorem assumptions and production Rust-backend recovery validation, which could not be executed here.
No open issues 🎉
2 resolved since last review
Review effort: Balanced
Replace the C++ Shamir arithmetic in src/crypto/sharing.cpp with a safe, dependency-free Rust core (src/rust/shamir) exposed through a C ABI from ccf-rs, and keep sampling, cleansing and the Share type in the C++ adapter. Extract a Lean model of the Rust core with Aeneas (lean/shamir/Generated) and prove that evaluation and recovery refine CSLib's Shamir share and reconstruct definitions. scripts/extract-shamir.sh reproduces the extraction; scripts/check-shamir.sh runs the kernel and axiom audit over every project declaration and its dependency closure. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Lean proofs show the core arithmetic cannot overflow, so the checks cost nothing on valid executions. Keeping them in release builds turns any violated assumption into an abort instead of a silently wrapped value. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Validate every buffer argument before it is turned into a slice: null, misaligned, empty, oversized, address-wrapping, or mutually overlapping buffers now return status 1 instead of violating slice preconditions. Dangling pointers remain the caller's obligation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the int32_t status codes with a ccf_shamir_status enum shared by the header and a repr(C) Rust enum, so each outcome is named on both sides and the C++ adapter switches over enumerators rather than magic numbers. Mark the entry points nodiscard so a silently ignored failure is a compile error under -Werror. Values and the ABI are unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Drop the derived Copy and Debug implementations from Share so that copies of secret material are visible as explicit clone calls and shares can never be printed through formatting machinery. Only the source-line comments in the extracted Lean model change; the model itself and its proofs are unaffected. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Take the recovered secret as a pointer to a ten-limb array so the length is part of the signature rather than a documented convention, and allocate the Lagrange weights workspace inside the FFI so callers can no longer pass a mis-sized one. Allocation failure is reported as CCF_SHAMIR_OUT_OF_MEMORY, which the adapter maps to std::bad_alloc, matching the previous behaviour of allocating the workspace in C++. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The per-share evaluation loop, the assignment of share coordinates (x = i + 1), and the share-count and threshold bounds checks now live in the safe, Aeneas-extracted ccf-shamir crate as share_limb. The FFI entry point ccf_shamir_share_limb only marshals buffers and status codes, and the C++ adapter no longer computes coordinates or iterates over shares, so the sharing side of the ABI has a single source of truth for the x coordinates and no arithmetic outside the verified core. SharingRefinement is restated for share_limb: filling a limb assigns coordinate k + 1 to share k, stores CSLib's share value for that coordinate in the requested limb, and leaves every other limb unchanged. The new share_limb_loop_spec and share_limb_spec proofs are kernel-checked with no axioms beyond propext, Quot.sound and Classical.choice. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
recover now takes the caller's secret array by mutable reference and fills it only after every Lagrange weight has been computed. The FFI no longer materialises a temporary copy of the recovered secret and the OPENSSL_cleanse foreign call it needed is gone, so the C++ caller's cleansed buffer is the only place the recovered secret ever lives. The Aeneas model threads the array through recover explicitly, which lets the "unchanged on error" contract of the C ABI be proved rather than inspected: recovery_error_preserves_secret shows that for any inputs, a recovery that reports an error returns the caller's array untouched, with no preconditions. RecoveryRefinement is restated for the in-place signature and holds for any initial array contents. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
src/crypto/sharing_ffi.h is now rendered by cbindgen 0.29.4 from the #[repr(C)] types and extern "C" functions in src/rust/src/shamir.rs via scripts/generate-shamir-header.sh, which also formats it with the pinned clang-format. Signatures, status codes, the limb count and the share layout can no longer drift between the two sides: the Lean workflow fails if the committed header differs from the generated one. Impact: - CCF_SHAMIR_LIMBS is defined by the header (from a LIMBS constant in the binding crate, compile-time asserted equal to the verified core's) and used in the share layout and the recover signature, with a C++ static_assert. - Status enumerators are now CCF_SHAMIR_STATUS_* (cbindgen's name-prefixed rendering) and the C++ adapter is updated accordingly. - All three entry points carry #[must_use] on the Rust side, which is what emits CCF_SHAMIR_NODISCARD in the header. - The C contract documentation lives on the Rust declarations and is copied into the header, so there is a single place to keep it accurate. - Re-extracted Lean model: only Source: line comments changed; proofs and the declaration check are unaffected. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The only unsafe code in the Shamir path is the pointer marshalling in src/rust/src/shamir.rs, which the Lean proofs deliberately do not cover. Its tests, including the rejected null, misaligned, overlapping and address-wrapping buffers and the error paths, now run under Miri in the Lean workflow with the default Stacked Borrows model and again with Tree Borrows and strict provenance. Impact: undefined behaviour in the FFI layer (out-of-bounds reads of exactly sized share buffers, aliasing violations, misaligned accesses, integer-to-pointer casts) now fails CI instead of going unnoticed. The current layer is clean under all three configurations; the verified core also passes Miri but is forbid(unsafe_code), so only the FFI crate runs in CI. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Generate narrow clang-tidy exceptions for the C/Rust ABI without changing its declarations. Build the Generated import root before formatting ListDeclarations.lean. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use explicit function addresses so Clang 18 with libstdc++ 13 deduces pointer members rather than invalid function members in the backend pairs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Depend on the common Lean package for its check-imports command. To satisfy its layering rules, state the three theorems in full in Shamir/Properties.lean with one-line links to their proofs, replacing the named propositions that Shamir/Proof.lean proved, and move the coefficient-vector encoding relation to Shamir/Properties/Encoding.lean so that supporting proofs do not import the theorem entrypoint. These changes were made in the merge commit 0cb522b ("Merge latest main into Shamir draft"). A rebase cannot keep a merge commit, so they are carried here. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
USE_RUST_SHAMIR now defaults to OFF and is documented as experimental, pending further review of the Rust backend. With the C++ backend selected, the Rust adapter is only built as the reference backend of sharing_test. Run every sharing and recovery test case against both backends, add a differential test on arbitrary unauthenticated recovery inputs, and keep the reference backend under clang-tidy, so that the Rust adapter keeps its coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Follow lean/AGENT.md: the model, the definitions needed to read the claims, and proof-only definitions now live under Model, Properties and Proofs in matching Shamir.* namespaces, every claim with premises has an adjacent non-vacuity witness, and the package uses the shared toolchain file and the stricter lint options of the other packages. The model is still extracted by Aeneas rather than aligned by trace validation, which AGENT.md now records as the one exception. Strengthen the claims while restating them: - recovery_succeeds_iff and sharing_succeeds_iff show that the extracted functions return for every input, so unauthenticated recovery shares cannot make the core panic through the C ABI, and characterize exactly when they succeed. - recovery_refines_cslib also shows the recovered limbs are canonical. - recovery_returns_secret states reconstruction correctness on the implementation, and reduction_is_remainder covers the exported reduction. The package needs Aeneas, CSLib and Mathlib, so it moves to its own lean-shamir.yml workflow, as mathlib-equivalence does, and is excluded from lean.yml. Enumerating project declarations for the kernel and axiom check now reads them from the module data, which takes seconds rather than minutes and yields the same set. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Lean workflow's final gate checks that every Lean package is either scheduled by its matrix or explicitly checked elsewhere. Record that lean-shamir.yml checks the Shamir package, and document that workflow. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Members are authenticated when they submit a recovery share, with a COSE signature checked against the member ID. What is not validated is the contents of the share, which is why the arithmetic must handle every input. Say so in the README, the Lean docstrings and the differential test, instead of describing the shares as submitted without authentication. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The bridge forms Rust slices and a mutable reference from the caller's pointers, which requires initialized memory as well as live storage, including for the share and secret buffers that are only written. State this in both safety contracts and regenerate the C header. The C++ adapter already value-initializes every buffer it passes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The extraction and verification pipeline for the Rust Shamir backend needs Aeneas, Charon, CSLib, Mathlib, Miri and cbindgen. It already runs in its own workflow so that the CCF build and the main Lean workflow do not pull any of them in. Run both of its jobs on the standard ubuntu-latest runner, installing Lean with elan from apt as the other Lean jobs do, and key the dependency cache on the runner image since ubuntu-latest moves between Ubuntu releases. Also trigger it only for the ccf-shamir core and the ccf-rs crate that exposes its C interface, rather than all of src/rust, so that unrelated Rust changes such as ccf-app do not run the Mathlib pipeline. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1600698 to
74ede7c
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
The cryptographic recovery change crosses formal-proof, unsafe Rust ABI, and C++ integration boundaries that require specialist review, while proof checks, Miri, production linking, and historical recovery compatibility could not be verified here.
No open issues 🎉
Review effort: Balanced
#8763 pinned every Ubuntu job to 26.04 so that CI does not change environment when GitHub moves ubuntu-latest. Do the same for the Shamir workflow, which was the only one left on ubuntu-latest, and key its dependency cache on that release as the other Lean workflows do. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Recovery-key cryptography introduces unsafe Rust/C++ and proof boundaries requiring specialist review of theorem adequacy and trust assumptions, while build and verification checks could not be run.
No open issues 🎉
Review effort: Balanced

Adds an experimental Rust implementation of recovery share secret sharing whose Lean model is extracted from the production Rust core by Aeneas, with machine-checked proofs about that model.
Recovery shares protect the key that wraps the ledger secrets. The member submitting a share is authenticated, but the share's contents are not validated, so this is a small piece of arithmetic where handling every input correctly matters. The proofs show that share generation and recovery compute what CSLib's Shamir scheme specifies, that recovery returns the shared secret, and that neither can panic, overflow or index out of bounds on any input.
The C++ implementation remains the default.
USE_RUST_SHAMIRis off by default and marked experimental, so that the Rust backend can be reviewed further before any switch. Both backends read and write the same recovery shares.What the proofs do and do not cover
The theorems in
lean/shamir/Shamir/Properties.leanare about the functions extracted fromsrc/rust/shamir, and CI checks that the committed model is reproduced exactly from that source.They do not cover the unsafe C ABI bridge in
src/rust/src/shamir.rs(exercised under Miri), the C++ adapter insrc/crypto/sharing_rust.cpp, the entropy source, timing, secret erasure, the Rust compiler or Aeneas's translation. The C++ backend is not covered at all.lean/shamir/README.mdhas the full statement list and proof boundary.Alternatives
lean/AGENT.mdrecords this as the one exception.Safety and compatibility
std::invalid_argument. A zero recovery threshold previously read out of bounds; share issuance already rejects it before reaching this code.-DUSE_RUST_SHAMIR=ON, the share serialization, field, coefficient sampling and key derivation match the C++ backend, and each backend recovers the other's shares, so shares issued by existing services remain recoverable.Lean Shamirworkflow rather than the sharedLeanworkflow.