Skip to content

Reimplement Shamir secret sharing in verified Rust - #8614

Open
Amaury Chamayou (achamayou) wants to merge 22 commits into
mainfrom
achamayou-upstream-shamir-draft
Open

Amaury Chamayou (achamayou) wants to merge 22 commits into
mainfrom
achamayou-upstream-shamir-draft

Conversation

@achamayou

@achamayou Amaury Chamayou (achamayou) commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Adds an experimental Rust implementation of recovery share secret sharing whose Lean model is extracted from the production Rust core by Aeneas, with machine-checked proofs about that model.

Recovery shares protect the key that wraps the ledger secrets. The member submitting a share is authenticated, but the share's contents are not validated, so this is a small piece of arithmetic where handling every input correctly matters. The proofs show that share generation and recovery compute what CSLib's Shamir scheme specifies, that recovery returns the shared secret, and that neither can panic, overflow or index out of bounds on any input.

The C++ implementation remains the default. USE_RUST_SHAMIR is off by default and marked experimental, so that the Rust backend can be reviewed further before any switch. Both backends read and write the same recovery shares.

What the proofs do and do not cover

The theorems in lean/shamir/Shamir/Properties.lean are about the functions extracted from src/rust/shamir, and CI checks that the committed model is reproduced exactly from that source.

They do not cover the unsafe C ABI bridge in src/rust/src/shamir.rs (exercised under Miri), the C++ adapter in src/crypto/sharing_rust.cpp, the entropy source, timing, secret erasure, the Rust compiler or Aeneas's translation. The C++ backend is not covered at all. lean/shamir/README.md has the full statement list and proof boundary.

Alternatives

  • Trace validation of a handwritten model, as the other Lean packages use: rejected because secret sharing is a pure computation, so extracting the model from the Rust source gives guarantees for every input rather than for recorded runs. lean/AGENT.md records this as the one exception.
  • Replacing the C++ backend outright: rejected for now. The C ABI bridge and C++ adapter are outside the proofs, and CI runs end-to-end recovery only against the default backend, so the Rust backend stays opt-in until it has had further review.

Safety and compatibility

  • Default builds change only by two input guards in the C++ backend: recovery with a zero threshold, and sharing among 2^31 - 1 or more shares, now throw std::invalid_argument. A zero recovery threshold previously read out of bounds; share issuance already rejects it before reaching this code.
  • With -DUSE_RUST_SHAMIR=ON, the share serialization, field, coefficient sampling and key derivation match the C++ backend, and each backend recovers the other's shares, so shares issued by existing services remain recoverable.
  • A Rust panic in the C ABI aborts the node. The proofs rule this out for the core on every input, including repeated, zero and noncanonical share coordinates.
  • The Lean package depends on Aeneas, CSLib and Mathlib, unlike the other Lean packages, so it is checked by its own Lean Shamir workflow rather than the shared Lean workflow.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The FFI safety contracts omit initialized-buffer requirements, and the README incorrectly conflates submitter authentication with validation of share contents.

2 open issues
What changed in this PR

Adds an experimental, opt-in Rust backend for CCF recovery-share secret sharing, with Lean proofs over an Aeneas-extracted arithmetic core. The C++ backend remains the default, and the proofs explicitly exclude the foreign-function interface and C++ adapter.

Changes:

  • Implements the Rust core, C interface, C++ adapter, and backend selection while preserving share compatibility.
  • Adds refinement and safety proofs, reproducible extraction checks, Miri checks, and cross-backend compatibility tests.
  • Documents the proof boundary and experimental status, and adds two input guards to the existing C++ implementation.

Custom instructions used

  • .github/copilot-instructions.md
  • .github/instructions/reviewing.instructions.md
File Description
src/​rust/​src/​shamir.rs Adds the checked C interface and interface tests.
src/​rust/​src/​lib.rs Includes the Shamir interface module.
src/​rust/​shamir/​tests/​compatibility.rs Tests arithmetic, recovery, and invalid inputs.
src/​rust/​shamir/​src/​lib.rs Implements the safe, allocation-free arithmetic core.
src/​rust/​shamir/​Cargo.toml Defines the standalone core crate.
src/​rust/​shamir/​Cargo.lock Records the core crate’s dependency state.
src/​rust/​shamir/​.gitignore Excludes the extraction intermediate.
src/​rust/​cbindgen.toml Configures C header generation.
src/​rust/​Cargo.toml Links the core and enables release overflow checks.
src/​rust/​Cargo.lock Records the new core dependency.
src/​crypto/​test/​secret_sharing.cpp Tests both backends and serialized-share compatibility.
src/​crypto/​sharing.h Documents supported thresholds and input errors.
src/​crypto/​sharing.cpp Rejects zero recovery thresholds and excessive share counts.
src/​crypto/​sharing_rust.cpp Adapts the C++ API to Rust with cleansed scratch buffers.
src/​crypto/​sharing_ffi.h Provides the generated C interface declarations.
scripts/​lean-format-checks.sh Supports Shamir while excluding generated model files.
scripts/​generate-shamir-header.sh Generates or checks the C header.
scripts/​extract-shamir.sh Extracts the Lean model using pinned tools.
scripts/​check-shamir.sh Builds proofs and audits declarations and axioms.
lean/​shamir/​Shamir/​Properties/​Encoding.lean Defines the machine-value interpretation used by theorem statements.
lean/​shamir/​Shamir/​Properties.lean States public correctness and safety guarantees.
lean/​shamir/​Shamir/​Proofs/​Witnesses.lean Supplies witnesses that theorem premises can hold.
lean/​shamir/​Shamir/​Proofs/​Sharing.lean Proves share generation behavior.
lean/​shamir/​Shamir/​Proofs/​Refinement.lean Connects the extracted implementation to CSLib.
lean/​shamir/​Shamir/​Proofs/​Recovery.lean Proves reconstruction and output preservation on errors.
lean/​shamir/​Shamir/​Proofs/​Outcome.lean Proves termination and success conditions for arbitrary inputs.
lean/​shamir/​Shamir/​Proofs/​Inverse.lean Proves exponentiation and field inversion.
lean/​shamir/​Shamir/​Proofs/​Interpolation.lean Proves interpolation-weight computation.
lean/​shamir/​Shamir/​Proofs/​Evaluation.lean Proves Horner polynomial evaluation.
lean/​shamir/​Shamir/​Proofs/​Definitions.lean Defines mathematical helpers for proof invariants.
lean/​shamir/​Shamir/​Proofs/​Arithmetic.lean Proves reduction and bounded field arithmetic.
lean/​shamir/​Shamir/​Model/​Field.lean Defines the sharing field and proves primality.
lean/​shamir/​Shamir/​Model.lean Combines the extracted model with its field interpretation.
lean/​shamir/​Shamir.lean Collects handwritten model and proof modules.
lean/​shamir/​README.md Documents compatibility, proof scope, and reproduction steps.
lean/​shamir/​ListDeclarations.lean Enumerates project declarations for kernel checking.
lean/​shamir/​lakefile.toml Configures Lean libraries, dependencies, and checks.
lean/​shamir/​lake-manifest.json Pins Lean dependency revisions.
lean/​shamir/​Generated/​CcfShamir.lean Contains the extracted executable model.
lean/​shamir/​Generated.lean Exposes the generated model library.
lean/​shamir/​.gitignore Excludes local Lake artifacts.
lean/​common/​README.md Documents Shamir’s separate dependency and workflow requirements.
lean/​AGENT.md Documents the extraction-based exception to trace validation.
doc/​contribute/​build_ccf.rst Describes the experimental backend build option.
doc/​architecture/​verification.rst Adds Shamir to the verification architecture documentation.
CMakeLists.txt Links both backends into compatibility tests.
cmake/​verify_ccf_rs_exports.cmake Validates the added Shamir exports.
cmake/​crypto.cmake Selects the production and reference backends.
cmake/​ccf_rs.cmake Preserves Shamir exports during Rust library integration.
.github/​workflows/​README.md Documents the dedicated Shamir workflow.
.github/​workflows/​lean.yml Excludes Shamir from the shared Lean workflow.
.github/​workflows/​lean-shamir.yml Adds extraction, interface, proof, and consistency checks.
.github/​skills/​formatting-and-linting/​SKILL.md Documents Shamir-specific formatting exclusions and routing.
.changes/​rust-shamir-backend.added.md Announces the experimental backend and compatibility intent.

Review effort: Balanced

Comment thread lean/shamir/README.md Resolved
Comment thread src/rust/src/shamir.rs Resolved
Copilot AI balanced review requested due to automatic review settings November 13, 2026 07:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human review is needed for the security-critical recovery proof/extraction boundary and unverified unsafe C ABI/C++ integration, with Rust-backend network recovery not exercised by CI.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 13, 2026 09:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The new cryptographic extraction/proof trust boundary and unsafe cross-language bridge require human review of theorem assumptions and production Rust-backend recovery validation, which could not be executed here.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

Replace the C++ Shamir arithmetic in src/crypto/sharing.cpp with a safe,
dependency-free Rust core (src/rust/shamir) exposed through a C ABI from
ccf-rs, and keep sampling, cleansing and the Share type in the C++ adapter.

Extract a Lean model of the Rust core with Aeneas (lean/shamir/Generated)
and prove that evaluation and recovery refine CSLib's Shamir share and
reconstruct definitions. scripts/extract-shamir.sh reproduces the
extraction; scripts/check-shamir.sh runs the kernel and axiom audit over
every project declaration and its dependency closure.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Lean proofs show the core arithmetic cannot overflow, so the checks
cost nothing on valid executions. Keeping them in release builds turns any
violated assumption into an abort instead of a silently wrapped value.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Validate every buffer argument before it is turned into a slice: null,
misaligned, empty, oversized, address-wrapping, or mutually overlapping
buffers now return status 1 instead of violating slice preconditions.
Dangling pointers remain the caller's obligation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the int32_t status codes with a ccf_shamir_status enum shared by
the header and a repr(C) Rust enum, so each outcome is named on both
sides and the C++ adapter switches over enumerators rather than magic
numbers. Mark the entry points nodiscard so a silently ignored failure
is a compile error under -Werror. Values and the ABI are unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Drop the derived Copy and Debug implementations from Share so that
copies of secret material are visible as explicit clone calls and
shares can never be printed through formatting machinery. Only the
source-line comments in the extracted Lean model change; the model
itself and its proofs are unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Take the recovered secret as a pointer to a ten-limb array so the
length is part of the signature rather than a documented convention,
and allocate the Lagrange weights workspace inside the FFI so callers
can no longer pass a mis-sized one. Allocation failure is reported as
CCF_SHAMIR_OUT_OF_MEMORY, which the adapter maps to std::bad_alloc,
matching the previous behaviour of allocating the workspace in C++.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The per-share evaluation loop, the assignment of share coordinates
(x = i + 1), and the share-count and threshold bounds checks now live in
the safe, Aeneas-extracted ccf-shamir crate as share_limb. The FFI entry
point ccf_shamir_share_limb only marshals buffers and status codes, and
the C++ adapter no longer computes coordinates or iterates over shares,
so the sharing side of the ABI has a single source of truth for the
x coordinates and no arithmetic outside the verified core.

SharingRefinement is restated for share_limb: filling a limb assigns
coordinate k + 1 to share k, stores CSLib's share value for that
coordinate in the requested limb, and leaves every other limb
unchanged. The new share_limb_loop_spec and share_limb_spec proofs are
kernel-checked with no axioms beyond propext, Quot.sound and
Classical.choice.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
recover now takes the caller's secret array by mutable reference and
fills it only after every Lagrange weight has been computed. The FFI no
longer materialises a temporary copy of the recovered secret and the
OPENSSL_cleanse foreign call it needed is gone, so the C++ caller's
cleansed buffer is the only place the recovered secret ever lives.

The Aeneas model threads the array through recover explicitly, which
lets the "unchanged on error" contract of the C ABI be proved rather
than inspected: recovery_error_preserves_secret shows that for any
inputs, a recovery that reports an error returns the caller's array
untouched, with no preconditions. RecoveryRefinement is restated for
the in-place signature and holds for any initial array contents.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
src/crypto/sharing_ffi.h is now rendered by cbindgen 0.29.4 from the
#[repr(C)] types and extern "C" functions in src/rust/src/shamir.rs via
scripts/generate-shamir-header.sh, which also formats it with the pinned
clang-format. Signatures, status codes, the limb count and the share layout
can no longer drift between the two sides: the Lean workflow fails if the
committed header differs from the generated one.

Impact:
- CCF_SHAMIR_LIMBS is defined by the header (from a LIMBS constant in the
  binding crate, compile-time asserted equal to the verified core's) and used
  in the share layout and the recover signature, with a C++ static_assert.
- Status enumerators are now CCF_SHAMIR_STATUS_* (cbindgen's name-prefixed
  rendering) and the C++ adapter is updated accordingly.
- All three entry points carry #[must_use] on the Rust side, which is what
  emits CCF_SHAMIR_NODISCARD in the header.
- The C contract documentation lives on the Rust declarations and is copied
  into the header, so there is a single place to keep it accurate.
- Re-extracted Lean model: only Source: line comments changed; proofs and the
  declaration check are unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The only unsafe code in the Shamir path is the pointer marshalling in
src/rust/src/shamir.rs, which the Lean proofs deliberately do not cover.
Its tests, including the rejected null, misaligned, overlapping and
address-wrapping buffers and the error paths, now run under Miri in the
Lean workflow with the default Stacked Borrows model and again with Tree
Borrows and strict provenance.

Impact: undefined behaviour in the FFI layer (out-of-bounds reads of
exactly sized share buffers, aliasing violations, misaligned accesses,
integer-to-pointer casts) now fails CI instead of going unnoticed. The
current layer is clean under all three configurations; the verified core
also passes Miri but is forbid(unsafe_code), so only the FFI crate runs
in CI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Generate narrow clang-tidy exceptions for the C/Rust ABI without changing its declarations. Build the Generated import root before formatting ListDeclarations.lean.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use explicit function addresses so Clang 18 with libstdc++ 13 deduces pointer members rather than invalid function members in the backend pairs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Depend on the common Lean package for its check-imports command. To
satisfy its layering rules, state the three theorems in full in
Shamir/Properties.lean with one-line links to their proofs, replacing
the named propositions that Shamir/Proof.lean proved, and move the
coefficient-vector encoding relation to Shamir/Properties/Encoding.lean
so that supporting proofs do not import the theorem entrypoint.

These changes were made in the merge commit 0cb522b ("Merge latest
main into Shamir draft"). A rebase cannot keep a merge commit, so they
are carried here.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
USE_RUST_SHAMIR now defaults to OFF and is documented as experimental,
pending further review of the Rust backend.

With the C++ backend selected, the Rust adapter is only built as the
reference backend of sharing_test. Run every sharing and recovery test
case against both backends, add a differential test on arbitrary
unauthenticated recovery inputs, and keep the reference backend under
clang-tidy, so that the Rust adapter keeps its coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Follow lean/AGENT.md: the model, the definitions needed to read the
claims, and proof-only definitions now live under Model, Properties and
Proofs in matching Shamir.* namespaces, every claim with premises has an
adjacent non-vacuity witness, and the package uses the shared toolchain
file and the stricter lint options of the other packages. The model is
still extracted by Aeneas rather than aligned by trace validation, which
AGENT.md now records as the one exception.

Strengthen the claims while restating them:
- recovery_succeeds_iff and sharing_succeeds_iff show that the extracted
  functions return for every input, so unauthenticated recovery shares
  cannot make the core panic through the C ABI, and characterize exactly
  when they succeed.
- recovery_refines_cslib also shows the recovered limbs are canonical.
- recovery_returns_secret states reconstruction correctness on the
  implementation, and reduction_is_remainder covers the exported
  reduction.

The package needs Aeneas, CSLib and Mathlib, so it moves to its own
lean-shamir.yml workflow, as mathlib-equivalence does, and is excluded
from lean.yml. Enumerating project declarations for the kernel and axiom
check now reads them from the module data, which takes seconds rather
than minutes and yields the same set.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Lean workflow's final gate checks that every Lean package is either
scheduled by its matrix or explicitly checked elsewhere. Record that
lean-shamir.yml checks the Shamir package, and document that workflow.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Members are authenticated when they submit a recovery share, with a COSE
signature checked against the member ID. What is not validated is the
contents of the share, which is why the arithmetic must handle every
input. Say so in the README, the Lean docstrings and the differential
test, instead of describing the shares as submitted without
authentication.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The bridge forms Rust slices and a mutable reference from the caller's
pointers, which requires initialized memory as well as live storage,
including for the share and secret buffers that are only written. State
this in both safety contracts and regenerate the C header. The C++
adapter already value-initializes every buffer it passes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The extraction and verification pipeline for the Rust Shamir backend
needs Aeneas, Charon, CSLib, Mathlib, Miri and cbindgen. It already runs
in its own workflow so that the CCF build and the main Lean workflow do
not pull any of them in. Run both of its jobs on the standard
ubuntu-latest runner, installing Lean with elan from apt as the other
Lean jobs do, and key the dependency cache on the runner image since
ubuntu-latest moves between Ubuntu releases.

Also trigger it only for the ccf-shamir core and the ccf-rs crate that
exposes its C interface, rather than all of src/rust, so that unrelated
Rust changes such as ccf-app do not run the Mathlib pipeline.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 16, 2026 11:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The cryptographic recovery change crosses formal-proof, unsafe Rust ABI, and C++ integration boundaries that require specialist review, while proof checks, Miri, production linking, and historical recovery compatibility could not be verified here.

No open issues 🎉

Review effort: Balanced

#8763 pinned every Ubuntu job to 26.04 so that CI does not change
environment when GitHub moves ubuntu-latest. Do the same for the Shamir
workflow, which was the only one left on ubuntu-latest, and key its
dependency cache on that release as the other Lean workflows do.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 16, 2026 12:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Recovery-key cryptography introduces unsafe Rust/C++ and proof boundaries requiring specialist review of theorem adequacy and trust assumptions, while build and verification checks could not be run.

No open issues 🎉

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants