Skip to content

Commit

Permalink
chore: book + tracing (#1932)
Browse files Browse the repository at this point in the history
  • Loading branch information
ratankaliani authored Jan 15, 2025
1 parent 43b82ce commit c6ddc2e
Show file tree
Hide file tree
Showing 10 changed files with 126 additions and 75 deletions.
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,9 @@ examples/fibonacci/fibonacci-plonk.bin

# C++
.vscode/c_cpp_properties.json

**/.yarn
**/yarn.lock
book/.pnp.cjs
book/.pnp.loader.mjs
book/.yarnrc.yml
6 changes: 3 additions & 3 deletions book/docs/developers/common-issues.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
If you are using a library that has an MSRV specified, you may encounter an error like this when building your program.

```txt
package `alloy v0.1.1 cannot be built because it requires rustc 1.76 or newer, while the currently active rustc version is 1.75.0-nightly`
package `alloy cannot be built because it requires rustc 1.83 or newer, while the currently active rustc version is 1.81.0`
```

This is due to the fact that your current Succinct Rust toolchain has been built with a lower version than the MSRV of the crates you are using.
Expand All @@ -18,9 +18,9 @@ You can check the version of your local Succinct Rust toolchain by running `carg
cargo 1.81.0-dev (2dbb1af80 2024-08-20)
```

A Succinct Rust toolchain with version **1.81** should work for all crates that have an MSRV of **1.81** or lower.
A Succinct Rust toolchain with version **1.82** should work for all crates that have an MSRV of **1.82** or lower.

If the MSRV of your crate is higher than **1.81**, try the following:
If the MSRV of your crate is higher than **1.82**, try the following:

- If using `cargo prove build` directly, pass the `--ignore-rust-version` flag:

Expand Down
2 changes: 1 addition & 1 deletion book/docs/verification/off-chain-verification.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import ProgramScript from "@site/static/examples_groth16_script_src_main.rs.mdx"

You can verify SP1 Groth16 and Plonk proofs in `no_std` environments with [`sp1-verifier`](https://docs.rs/sp1-verifier/latest/sp1_verifier/).

`sp1-verifier` is also patched to verify Groth16 and Plonk proofs within the SP1 ZKVM, using
`sp1-verifier` is also patched to verify Groth16 and Plonk proofs within the SP1 zkVM, using
[bn254](https://blog.succinct.xyz/succinctshipsprecompiles/) precompiles. For an example of this, see
the [Groth16 Example](https://github.com/succinctlabs/sp1/tree/main/examples/groth16/).

Expand Down
36 changes: 24 additions & 12 deletions book/docs/writing-programs/compiling.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,21 @@ Once you have written an SP1 program, you must compile it to an ELF file that ca

## Development Builds

> WARNING: This may not generate a reproducible ELF which is necessary for verifying that your binary corresponds to given source code.
> WARNING: Running `cargo prove build` may not generate a reproducible ELF which is necessary for verifying that your binary corresponds to given source code.
>
> Use the [reproducible build system](#production-builds) for production builds.
> Use SP1's [reproducible build system](#production-builds) for production builds.
To build a program while developing, simply run the following command in the crate that contains your SP1 program:

```bash
# Enter the directory containing your SP1 program.
cd path/to/your/program

# Build the program.
cargo prove build
```

This will compile the ELF that can be executed in the zkVM. The output from the command will look something like this:
This will compile the ELF that can be executed in the zkVM. The output from the command will look similar to this:

```bash
[sp1] Compiling version_check v0.9.4
Expand All @@ -34,17 +38,27 @@ Under the hood, this CLI command calls `cargo build` with the `riscv32im-succinc

### Advanced Build Options

You can pass additional arguments to the `cargo prove build` command to customize the build process, like configuring what features are enabled, customizing the output directory and more. To see all available options, run `cargo prove build --help`. Many of these options mirror the options available in the `cargo build` command.
The `cargo prove build` command supports several configuration options to customize the build process for your program:

- `--features`: Enable specific features
- `--output-directory`: Specify a custom output location for the ELF
- `--elf-name`: Set a custom name for the output ELF file
- `--no-default-features`: Disable default features
- `--locked`: Ensure Cargo.lock remains unchanged
- `--packages`: Build only specified packages
- `--binaries`: Build only specified binaries

Run `cargo prove build --help` to see the complete list of options. Some options mirror those available in the standard `cargo build` command.

## Production Builds

For production builds of programs, you can build your program inside a Docker container which will generate a **reproducible ELF** on all platforms. To do so, just use the `--docker` flag and optionally the `--tag` flag with the release version you want to use (defaults to `latest`). For example:
For production builds, use Docker to generate a **reproducible ELF** that will be identical across all platforms. Simply add the `--docker` flag to your build command. You can also specify a release version using `--tag`, otherwise the tag defaults to the latest release. For example:

```bash
cargo prove build --docker --tag v1.0.1
cargo prove build --docker --tag v4.0.0
```

To verify that your build is reproducible, you can compute the SHA-512 hash of the ELF on different platforms and systems with:
To verify that your build is truly reproducible across different platforms and systems, compute the SHA-512 hash of the generated ELF file. The hash should be identical regardless of where you build it:

```bash
$ shasum -a 512 elf/riscv32im-succinct-zkvm-elf
Expand Down Expand Up @@ -82,19 +96,17 @@ The above output was generated by running `RUST_LOG=info cargo run --release -vv

To configure the build process when using the `sp1-build` crate, you can pass a [`BuildArgs`](https://docs.rs/sp1-build/latest/sp1_build/struct.BuildArgs.html) struct to to the [`build_program_with_args`](https://docs.rs/sp1-build/latest/sp1_build/fn.build_program_with_args.html) function. The build arguments are the same as the ones available from the `cargo prove build` command.

As an example, you could use the following code to build the Fibonacci example with the `docker` flag set to `true` and a custom output directory for the generated ELF:
As an example, you could use the following code to build the Fibonacci example with the `docker` flag set to `true` and a custom name for the generated ELF. This will generate a reproducible ELF file (with Docker) with the name `fibonacci-elf`:

```rust
use sp1_build::{build_program_with_args, BuildArgs};

fn main() {
let args = BuildArgs {
docker: true,
output_directory: "./fibonacci-program".to_string(),
elf_name: "fibonacci-elf".to_string(),
..Default::default()
};
build_program_with_args("../program", &args);
}
```

**Note:** If you want reproducible builds with the `build.rs` approach, you should use the `docker` flag and the `build_program_with_args` function, as shown in the example above.
```
90 changes: 53 additions & 37 deletions book/docs/writing-programs/cycle-tracking.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,59 +4,75 @@ import Example from "@site/static/examples_cycle-tracking_program_bin_normal.rs.

When writing a program, it is useful to know how many RISC-V cycles a portion of the program takes to identify potential performance bottlenecks. SP1 provides a way to track the number of cycles spent in a portion of the program.

## Tracking Cycles with Annotations
## Tracking Cycles

To track the number of cycles spent in a portion of the program, you can either put `println!("cycle-tracker-start: block name")` + `println!("cycle-tracker-end: block name")` statements (block name must be same between start and end) around the portion of your program you want to profile or use the `#[sp1_derive::cycle_tracker]` macro on a function. An example is shown below:
### Using Print Annotations
For simple debugging, use these annotations to log cycle counts to stdout:

<Example />
```rust
#![no_main]
sp1_zkvm::entrypoint!(main);

Note that to use the macro, you must add the `sp1-derive` crate to your dependencies for your program.
fn main() {
let mut nums = vec![1, 1];

```toml
[dependencies]
sp1-derive = "4.0.0"
```
// Compute the sum of the numbers.
println!("cycle-tracker-start: compute");
let sum: u64 = nums.iter().sum();
println!("cycle-tracker-end: compute");
}
```

In the script for proof generation, setup the logger with `utils::setup_logger()` and run the script with `RUST_LOG=info cargo run --release`. You should see the following output:
With this code, you will see output like the following in your logs:

```
$ RUST_LOG=info cargo run --release
Finished release [optimized] target(s) in 0.21s
Running `target/release/cycle-tracking-script`
2024-03-13T02:03:40.567500Z INFO execute: loading memory image
2024-03-13T02:03:40.567751Z INFO execute: starting execution
2024-03-13T02:03:40.567760Z INFO execute: clk = 0 pc = 0x2013b8
2024-03-13T02:03:40.567822Z INFO execute: ┌╴setup
2024-03-13T02:03:40.568095Z INFO execute: └╴4,398 cycles
2024-03-13T02:03:40.568122Z INFO execute: ┌╴main-body
2024-03-13T02:03:40.568149Z INFO execute: │ ┌╴expensive_function
2024-03-13T02:03:40.568250Z INFO execute: │ └╴1,368 cycles
stdout: result: 5561
2024-03-13T02:03:40.568373Z INFO execute: │ ┌╴expensive_function
2024-03-13T02:03:40.568470Z INFO execute: │ └╴1,368 cycles
stdout: result: 2940
2024-03-13T02:03:40.568556Z INFO execute: └╴5,766 cycles
2024-03-13T02:03:40.568566Z INFO execute: finished execution clk = 11127 pc = 0x0
2024-03-13T02:03:40.569251Z INFO execute: close time.busy=1.78ms time.idle=21.1µs
[INFO] compute: 1234 cycles
```

Note that we elegantly handle nested cycle tracking, as you can see above.

### Get Tracked Cycle Counts

To include tracked cycle counts in the `ExecutionReport` when using `ProverClient::execute`, use the following annotations:
### Using Report Annotations
To store cycle counts across multiple invocations in the `ExecutionReport`, use the report annotations:

```rust
#![no_main]
sp1_zkvm::entrypoint!(main);

fn main() {
println!("cycle-tracker-report-start: block name");
// ...
println!("cycle-tracker-report-end: block name");
// Track cycles across multiple computations
for i in 0..10 {
println!("cycle-tracker-report-start: compute");
expensive_computation(i);
println!("cycle-tracker-report-end: compute");
}
}

```

This will log the cycle count for `block name` and include it in the `ExecutionReport` in the `cycle_tracker` map.
Access total cycles from all invocations
```rust
let report = client.execute(ELF, &stdin).run().unwrap();
let total_compute_cycles = report.cycle_tracker.get("compute").unwrap();
```

### Using the Cycle Tracker Macro
Add `sp1-derive` to your dependencies:
```toml
sp1-derive = "4.0.0"
```

Then annotate your functions:
```rust
#[sp1_derive::cycle_tracker]
pub fn expensive_function(x: usize) -> usize {
let mut y = 1;
for _ in 0..100 {
y *= x;
y %= 7919;
}
y
}
```

### Profiling a ZKVM program
## Profiling a zkVM program

Profiling a zkVM program produces a useful visualization ([example profile](https://share.firefox.dev/3Om1pzz)) which makes it easy to examine program performance and see exactly where VM cycles are being spent without needing to modify the program at all.

Expand Down
40 changes: 26 additions & 14 deletions book/docs/writing-programs/patched-crates.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,17 +28,27 @@ To use the patched libraries, you can use corresponding patch entries in your pr

```toml
[patch.crates-io]
# SHA2
sha2-v0-9-9 = { git = "https://github.com/sp1-patches/RustCrypto-hashes", package = "sha2", tag = "patch-sha2-0.9.9-sp1-4.0.0" }
sha2-v0-10-6 = { git = "https://github.com/sp1-patches/RustCrypto-hashes", package = "sha2", tag = "patch-sha2-0.10.6-sp1-4.0.0" }
sha2-v0-10-8 = { git = "https://github.com/sp1-patches/RustCrypto-hashes", package = "sha2", tag = "patch-sha2-0.10.8-sp1-4.0.0" }
# SHA3
sha3-v0-10-8 = { git = "https://github.com/sp1-patches/RustCrypto-hashes", package = "sha3", tag = "patch-sha3-0.10.8-sp1-4.0.0" }
# BigInt
crypto-bigint = { git = "https://github.com/sp1-patches/RustCrypto-bigint", tag = "patch-0.5.5-sp1-4.0.0" }
tiny-keccak = { git = "https://github.com/sp1-patches/tiny-keccak", tag = "patch-2.0.2-sp1-4.0.0 }
# Keccak
tiny-keccak = { git = "https://github.com/sp1-patches/tiny-keccak", tag = "patch-2.0.2-sp1-4.0.0" }
# Ed25519
curve25519-dalek = { git = "https://github.com/sp1-patches/curve25519-dalek", tag = "patch-4.1.3-sp1-4.0.0" }
curve25519-dalek-ng = { git = "https://github.com/sp1-patches/curve25519-dalek-ng", tag = "patch-4.1.1-sp1-4.0.0" }
# ECDSA
ecdsa-core = { git = "https://github.com/sp1-patches/signatures", package = "ecdsa", tag = "patch-0.16.9-sp1-4.0.0" }
secp256k1 = { git = "https://github.com/sp1-patches/rust-secp256k1", tag = "patch-0.29.1-sp1-4.0.0" }
# BN254
substrate-bn = { git = "https://github.com/sp1-patches/bn", tag = "patch-0.6.0-sp1-4.0.0" }
# BLS12-381
bls12_381 = { git = "https://github.com/sp1-patches/bls12_381", tag = "patch-0.8.0-sp1-4.0.0", features = ["groups"] }
# RSA
rsa = { git = "https://github.com/sp1-patches/RustCrypto-RSA/", tag = "patch-0.9.6-sp1-4.0.0" }
```

Expand All @@ -47,33 +57,35 @@ repository in the patch section. For example:

```toml
[patch."https://github.com/RustCrypto/hashes"]
sha3 = { git = "https://github.com/sp1-patches/RustCrypto-hashes", package = "sha3", tag = "sha3-v0.10.8-patch-v1" }
sha3 = { git = "https://github.com/sp1-patches/RustCrypto-hashes", package = "sha3", tag = "patch-sha3-0.10.8-sp1-4.0.0" }
```

An example of using patched crates is available in our [Tendermint Example](https://github.com/succinctlabs/sp1/blob/main/examples/tendermint/program/Cargo.toml#L22-L25).
An example of using patched crates is available in [SP1 Blobstream](https://github.com/succinctlabs/sp1-blobstream/blob/89e058052c0b691898c5b56a62a6fa0270b31627/Cargo.toml#L40-L43).

## Ed25519 Acceleration

To accelerate Ed25519 operations, you'll need to patch crates depending on if you're using the `ed25519-consensus` or `ed25519-dalek` library in your program or dependencies.

Generally, `ed25519-consensus` has better performance than `ed25519-dalek` by a factor of 2.
Generally, `ed25519-consensus` has better performance for Ed25519 operations than `ed25519-dalek` by a factor of 2.

### Patches

Apply the following patches based on what crates are in your dependencies.

- `ed25519-consensus`

If using `ed25519-consensus`, you should patch `curve25519-dalek-ng` to accelerate ed25519 operations:

```toml
curve25519-dalek-ng = { git = "https://github.com/sp1-patches/curve25519-dalek-ng", tag = "patch-4.1.1-sp1-4.0.0" }
```

- `ed25519-dalek`

If using `ed25519-dalek` version `2.1`, you can patch it with the following:
If using `ed25519-dalek` version `2.1`, you should patch `curve25519-dalek` to accelerate ed25519 operations:

```toml
curve25519-dalek = { git = "https://github.com/sp1-patches/curve25519-dalek", tag = "curve25519_dalek-v4.1.3-patch-v1" }
curve25519-dalek = { git = "https://github.com/sp1-patches/curve25519-dalek", tag = "patch-4.1.3-sp1-4.0.0" }
```

## Secp256k1 Acceleration
Expand All @@ -89,16 +101,16 @@ Apply the following patches based on what crates are in your dependencies.
- `k256`

```toml
ecdsa-core = { git = "https://github.com/sp1-patches/signatures", package = "ecdsa", tag = "ecdsa-v0.16.9-patch-v1" }
ecdsa-core = { git = "https://github.com/sp1-patches/signatures", package = "ecdsa", tag = "patch-0.16.9-sp1-4.0.0" }
```

Note: The curve operations for `k256` are inside of the `ecdsa-core` crate, so you don't need to patch `k256` itself, and just patching `ecdsa-core` is enough.

- `secp256k1`

```toml
secp256k1 = { git = "https://github.com/sp1-patches/rust-secp256k1", tag = "secp256k1-v0.29.0-patch-v1" }
ecdsa-core = { git = "https://github.com/sp1-patches/signatures", package = "ecdsa", tag = "patch-0.16.9-sp1-4.0.0 }
secp256k1 = { git = "https://github.com/sp1-patches/rust-secp256k1", tag = "patch-0.29.1-sp1-4.0.0" }
ecdsa-core = { git = "https://github.com/sp1-patches/signatures", package = "ecdsa", tag = "patch-0.16.9-sp1-4.0.0" }
```

While `secp256k1` doesnt usually rely on `ecdsa-core` the patched version does, so you must patch it as well.
Expand All @@ -112,7 +124,7 @@ To accelerate BN254 (Also known as BN128 and Alt-BN128), you will need to patch
Apply the patch by adding the following to your list of dependencies:

```rust
substrate-bn = { git = "https://github.com/sp1-patches/bn", tag = "substrate_bn-v0.6.0-patch-v1" }
substrate-bn = { git = "https://github.com/sp1-patches/bn", tag = "patch-0.6.0-sp1-4.0.0" }
```

### Performance Benchmarks for Patched `substrate-bn` in `revm`
Expand All @@ -128,15 +140,15 @@ Note: The operations `run-add`, `run-mul`, and `run-pair` are from the `revm` cr
To accelerate [revm](https://github.com/bluealloy/revm) in SP1 using the BN254 patched crate, replace the `substrate-bn` crate with the patched crate by adding the following to `crates/precompile/Cargo.toml`:

```toml
bn = { git = "https://github.com/sp1-patches/bn", package = "substrate-bn", tag = "substrate_bn-v0.6.0-patch-v1" }
bn = { git = "https://github.com/sp1-patches/bn", package = "substrate-bn", tag = "patch-0.6.0-sp1-4.0.0" }
```

## BLS12-381 Acceleration

To accelerate BLS12-381 operations, you'll need to patch the `bls12_381` crate. Apply the following patch by adding the following to your list of dependencies:

```toml
bls12_381 = { git = "https://github.com/sp1-patches/bls12_381", tag = "bls12_381-v0.8.0-patch-v1" }
bls12_381 = { git = "https://github.com/sp1-patches/bls12_381", tag = "patch-0.8.0-sp1-4.0.0" }
```

This patch significantly improves the performance of BLS12-381 operations, making it essential for applications that rely heavily on these cryptographic primitives.
Expand All @@ -158,15 +170,15 @@ This patch significantly improves the performance of BLS12-381 operations, makin
You can check if the patch was applied by using cargo's tree command to print the dependencies of the crate you patched.

```bash
cargo tree -p sha2@0.9.8
cargo tree -p sha2@0.10.8
```

Next to the package name, it should have a link to the Github repository that you patched with.

Ex.

```text
sha2 v0.9.8 (https://github.com/sp1-patches/RustCrypto-hashes?branch=patch-sha2-v0.9.8#afdbfb09)
sha2 v0.10.8 (https://github.com/sp1-patches/RustCrypto-hashes?tag=patch-sha2-0.10.8-sp1-4.0.0)
├── ...
```

Expand Down
Loading

0 comments on commit c6ddc2e

Please sign in to comment.