From ec4678cdfdb4bce3952ef02733579814faaf18c4 Mon Sep 17 00:00:00 2001 From: Lee Bousfield Date: Wed, 11 Oct 2023 22:56:05 -0600 Subject: [PATCH 1/7] Add documentation on customizing the Orbit STF --- .../how-tos/customize-precompile.mdx | 18 +- .../how-tos/customize-stf.mdx | 213 ++++++++++++++++++ .../how-tos/build-nitro-locally.md | 86 +++---- 3 files changed, 264 insertions(+), 53 deletions(-) create mode 100644 arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx diff --git a/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx b/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx index 5b689a58d..4147dfc01 100644 --- a/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx +++ b/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx @@ -3,7 +3,7 @@ title: "How to customize your Orbit chain's precompiles" sidebar_label: "Customize your chain's precompiles" description: "Learn how (and when) to customize your Orbit chain's precompiles" author: jasonwan -sidebar_position: 2 +sidebar_position: 3 --- import PublicPreviewBannerPartial from '../partials/_orbit-public-preview-banner-partial.md'; @@ -41,13 +41,7 @@ Then, open the corresponding Solidity interface file (`ArbSys.sol`) from the [pr function sayHi() external view returns(string memory); ``` -Next, build Nitro by following steps 3-7 of the instructions in [How to build Nitro locally](/node-running/how-tos/build-nitro-locally). Note that if you've already built the Docker image, you still need run the last step to rebuild. - -Run Nitro with the following command: - -```shell -docker run --rm -it -v /some/local/dir/arbitrum:/home/user/.arbitrum -p 0.0.0.0:8547:8547 -p 0.0.0.0:8548:8548 @latestNitroNodeImage@ --parent-chain.connection.url= --chain.id= --http.api=net,web3,eth,debug --http.corsdomain=* --http.addr=0.0.0.0 --http.vhosts=* -``` +Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx) to build a modified Arbitrum Nitro node docker image and run it. :::info @@ -122,13 +116,7 @@ interface ArbHi { } ``` -Next, build Nitro by following the instructions in [How to build Nitro locally](/node-running/how-tos/build-nitro-locally). Note that if you've already built the Docker image, you still need run the last step to rebuild. - -Run Nitro with the following command: - -```shell -docker run --rm -it -v /some/local/dir/arbitrum:/home/user/.arbitrum -p 0.0.0.0:8547:8547 -p 0.0.0.0:8548:8548 @latestNitroNodeImage@ --parent-chain.connection.url= --chain.id= --http.api=net,web3,eth,debug --http.corsdomain=* --http.addr=0.0.0.0 --http.vhosts=* -``` +Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx) to build a modified Arbitrum Nitro node docker image and run it. :::info diff --git a/arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx b/arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx new file mode 100644 index 000000000..8c0fffef7 --- /dev/null +++ b/arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx @@ -0,0 +1,213 @@ +--- +title: "How to customize your Orbit chain's behavior" +sidebar_label: "Customize your chain's behavior" +description: "Learn how to customize your Orbit chain's behavior (known as the State Transition Function or STF)" +author: plasmapower +sidebar_position: 2 +--- + +import PublicPreviewBannerPartial from '../partials/_orbit-public-preview-banner-partial.md'; + + + +# Preface + +Before customizing your Orbit chain, it's important to understand what the State Transition Function (aka the STF) is. +The STF defines how new blocks are produced from input messages (i.e. transactions). +This guide is only necessary for changes that modify the State Transition Function. +To customize other node behavior, such as RPC behavior or the sequencer's ordering policy, you can simply +[build your own node](/node-running/how-tos/build-nitro-locally) without worrying about the rest of this guide. +However, changes that modify the STF require updating the fraud proving system to recognize the new behavior as correct. +Otherwise, the fraud prover would side with unmodified nodes, which would win fraud proofs against your modified node. + +Here's some examples of modifications that affect the STF: +- Adding a new EVM opcode or precompile: + This modifies the STF because a node with this change would disagree about the outcome of EVM execution compared to an unmodified nitro node + when the new opcode or precompile is invoked. +- Rewarding the deployer of a smart contract with a portion of gas spent in the smart contract's execution: + This modifies the STF because a node which has this change applied would disagree about the balance of the deployer after transactions. Such changes would lead to disagreements about block hashes compared to unmodified Nitro nodes. + +Here's some examples of modifications that don't affect the STF: +- Adding a new RPC method to query an address's balance across multiple blocks: + This doesn't modify the STF because this doesn't change on-chain balances or block hashes. +- Changing the sequencer to order blocks by tip: + The sequencer is trusted to order transactions in Arbitrum Nitro, and it can chose any ordering it wants. + Nodes (and the fraud proofs) will simply accept the new transaction ordering as there is no single ordering they think is correct. + +## Modification Compatibility with Arbitrum Nitro + +Some potential modifications are incompatible with Arbitrum Nitro and would not result in a functioning blockchain. +Here are some requirements for the Arbitrum Nitro State Transition Function: +- The STF must be deterministic. For instance, if you gave an address a random balance using the Go randomness library, + every node would disagree on the correct amount of balance and the blockchain would not function correctly. + However, it is acceptable to take a non-deterministic path to a deterministic output. + For instance, if you randomly shuffled a list of addresses, and then gave them each 1 Ether, that would be fine, + because no matter how the list of addresses is shuffled the result is the same and all addresses are given 1 Ether. +- The STF must not reach a new result for old blocks. For instance, if you have been running an Arbitrum Nitro chain for a while, + and then you decide to modify the STF to not charge for gas, a new node that syncs the blockchain will reach a different result + for historical blocks. It's also important to synchronize between nodes when an upgrade takes effect. A common mechanism + for doing this is having an upgrade take effect at a certain timestamp, by which all nodes must be upgraded. +- The STF must be "pure" and not use external resources. For instance, it must not use the filesystem, make external network calls, + or launch processes. That's because the fraud proving system does not (and for the most part, cannot) support these resources. + For instance, it's impossible to fraud prove what the result of an external network call is, because the fraud prover smart contracts on L1 are unable to do networking. +- The STF must not carry state between blocks outside of the "global state". In practice this means persistent state must be stored within + block headers or the Ethereum state trie. For instance, ArbOS stores all retryables in contract storage under a special ArbOS address. +- The STF must not modify Ethereum state outside of a transaction. This is important to ensure that replaying old blocks reaches the same result, + both for tracing and for validation. The ArbOS internal transaction is useful to modify state at the start of blocks. +- The STF must reach a result in under a second. This is a rough rule, but for nodes to keep in sync it's highly recommended to keep blocks quick. + It's also important for the fraud proofs that execution reliably finishes in a relatively short amount of time. + A block gas limit of 32 million gas should safely fit within this limit. +- The STF must not fail or panic. It's important that the STF always produces a new block, even if user input is malformed. + For instance, if the STF receives an invalid transaction as input, it'll still produce an empty block. + +# Building the Modified Node + +To modify the State Transition Function, you'll need to build a modified Arbitrum Nitro node docker image. +This guide covers how to build the node and enable fraud proofs by building a new replay binary. + +## Prerequisites + +Clone the Nitro repository before you begin: + +```shell +git clone --branch @nitroVersionTag@ +cd nitro +git submodule update --init --recursive --force +``` + +## Apply Modifications + +Next, make your changes to the State Transition Function. For example, you could [add a custom precompile](./customize-precompile). + +## Running the Node Without Fraud Proofs + +To build the Arbitrum Nitro node image, you'll first need to install Docker. +You can confirm if it's already setup by running `docker version` in a terminal. +If not, try following [Docker's getting started guide](https://www.docker.com/get-started/), or if you're on Linux, +install Docker from your distribution's package manager and start the docker service. + +Once you have Docker installed, you can simply run `docker build . --tag custom-nitro-node` in the `nitro` folder to build your custom node. + +Once you've built your new Nitro node image, you can run it with the following command. The last argument `--node.staker.dangerous.without-block-validator` disables fraud proof verification: + +```shell +docker run --rm -it -v /some/local/dir/arbitrum:/home/user/.arbitrum -p 0.0.0.0:8547:8547 -p 0.0.0.0:8548:8548 custom-nitro-node --parent-chain.connection.url= --chain.id= --http.api=net,web3,eth,debug --http.corsdomain=* --http.addr=0.0.0.0 --http.vhosts=* --node.staker.dangerous.without-block-validator +``` + +:::info + +Note that the instructions provided in [How to run a full node](/node-running/how-tos/running-a-full-node) **will not** work with your Orbit node. See [Command-line options (Orbit)](/launch-orbit-chain/reference/command-line-options) for Orbit-specific CLI flags. + +::: + +Once your node is running, you can try out your modifications to the State Transition Function and confirm they work as expected. + +## Enabling Fraud Proofs + +To enable fraud proofs, you'll need to build the "replay binary", which defines the State Transition Function for the fraud prover. +The replay binary (sometimes called the machine) re-executes the State Transition Function against input messages to determine the correct output block. +It has three forms: +- The `replay.wasm` binary is the Go replay binary compiled to WASM. It's used by the JIT validator to verify blocks against the fraud prover. +- The `machine.wavm.br` binary is a compressed binary containing the Go replay binary and all its dependencies, compiled to WASM, then translated to the Arbitrum fraud proving variant WAVM. + It's used by Arbitrator when actually entering a challenge and performing the fraud proofs, + and has identical behavior to `replay.wasm`. +- The WASM module root (stored in `module-root.txt`) is a 32 byte hash usually expressed in hexadecimal which is a merkelization of `machine.wavm.br`. + The replay binary is much too large to post on-chain, so this hash is set in the L1 rollup contract to determine the correct replay binary during fraud proofs. + +To run a validator node with fraud proofs enabled, the validator node's Docker image will need to contain all three of these versions of the replay binary. + +### Building a Dev Image + +The simplest way to build a Docker image with the new replay binary is to build a dev image. +These images contain a freshly built replay binary, but note that the replay binary and corresponding WASM module root will generally change when the code is updated, +even if the State Transition Function has equivalent behavior. +It's important that the validator's WASM module root matches the on-chain WASM module root, which is why this approach is harder to maintain. +Over the longer term, you'll want to maintain a separate build of the replay binary that matches the one currently on-chain, usable by any node image. + +To build the dev node image and get the WASM module root, run: + +```shell +docker build . --target nitro-node-dev --tag custom-nitro-node-dev +docker run --rm --entrypoint cat custom-nitro-node-dev target/machines/latest/module-root.txt +``` + +Once you have the WASM module root, you can put it on-chain by calling `setWasmModuleRoot(newWasmModuleRoot)` on the rollup contract as the owner. +The rollup contract address can be found in the chain deployment info JSON. +You can confirm that the WASM module root was updated by calling `wasmModuleRoot()` on the rollup contract. + +Once you have set the new WASM module root on-chain, you may then run the new node image with: + +```shell +docker run --rm -it -v /some/local/dir/arbitrum:/home/user/.arbitrum -p 0.0.0.0:8547:8547 -p 0.0.0.0:8548:8548 custom-nitro-node-dev --parent-chain.connection.url= --chain.id= --http.api=net,web3,eth,debug --http.corsdomain=* --http.addr=0.0.0.0 --http.vhosts=* +``` + +Note that `--node.staker.dangerous.without-block-validator` has been removed from this invocation now that fraud proofs are working again. + +### Preserving the Replay Binary + +The primary issue with simply using a nitro-node-dev build is that, whenever the code changes at all, the replay binary will also change. + +If the node is missing the replay binary corresponding to the on-chain WASM module root, it will be unable to act as a validator. +Therefore, when releasing new node Docker images it's important to include the currently on-chain WASM module root. + +To do that, you'll need to first extract the replay binary from the `nitro-node-dev` Docker image built earlier: + +```shell +docker run --rm --name replay-binary-extractor --entrypoint sleep custom-nitro-node-dev infinity +docker cp replay-binary-extractor:/home/user/target/machines/latest extracted-replay-binary +docker stop replay-binary-extractor +cat extracted-replay-binary/module.root +mv extracted-replay-binary "target/machines/$(cat extracted-replay-binary/module.root)" +``` + +These commands will output the new WASM module root, and create the directory `target/machines/`. +There you'll find the three versions of the replay binary mentioned earlier: +`replay.wasm`, `machine.wavm.br`, and `module-root.txt`, along with some other optional files. +Now that you've extracted the replay binary, there are two ways to add it to future Docker images, +including non-dev image builds. You can either keep it locally and copy it in, or host it on the web. + +#### Option 1: Store the extracted replay binary locally + +Now that we've extracted the replay binary, we can modify the Dockerfile to copy it into new Docker builds. +Edit the `Dockerfile` file in the root of the nitro folder, and after all the `RUN ./download-machines.sh ...` lines, add: + +```dockerfile +COPY target/machines/ +RUN ln -sfT latest +``` + +Replace each `` with the WASM module root you got earlier. + +#### Option 2: Host the replay binary on the web + +To support building the Docker image on other computers without this local machine directory, +you'll need to either commit the machine to git, or preferably, host the replay binary on the web. + +To host the replay binary on the web, you'll need to host the `replay.wasm` and `machine.wavm.br` files somewhere. +One good option is GitHub releases, but any hosting service works. + +Once you have those two files hosted, instead of the `COPY` and `RUN` command mentioned in option 1, +you'll need to add these new lines to the `Dockerfile` file in the root of the nitro folder, +after all the `RUN ./download-machines.sh ...` lines: + +```dockerfile +RUN wasm_module_root="" && \ + mkdir "$wasm_module_root" && \ + wget -O "$wasm_module_root/replay.wasm" && \ + wget -O "$wasm_module_root/machine.wavm.br" && \ + echo "$wasm_module_root" > "$wasm_module_root/module-root.txt" && \ + ln -sfT "$wasm_module_root" latest +``` + +Replace the `` with the WASM module root you got earlier, +the `` with the direct link to the `replay.wasm` file (it must be a direct link to the file and not just a download site), +and the `` with the direct link to the `machine.wavm.br` file. + +### Verifying the Fraud Proofs + +In theory, fraud proofs should now be working with your newly built Docker images. +Make some transactions on your new blockchain, test out your modifications to the State Transition Function, +wait for a batch to be posted, and you should be seeing "validation succeeded" log lines! + +If you see "Error during validation", then the replay binary is likely not up-to-date with your modifications to the State Transition Function. +Ensure that the replay binary is freshly built and is not missing any modifications, and that the WASM module root set in the rollup contract matches your replay binary. diff --git a/arbitrum-docs/node-running/how-tos/build-nitro-locally.md b/arbitrum-docs/node-running/how-tos/build-nitro-locally.md index da6bfd6ae..1dbeb573f 100644 --- a/arbitrum-docs/node-running/how-tos/build-nitro-locally.md +++ b/arbitrum-docs/node-running/how-tos/build-nitro-locally.md @@ -16,7 +16,52 @@ This how-to assumes that you're running one of the following operating systems: - [Ubuntu 22.04 (amd64)](https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-desktop-amd64.iso) - [MacOS Ventura 13.4](https://developer.apple.com/documentation/macos-release-notes/macos-13_4-release-notes). -### 1. Configure prerequisites +### 1. Configure [Docker](https://docs.docker.com/engine/install) + +#### For [Debian](https://docs.docker.com/engine/install/debian)/[Ubuntu](https://docs.docker.com/engine/install/ubuntu) + +```bash +for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove $pkg; done +# Add Docker's official GPG key: +sudo apt-get update +sudo apt-get install ca-certificates curl gnupg +sudo install -m 0755 -d /etc/apt/keyrings +curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg +sudo chmod a+r /etc/apt/keyrings/docker.gpg + +# Add the repository to Apt sources: +echo \ + "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \ + "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \ + sudo tee /etc/apt/sources.list.d/docker.list > /dev/null +sudo apt-get update +sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin +sudo service docker start +``` + +#### For [MacOS](https://docs.docker.com/desktop/install/mac-install/) + +Depending on whether your Mac has an Intel processor or Apple silicon, download the corresponding disk image from [Docker](https://docs.docker.com/desktop/install/mac-install/), and move it into your Applications folder. + +### 2. Download the Nitro source code + +```bash +git clone https://github.com/OffchainLabs/nitro.git +cd nitro +git submodule update --init --recursive --force +``` + +### Intermission: Building a Nitro node Docker image + +If you want to build a Docker image for the Arbitrum Nitro node, +now that you have Docker setup and the Nitro source code checked out, +all you need to do is run `docker build . --tag nitro-node` in the `nitro` folder. +That command will build a Docker image called `nitro-node` from the local source. + +The rest of the guide covers building the node binary natively, outside of Docker. +However, Docker is still used to help build some WebAssembly components. + +### 3. Configure prerequisites #### For Debian/Ubuntu @@ -46,15 +91,7 @@ sudo mkdir -p /usr/local/bin sudo ln -s /opt/homebrew/opt/llvm/bin/wasm-ld /usr/local/bin/wasm-ld ``` -### 2. Configure Nitro - -```bash -git clone https://github.com/OffchainLabs/nitro.git -cd nitro -git submodule update --init --recursive --force -``` - -### 3. Configure Node [16.19](https://github.com/nvm-sh/nvm) +### 4. Configure Node [16.19](https://github.com/nvm-sh/nvm) #### For Debian/Ubuntu @@ -75,7 +112,7 @@ nvm install 16.19 nvm use 16.19 ``` -### 4. Configure Rust [1.72.1](https://www.rust-lang.org/tools/install) +### 5. Configure Rust [1.72.1](https://www.rust-lang.org/tools/install) ```bash curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh @@ -87,33 +124,6 @@ rustup target add wasm32-wasi --toolchain 1.72.1 cargo install cbindgen ``` -### 5. Configure [Docker](https://docs.docker.com/engine/install) - -#### For [Debian](https://docs.docker.com/engine/install/debian)/[Ubuntu](https://docs.docker.com/engine/install/ubuntu) - -```bash -for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove $pkg; done -# Add Docker's official GPG key: -sudo apt-get update -sudo apt-get install ca-certificates curl gnupg -sudo install -m 0755 -d /etc/apt/keyrings -curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg -sudo chmod a+r /etc/apt/keyrings/docker.gpg - -# Add the repository to Apt sources: -echo \ - "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \ - "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \ - sudo tee /etc/apt/sources.list.d/docker.list > /dev/null -sudo apt-get update -sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -sudo service docker start -``` - -#### For [MacOS](https://docs.docker.com/desktop/install/mac-install/) - -Depending on whether your Mac has an Intel processor or Apple silicon, download the corresponding disk image from [Docker](https://docs.docker.com/desktop/install/mac-install/), and move it into your Applications folder. - ### 6. Configure Go [1.20](https://github.com/moovweb/gvm) #### Install Bison From 2341c548c25326b580ff9ea3b99838b2cc840f93 Mon Sep 17 00:00:00 2001 From: Lee Bousfield Date: Thu, 12 Oct 2023 11:27:44 -0600 Subject: [PATCH 2/7] Fix formatting --- arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx b/arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx index 8c0fffef7..ebafab11c 100644 --- a/arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx +++ b/arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx @@ -21,6 +21,7 @@ However, changes that modify the STF require updating the fraud proving system t Otherwise, the fraud prover would side with unmodified nodes, which would win fraud proofs against your modified node. Here's some examples of modifications that affect the STF: + - Adding a new EVM opcode or precompile: This modifies the STF because a node with this change would disagree about the outcome of EVM execution compared to an unmodified nitro node when the new opcode or precompile is invoked. @@ -28,6 +29,7 @@ Here's some examples of modifications that affect the STF: This modifies the STF because a node which has this change applied would disagree about the balance of the deployer after transactions. Such changes would lead to disagreements about block hashes compared to unmodified Nitro nodes. Here's some examples of modifications that don't affect the STF: + - Adding a new RPC method to query an address's balance across multiple blocks: This doesn't modify the STF because this doesn't change on-chain balances or block hashes. - Changing the sequencer to order blocks by tip: @@ -38,6 +40,7 @@ Here's some examples of modifications that don't affect the STF: Some potential modifications are incompatible with Arbitrum Nitro and would not result in a functioning blockchain. Here are some requirements for the Arbitrum Nitro State Transition Function: + - The STF must be deterministic. For instance, if you gave an address a random balance using the Go randomness library, every node would disagree on the correct amount of balance and the blockchain would not function correctly. However, it is acceptable to take a non-deterministic path to a deterministic output. @@ -107,6 +110,7 @@ Once your node is running, you can try out your modifications to the State Trans To enable fraud proofs, you'll need to build the "replay binary", which defines the State Transition Function for the fraud prover. The replay binary (sometimes called the machine) re-executes the State Transition Function against input messages to determine the correct output block. It has three forms: + - The `replay.wasm` binary is the Go replay binary compiled to WASM. It's used by the JIT validator to verify blocks against the fraud prover. - The `machine.wavm.br` binary is a compressed binary containing the Go replay binary and all its dependencies, compiled to WASM, then translated to the Arbitrum fraud proving variant WAVM. It's used by Arbitrator when actually entering a challenge and performing the fraud proofs, From 5e5cd1938a23da28cfef550aa022dd54c84d4104 Mon Sep 17 00:00:00 2001 From: TucksonDev Date: Fri, 13 Oct 2023 12:43:11 +0100 Subject: [PATCH 3/7] Structural changes --- .../how-tos/customize-precompile.mdx | 15 +++-- .../how-tos/customize-stf.mdx | 35 +++++------ .../how-tos/build-nitro-locally.md | 60 ++++++++++++++----- 3 files changed, 71 insertions(+), 39 deletions(-) diff --git a/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx b/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx index 3a7d89d87..0825ca3b5 100644 --- a/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx +++ b/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx @@ -3,7 +3,8 @@ title: "How to customize your Orbit chain's precompiles" sidebar_label: "Customize your chain's precompiles" description: "Learn how (and when) to customize your Orbit chain's precompiles" author: jasonwan -sidebar_position: 3 +sidebar_position: 4 +content-type: how-to --- import PublicPreviewBannerPartial from '../partials/_orbit-public-preview-banner-partial.md'; @@ -14,9 +15,7 @@ import PublicPreviewBannerPartial from '../partials/_orbit-public-preview-banner The guidance in this document will work only if you use `eth_call` to call the new precompiles. If you're calling from other contracts or adding non-view/pure methods, this approach will break block validation. -To support these additional use-cases, stay tuned for instructions that walk you through the process of updating the Wasm module root. - -If you want to test modifying the State Transition System (STS) before this guidance is available, use the `--node.staker.dangerous.without-block-validator` config flag when you start your node. +To support these additional use-cases, follow the instructions described in [How to customize your Orbit chain's behavior](/launch-orbit-chain/how-tos/customize-stf.mdx). ::: @@ -51,11 +50,11 @@ Then, open the corresponding Solidity interface file (`ArbSys.sol`) from the [pr function sayHi() external view returns(string memory); ``` -Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx) to build a modified Arbitrum Nitro node docker image and run it. +Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#building-a-dev-image) to build a modified Arbitrum Nitro node docker image and run it. :::info -Note that the instructions provided in [How to run a full node](/node-running/how-tos/running-a-full-node) **will not** work with your Orbit node. See [Command-line options (Orbit)](/launch-orbit-chain/reference/command-line-options) for Orbit-specific CLI flags. +Note that the instructions provided in [How to run a full node](/node-running/how-tos/running-a-full-node.mdx) **will not** work with your Orbit node. See [Command-line options (Orbit)](/launch-orbit-chain/reference/command-line-options.md) for Orbit-specific CLI flags. ::: @@ -126,11 +125,11 @@ interface ArbHi { } ``` -Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx) to build a modified Arbitrum Nitro node docker image and run it. +Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#building-a-dev-image) to build a modified Arbitrum Nitro node docker image and run it. :::info -Note that the instructions provided in [How to run a full node](/node-running/how-tos/running-a-full-node) **will not** work with your Orbit node. See [Command-line options (Orbit)](/launch-orbit-chain/reference/command-line-options) for Orbit-specific CLI flags. +Note that the instructions provided in [How to run a full node](/node-running/how-tos/running-a-full-node.mdx) **will not** work with your Orbit node. See [Command-line options (Orbit)](/launch-orbit-chain/reference/command-line-options.md) for Orbit-specific CLI flags. ::: diff --git a/arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx b/arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx index ebafab11c..020d3e3dc 100644 --- a/arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx +++ b/arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx @@ -3,20 +3,21 @@ title: "How to customize your Orbit chain's behavior" sidebar_label: "Customize your chain's behavior" description: "Learn how to customize your Orbit chain's behavior (known as the State Transition Function or STF)" author: plasmapower -sidebar_position: 2 +sidebar_position: 3 +content-type: how-to --- import PublicPreviewBannerPartial from '../partials/_orbit-public-preview-banner-partial.md'; -# Preface +## Preface Before customizing your Orbit chain, it's important to understand what the State Transition Function (aka the STF) is. The STF defines how new blocks are produced from input messages (i.e. transactions). This guide is only necessary for changes that modify the State Transition Function. To customize other node behavior, such as RPC behavior or the sequencer's ordering policy, you can simply -[build your own node](/node-running/how-tos/build-nitro-locally) without worrying about the rest of this guide. +[build your own node](/node-running/how-tos/build-nitro-locally.md) without worrying about the rest of this guide. However, changes that modify the STF require updating the fraud proving system to recognize the new behavior as correct. Otherwise, the fraud prover would side with unmodified nodes, which would win fraud proofs against your modified node. @@ -36,7 +37,7 @@ Here's some examples of modifications that don't affect the STF: The sequencer is trusted to order transactions in Arbitrum Nitro, and it can chose any ordering it wants. Nodes (and the fraud proofs) will simply accept the new transaction ordering as there is no single ordering they think is correct. -## Modification Compatibility with Arbitrum Nitro +### Modification compatibility with Arbitrum Nitro Some potential modifications are incompatible with Arbitrum Nitro and would not result in a functioning blockchain. Here are some requirements for the Arbitrum Nitro State Transition Function: @@ -63,26 +64,26 @@ Here are some requirements for the Arbitrum Nitro State Transition Function: - The STF must not fail or panic. It's important that the STF always produces a new block, even if user input is malformed. For instance, if the STF receives an invalid transaction as input, it'll still produce an empty block. -# Building the Modified Node +## Building the modified node To modify the State Transition Function, you'll need to build a modified Arbitrum Nitro node docker image. This guide covers how to build the node and enable fraud proofs by building a new replay binary. -## Prerequisites +### Step 1. Download the Nitro source code Clone the Nitro repository before you begin: ```shell -git clone --branch @nitroVersionTag@ +git clone --branch @nitroVersionTag@ https://github.com/OffchainLabs/nitro.git cd nitro git submodule update --init --recursive --force ``` -## Apply Modifications +### Step 2. Apply modifications -Next, make your changes to the State Transition Function. For example, you could [add a custom precompile](./customize-precompile). +Next, make your changes to the State Transition Function. For example, you could [add a custom precompile](./customize-precompile.mdx). -## Running the Node Without Fraud Proofs +### Step 3. Run the node without fraud proofs To build the Arbitrum Nitro node image, you'll first need to install Docker. You can confirm if it's already setup by running `docker version` in a terminal. @@ -99,13 +100,13 @@ docker run --rm -it -v /some/local/dir/arbitrum:/home/user/.arbitrum -p 0.0.0.0: :::info -Note that the instructions provided in [How to run a full node](/node-running/how-tos/running-a-full-node) **will not** work with your Orbit node. See [Command-line options (Orbit)](/launch-orbit-chain/reference/command-line-options) for Orbit-specific CLI flags. +Note that the instructions provided in [How to run a full node](/node-running/how-tos/running-a-full-node.mdx) **will not** work with your Orbit node. See [Command-line options (Orbit)](/launch-orbit-chain/reference/command-line-options.md) for Orbit-specific CLI flags. ::: Once your node is running, you can try out your modifications to the State Transition Function and confirm they work as expected. -## Enabling Fraud Proofs +### Step 4. Enable fraud proofs To enable fraud proofs, you'll need to build the "replay binary", which defines the State Transition Function for the fraud prover. The replay binary (sometimes called the machine) re-executes the State Transition Function against input messages to determine the correct output block. @@ -120,7 +121,7 @@ It has three forms: To run a validator node with fraud proofs enabled, the validator node's Docker image will need to contain all three of these versions of the replay binary. -### Building a Dev Image +#### 4.1 Build a dev image The simplest way to build a Docker image with the new replay binary is to build a dev image. These images contain a freshly built replay binary, but note that the replay binary and corresponding WASM module root will generally change when the code is updated, @@ -147,7 +148,7 @@ docker run --rm -it -v /some/local/dir/arbitrum:/home/user/.arbitrum -p 0.0.0.0: Note that `--node.staker.dangerous.without-block-validator` has been removed from this invocation now that fraud proofs are working again. -### Preserving the Replay Binary +#### 4.2 Preserving the Replay Binary The primary issue with simply using a nitro-node-dev build is that, whenever the code changes at all, the replay binary will also change. @@ -170,7 +171,7 @@ There you'll find the three versions of the replay binary mentioned earlier: Now that you've extracted the replay binary, there are two ways to add it to future Docker images, including non-dev image builds. You can either keep it locally and copy it in, or host it on the web. -#### Option 1: Store the extracted replay binary locally +##### Option 1: Store the extracted replay binary locally Now that we've extracted the replay binary, we can modify the Dockerfile to copy it into new Docker builds. Edit the `Dockerfile` file in the root of the nitro folder, and after all the `RUN ./download-machines.sh ...` lines, add: @@ -182,7 +183,7 @@ RUN ln -sfT latest Replace each `` with the WASM module root you got earlier. -#### Option 2: Host the replay binary on the web +##### Option 2: Host the replay binary on the web To support building the Docker image on other computers without this local machine directory, you'll need to either commit the machine to git, or preferably, host the replay binary on the web. @@ -207,7 +208,7 @@ Replace the `` with the WASM module root you got earlier, the `` with the direct link to the `replay.wasm` file (it must be a direct link to the file and not just a download site), and the `` with the direct link to the `machine.wavm.br` file. -### Verifying the Fraud Proofs +### Step 5. Verify the fraud proofs In theory, fraud proofs should now be working with your newly built Docker images. Make some transactions on your new blockchain, test out your modifications to the State Transition Function, diff --git a/arbitrum-docs/node-running/how-tos/build-nitro-locally.md b/arbitrum-docs/node-running/how-tos/build-nitro-locally.md index 8ded69df2..614bf7343 100644 --- a/arbitrum-docs/node-running/how-tos/build-nitro-locally.md +++ b/arbitrum-docs/node-running/how-tos/build-nitro-locally.md @@ -11,13 +11,17 @@ import PublicPreviewBannerPartial from '../../partials/_public-preview-banner-pa +Arbitrum Nitro is the software that powers all Arbitrum chains. This how-to shows how you can build a docker image, or binaries, directly from Nitro's source code. If you want to run a node for one of the Arbitrum chains, however, it is recommended that you use the docker image available on DockerHub, as explained in [How to run a full node](/node-running/how-tos/running-a-full-node.mdx). + This how-to assumes that you're running one of the following operating systems: - [Debian 11.7 (arm64)](https://cdimage.debian.org/cdimage/archive/11.7.0/arm64/iso-cd/debian-11.7.0-arm64-netinst.iso) - [Ubuntu 22.04 (amd64)](https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-desktop-amd64.iso) - [MacOS Ventura 13.4](https://developer.apple.com/documentation/macos-release-notes/macos-13_4-release-notes). -### 1. Configure [Docker](https://docs.docker.com/engine/install) +## Build a Docker image + +### Step 1. Configure [Docker](https://docs.docker.com/engine/install) #### For [Debian](https://docs.docker.com/engine/install/debian)/[Ubuntu](https://docs.docker.com/engine/install/ubuntu) @@ -44,25 +48,39 @@ sudo service docker start Depending on whether your Mac has an Intel processor or Apple silicon, download the corresponding disk image from [Docker](https://docs.docker.com/desktop/install/mac-install/), and move it into your Applications folder. -### 2. Download the Nitro source code +#### [Optional] Run docker from a different user + +After installing docker, you might want to be able to run it with your current user instead of root. You can run the following commands to do so. ```bash -git clone https://github.com/OffchainLabs/nitro.git +sudo groupadd docker +sudo usermod -aG docker $USER +newgrp docker +``` + +For troubleshooting, check Docker's section in [their documentation](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user) + +### Step 2. Download the Nitro source code + +```bash +git clone --branch @nitroVersionTag@ https://github.com/OffchainLabs/nitro.git cd nitro git submodule update --init --recursive --force ``` -### Intermission: Building a Nitro node Docker image +### Step 3. Build the Nitro node docker image + +```bash +docker build . --tag nitro-node +``` -If you want to build a Docker image for the Arbitrum Nitro node, -now that you have Docker setup and the Nitro source code checked out, -all you need to do is run `docker build . --tag nitro-node` in the `nitro` folder. That command will build a Docker image called `nitro-node` from the local source. -The rest of the guide covers building the node binary natively, outside of Docker. -However, Docker is still used to help build some WebAssembly components. +## Build Nitro's binaries natively -### 3. Configure prerequisites +If you want to build the node binaries natively, execute steps 1-3 of the [Build a Docker image](#build-a-docker-image) section and continue with the steps described here. Notice that even though we are building the binaries outside of Docker, it is still used to help build some WebAssembly components. + +### Step 4. Configure prerequisites #### For Debian/Ubuntu @@ -92,7 +110,7 @@ sudo mkdir -p /usr/local/bin sudo ln -s /opt/homebrew/opt/llvm/bin/wasm-ld /usr/local/bin/wasm-ld ``` -### 4. Configure Node [16.19](https://github.com/nvm-sh/nvm) +### Step 5. Configure Node [16.19](https://github.com/nvm-sh/nvm) #### For Debian/Ubuntu @@ -113,7 +131,7 @@ nvm install 16.19 nvm use 16.19 ``` -### 5. Configure Rust [1.72.1](https://www.rust-lang.org/tools/install) +### Step 6. Configure Rust [1.72.1](https://www.rust-lang.org/tools/install) ```bash curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh @@ -125,7 +143,7 @@ rustup target add wasm32-wasi --toolchain 1.72.1 cargo install cbindgen ``` -### 6. Configure Go [1.20](https://github.com/moovweb/gvm) +### Step 7. Configure Go [1.20](https://github.com/moovweb/gvm) #### Install Bison @@ -153,8 +171,22 @@ curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/insta If you use zsh, replace `bash` with `zsh`. -### 7. Start build +### Step 8. Start build ```bash make ``` + +### Step 9. Produce binaries + +```bash +make build +``` + +### Step 10. Run your node + +To run your node using the generated binaries, use the following command from the `nitro` folder, with your desired parameters + +```bash +./target/bin/nitro +``` From 150e51ad586f44decf357ea3d37eb0fce9454f7b Mon Sep 17 00:00:00 2001 From: TucksonDev Date: Fri, 13 Oct 2023 12:53:09 +0100 Subject: [PATCH 4/7] Fix link from precompile page --- .../launch-orbit-chain/how-tos/customize-precompile.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx b/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx index 0825ca3b5..5b159b910 100644 --- a/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx +++ b/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx @@ -50,7 +50,7 @@ Then, open the corresponding Solidity interface file (`ArbSys.sol`) from the [pr function sayHi() external view returns(string memory); ``` -Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#building-a-dev-image) to build a modified Arbitrum Nitro node docker image and run it. +Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#41-build-a-dev-image) to build a modified Arbitrum Nitro node docker image and run it. :::info @@ -125,7 +125,7 @@ interface ArbHi { } ``` -Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#building-a-dev-image) to build a modified Arbitrum Nitro node docker image and run it. +Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#41-build-a-dev-image) to build a modified Arbitrum Nitro node docker image and run it. :::info From 2241085dfeec24ed38e8ea3b6385b0d55ebaa665 Mon Sep 17 00:00:00 2001 From: TucksonDev Date: Fri, 13 Oct 2023 12:56:32 +0100 Subject: [PATCH 5/7] Adjust link again --- .../launch-orbit-chain/how-tos/customize-precompile.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx b/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx index 5b159b910..44d31fe8c 100644 --- a/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx +++ b/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx @@ -50,7 +50,7 @@ Then, open the corresponding Solidity interface file (`ArbSys.sol`) from the [pr function sayHi() external view returns(string memory); ``` -Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#41-build-a-dev-image) to build a modified Arbitrum Nitro node docker image and run it. +Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#step-4-enable-fraud-proofs) to build a modified Arbitrum Nitro node docker image and run it. :::info @@ -125,7 +125,7 @@ interface ArbHi { } ``` -Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#41-build-a-dev-image) to build a modified Arbitrum Nitro node docker image and run it. +Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#step-4-enable-fraud-proofs) to build a modified Arbitrum Nitro node docker image and run it. :::info From 9d583907adfa3ba757f06cd3eccf5dba63b96395 Mon Sep 17 00:00:00 2001 From: TucksonDev Date: Fri, 13 Oct 2023 13:01:57 +0100 Subject: [PATCH 6/7] Changing the link again --- .../launch-orbit-chain/how-tos/customize-precompile.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx b/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx index 44d31fe8c..181394a33 100644 --- a/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx +++ b/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx @@ -50,7 +50,7 @@ Then, open the corresponding Solidity interface file (`ArbSys.sol`) from the [pr function sayHi() external view returns(string memory); ``` -Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#step-4-enable-fraud-proofs) to build a modified Arbitrum Nitro node docker image and run it. +Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#step-3-run-the-node-without-fraud-proofs) to build a modified Arbitrum Nitro node docker image and run it. :::info @@ -125,7 +125,7 @@ interface ArbHi { } ``` -Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#step-4-enable-fraud-proofs) to build a modified Arbitrum Nitro node docker image and run it. +Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#step-3-run-the-node-without-fraud-proofs) to build a modified Arbitrum Nitro node docker image and run it. :::info From 24d7e4b337fb047053d7461203c27a02f6423c82 Mon Sep 17 00:00:00 2001 From: Harry Kalodner Date: Fri, 13 Oct 2023 08:13:00 -0400 Subject: [PATCH 7/7] Fix formatting --- arbitrum-docs/for-devs/quickstart-solidity-hardhat.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arbitrum-docs/for-devs/quickstart-solidity-hardhat.md b/arbitrum-docs/for-devs/quickstart-solidity-hardhat.md index 0bcbac1cc..6794cf38c 100644 --- a/arbitrum-docs/for-devs/quickstart-solidity-hardhat.md +++ b/arbitrum-docs/for-devs/quickstart-solidity-hardhat.md @@ -173,7 +173,7 @@ Open `scripts/deploy.js` and replace its contents with the following: const hre = require('hardhat'); async function main() { - const vendingMachine = await hre.ethers.deployContract("VendingMachine"); + const vendingMachine = await hre.ethers.deployContract('VendingMachine'); await vendingMachine.waitForDeployment(); console.log(`Cupcake vending machine deployed to ${vendingMachine.target}`); }