diff --git a/arbitrum-docs/for-devs/quickstart-solidity-hardhat.md b/arbitrum-docs/for-devs/quickstart-solidity-hardhat.md index 0bcbac1cc..6794cf38c 100644 --- a/arbitrum-docs/for-devs/quickstart-solidity-hardhat.md +++ b/arbitrum-docs/for-devs/quickstart-solidity-hardhat.md @@ -173,7 +173,7 @@ Open `scripts/deploy.js` and replace its contents with the following: const hre = require('hardhat'); async function main() { - const vendingMachine = await hre.ethers.deployContract("VendingMachine"); + const vendingMachine = await hre.ethers.deployContract('VendingMachine'); await vendingMachine.waitForDeployment(); console.log(`Cupcake vending machine deployed to ${vendingMachine.target}`); } diff --git a/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx b/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx index 28bd6175b..181394a33 100644 --- a/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx +++ b/arbitrum-docs/launch-orbit-chain/how-tos/customize-precompile.mdx @@ -3,7 +3,8 @@ title: "How to customize your Orbit chain's precompiles" sidebar_label: "Customize your chain's precompiles" description: "Learn how (and when) to customize your Orbit chain's precompiles" author: jasonwan -sidebar_position: 2 +sidebar_position: 4 +content-type: how-to --- import PublicPreviewBannerPartial from '../partials/_orbit-public-preview-banner-partial.md'; @@ -14,9 +15,7 @@ import PublicPreviewBannerPartial from '../partials/_orbit-public-preview-banner The guidance in this document will work only if you use `eth_call` to call the new precompiles. If you're calling from other contracts or adding non-view/pure methods, this approach will break block validation. -To support these additional use-cases, stay tuned for instructions that walk you through the process of updating the Wasm module root. - -If you want to test modifying the State Transition System (STS) before this guidance is available, use the `--node.staker.dangerous.without-block-validator` config flag when you start your node. +To support these additional use-cases, follow the instructions described in [How to customize your Orbit chain's behavior](/launch-orbit-chain/how-tos/customize-stf.mdx). ::: @@ -51,17 +50,11 @@ Then, open the corresponding Solidity interface file (`ArbSys.sol`) from the [pr function sayHi() external view returns(string memory); ``` -Next, build Nitro by following steps 3-7 of the instructions in [How to build Nitro locally](/node-running/how-tos/build-nitro-locally). Note that if you've already built the Docker image, you still need run the last step to rebuild. - -Run Nitro with the following command: - -```shell -docker run --rm -it -v /some/local/dir/arbitrum:/home/user/.arbitrum -p 0.0.0.0:8547:8547 -p 0.0.0.0:8548:8548 @latestNitroNodeImage@ --parent-chain.connection.url= --chain.id= --http.api=net,web3,eth,debug --http.corsdomain=* --http.addr=0.0.0.0 --http.vhosts=* -``` +Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#step-3-run-the-node-without-fraud-proofs) to build a modified Arbitrum Nitro node docker image and run it. :::info -Note that the instructions provided in [How to run a full node](/node-running/how-tos/running-a-full-node) **will not** work with your Orbit node. See [Command-line options (Orbit)](/launch-orbit-chain/reference/command-line-options) for Orbit-specific CLI flags. +Note that the instructions provided in [How to run a full node](/node-running/how-tos/running-a-full-node.mdx) **will not** work with your Orbit node. See [Command-line options (Orbit)](/launch-orbit-chain/reference/command-line-options.md) for Orbit-specific CLI flags. ::: @@ -132,17 +125,11 @@ interface ArbHi { } ``` -Next, build Nitro by following the instructions in [How to build Nitro locally](/node-running/how-tos/build-nitro-locally). Note that if you've already built the Docker image, you still need run the last step to rebuild. - -Run Nitro with the following command: - -```shell -docker run --rm -it -v /some/local/dir/arbitrum:/home/user/.arbitrum -p 0.0.0.0:8547:8547 -p 0.0.0.0:8548:8548 @latestNitroNodeImage@ --parent-chain.connection.url= --chain.id= --http.api=net,web3,eth,debug --http.corsdomain=* --http.addr=0.0.0.0 --http.vhosts=* -``` +Next, follow the steps in [How to customize your Orbit chain's behavior](./customize-stf.mdx#step-3-run-the-node-without-fraud-proofs) to build a modified Arbitrum Nitro node docker image and run it. :::info -Note that the instructions provided in [How to run a full node](/node-running/how-tos/running-a-full-node) **will not** work with your Orbit node. See [Command-line options (Orbit)](/launch-orbit-chain/reference/command-line-options) for Orbit-specific CLI flags. +Note that the instructions provided in [How to run a full node](/node-running/how-tos/running-a-full-node.mdx) **will not** work with your Orbit node. See [Command-line options (Orbit)](/launch-orbit-chain/reference/command-line-options.md) for Orbit-specific CLI flags. ::: diff --git a/arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx b/arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx new file mode 100644 index 000000000..020d3e3dc --- /dev/null +++ b/arbitrum-docs/launch-orbit-chain/how-tos/customize-stf.mdx @@ -0,0 +1,218 @@ +--- +title: "How to customize your Orbit chain's behavior" +sidebar_label: "Customize your chain's behavior" +description: "Learn how to customize your Orbit chain's behavior (known as the State Transition Function or STF)" +author: plasmapower +sidebar_position: 3 +content-type: how-to +--- + +import PublicPreviewBannerPartial from '../partials/_orbit-public-preview-banner-partial.md'; + + + +## Preface + +Before customizing your Orbit chain, it's important to understand what the State Transition Function (aka the STF) is. +The STF defines how new blocks are produced from input messages (i.e. transactions). +This guide is only necessary for changes that modify the State Transition Function. +To customize other node behavior, such as RPC behavior or the sequencer's ordering policy, you can simply +[build your own node](/node-running/how-tos/build-nitro-locally.md) without worrying about the rest of this guide. +However, changes that modify the STF require updating the fraud proving system to recognize the new behavior as correct. +Otherwise, the fraud prover would side with unmodified nodes, which would win fraud proofs against your modified node. + +Here's some examples of modifications that affect the STF: + +- Adding a new EVM opcode or precompile: + This modifies the STF because a node with this change would disagree about the outcome of EVM execution compared to an unmodified nitro node + when the new opcode or precompile is invoked. +- Rewarding the deployer of a smart contract with a portion of gas spent in the smart contract's execution: + This modifies the STF because a node which has this change applied would disagree about the balance of the deployer after transactions. Such changes would lead to disagreements about block hashes compared to unmodified Nitro nodes. + +Here's some examples of modifications that don't affect the STF: + +- Adding a new RPC method to query an address's balance across multiple blocks: + This doesn't modify the STF because this doesn't change on-chain balances or block hashes. +- Changing the sequencer to order blocks by tip: + The sequencer is trusted to order transactions in Arbitrum Nitro, and it can chose any ordering it wants. + Nodes (and the fraud proofs) will simply accept the new transaction ordering as there is no single ordering they think is correct. + +### Modification compatibility with Arbitrum Nitro + +Some potential modifications are incompatible with Arbitrum Nitro and would not result in a functioning blockchain. +Here are some requirements for the Arbitrum Nitro State Transition Function: + +- The STF must be deterministic. For instance, if you gave an address a random balance using the Go randomness library, + every node would disagree on the correct amount of balance and the blockchain would not function correctly. + However, it is acceptable to take a non-deterministic path to a deterministic output. + For instance, if you randomly shuffled a list of addresses, and then gave them each 1 Ether, that would be fine, + because no matter how the list of addresses is shuffled the result is the same and all addresses are given 1 Ether. +- The STF must not reach a new result for old blocks. For instance, if you have been running an Arbitrum Nitro chain for a while, + and then you decide to modify the STF to not charge for gas, a new node that syncs the blockchain will reach a different result + for historical blocks. It's also important to synchronize between nodes when an upgrade takes effect. A common mechanism + for doing this is having an upgrade take effect at a certain timestamp, by which all nodes must be upgraded. +- The STF must be "pure" and not use external resources. For instance, it must not use the filesystem, make external network calls, + or launch processes. That's because the fraud proving system does not (and for the most part, cannot) support these resources. + For instance, it's impossible to fraud prove what the result of an external network call is, because the fraud prover smart contracts on L1 are unable to do networking. +- The STF must not carry state between blocks outside of the "global state". In practice this means persistent state must be stored within + block headers or the Ethereum state trie. For instance, ArbOS stores all retryables in contract storage under a special ArbOS address. +- The STF must not modify Ethereum state outside of a transaction. This is important to ensure that replaying old blocks reaches the same result, + both for tracing and for validation. The ArbOS internal transaction is useful to modify state at the start of blocks. +- The STF must reach a result in under a second. This is a rough rule, but for nodes to keep in sync it's highly recommended to keep blocks quick. + It's also important for the fraud proofs that execution reliably finishes in a relatively short amount of time. + A block gas limit of 32 million gas should safely fit within this limit. +- The STF must not fail or panic. It's important that the STF always produces a new block, even if user input is malformed. + For instance, if the STF receives an invalid transaction as input, it'll still produce an empty block. + +## Building the modified node + +To modify the State Transition Function, you'll need to build a modified Arbitrum Nitro node docker image. +This guide covers how to build the node and enable fraud proofs by building a new replay binary. + +### Step 1. Download the Nitro source code + +Clone the Nitro repository before you begin: + +```shell +git clone --branch @nitroVersionTag@ https://github.com/OffchainLabs/nitro.git +cd nitro +git submodule update --init --recursive --force +``` + +### Step 2. Apply modifications + +Next, make your changes to the State Transition Function. For example, you could [add a custom precompile](./customize-precompile.mdx). + +### Step 3. Run the node without fraud proofs + +To build the Arbitrum Nitro node image, you'll first need to install Docker. +You can confirm if it's already setup by running `docker version` in a terminal. +If not, try following [Docker's getting started guide](https://www.docker.com/get-started/), or if you're on Linux, +install Docker from your distribution's package manager and start the docker service. + +Once you have Docker installed, you can simply run `docker build . --tag custom-nitro-node` in the `nitro` folder to build your custom node. + +Once you've built your new Nitro node image, you can run it with the following command. The last argument `--node.staker.dangerous.without-block-validator` disables fraud proof verification: + +```shell +docker run --rm -it -v /some/local/dir/arbitrum:/home/user/.arbitrum -p 0.0.0.0:8547:8547 -p 0.0.0.0:8548:8548 custom-nitro-node --parent-chain.connection.url= --chain.id= --http.api=net,web3,eth,debug --http.corsdomain=* --http.addr=0.0.0.0 --http.vhosts=* --node.staker.dangerous.without-block-validator +``` + +:::info + +Note that the instructions provided in [How to run a full node](/node-running/how-tos/running-a-full-node.mdx) **will not** work with your Orbit node. See [Command-line options (Orbit)](/launch-orbit-chain/reference/command-line-options.md) for Orbit-specific CLI flags. + +::: + +Once your node is running, you can try out your modifications to the State Transition Function and confirm they work as expected. + +### Step 4. Enable fraud proofs + +To enable fraud proofs, you'll need to build the "replay binary", which defines the State Transition Function for the fraud prover. +The replay binary (sometimes called the machine) re-executes the State Transition Function against input messages to determine the correct output block. +It has three forms: + +- The `replay.wasm` binary is the Go replay binary compiled to WASM. It's used by the JIT validator to verify blocks against the fraud prover. +- The `machine.wavm.br` binary is a compressed binary containing the Go replay binary and all its dependencies, compiled to WASM, then translated to the Arbitrum fraud proving variant WAVM. + It's used by Arbitrator when actually entering a challenge and performing the fraud proofs, + and has identical behavior to `replay.wasm`. +- The WASM module root (stored in `module-root.txt`) is a 32 byte hash usually expressed in hexadecimal which is a merkelization of `machine.wavm.br`. + The replay binary is much too large to post on-chain, so this hash is set in the L1 rollup contract to determine the correct replay binary during fraud proofs. + +To run a validator node with fraud proofs enabled, the validator node's Docker image will need to contain all three of these versions of the replay binary. + +#### 4.1 Build a dev image + +The simplest way to build a Docker image with the new replay binary is to build a dev image. +These images contain a freshly built replay binary, but note that the replay binary and corresponding WASM module root will generally change when the code is updated, +even if the State Transition Function has equivalent behavior. +It's important that the validator's WASM module root matches the on-chain WASM module root, which is why this approach is harder to maintain. +Over the longer term, you'll want to maintain a separate build of the replay binary that matches the one currently on-chain, usable by any node image. + +To build the dev node image and get the WASM module root, run: + +```shell +docker build . --target nitro-node-dev --tag custom-nitro-node-dev +docker run --rm --entrypoint cat custom-nitro-node-dev target/machines/latest/module-root.txt +``` + +Once you have the WASM module root, you can put it on-chain by calling `setWasmModuleRoot(newWasmModuleRoot)` on the rollup contract as the owner. +The rollup contract address can be found in the chain deployment info JSON. +You can confirm that the WASM module root was updated by calling `wasmModuleRoot()` on the rollup contract. + +Once you have set the new WASM module root on-chain, you may then run the new node image with: + +```shell +docker run --rm -it -v /some/local/dir/arbitrum:/home/user/.arbitrum -p 0.0.0.0:8547:8547 -p 0.0.0.0:8548:8548 custom-nitro-node-dev --parent-chain.connection.url= --chain.id= --http.api=net,web3,eth,debug --http.corsdomain=* --http.addr=0.0.0.0 --http.vhosts=* +``` + +Note that `--node.staker.dangerous.without-block-validator` has been removed from this invocation now that fraud proofs are working again. + +#### 4.2 Preserving the Replay Binary + +The primary issue with simply using a nitro-node-dev build is that, whenever the code changes at all, the replay binary will also change. + +If the node is missing the replay binary corresponding to the on-chain WASM module root, it will be unable to act as a validator. +Therefore, when releasing new node Docker images it's important to include the currently on-chain WASM module root. + +To do that, you'll need to first extract the replay binary from the `nitro-node-dev` Docker image built earlier: + +```shell +docker run --rm --name replay-binary-extractor --entrypoint sleep custom-nitro-node-dev infinity +docker cp replay-binary-extractor:/home/user/target/machines/latest extracted-replay-binary +docker stop replay-binary-extractor +cat extracted-replay-binary/module.root +mv extracted-replay-binary "target/machines/$(cat extracted-replay-binary/module.root)" +``` + +These commands will output the new WASM module root, and create the directory `target/machines/`. +There you'll find the three versions of the replay binary mentioned earlier: +`replay.wasm`, `machine.wavm.br`, and `module-root.txt`, along with some other optional files. +Now that you've extracted the replay binary, there are two ways to add it to future Docker images, +including non-dev image builds. You can either keep it locally and copy it in, or host it on the web. + +##### Option 1: Store the extracted replay binary locally + +Now that we've extracted the replay binary, we can modify the Dockerfile to copy it into new Docker builds. +Edit the `Dockerfile` file in the root of the nitro folder, and after all the `RUN ./download-machines.sh ...` lines, add: + +```dockerfile +COPY target/machines/ +RUN ln -sfT latest +``` + +Replace each `` with the WASM module root you got earlier. + +##### Option 2: Host the replay binary on the web + +To support building the Docker image on other computers without this local machine directory, +you'll need to either commit the machine to git, or preferably, host the replay binary on the web. + +To host the replay binary on the web, you'll need to host the `replay.wasm` and `machine.wavm.br` files somewhere. +One good option is GitHub releases, but any hosting service works. + +Once you have those two files hosted, instead of the `COPY` and `RUN` command mentioned in option 1, +you'll need to add these new lines to the `Dockerfile` file in the root of the nitro folder, +after all the `RUN ./download-machines.sh ...` lines: + +```dockerfile +RUN wasm_module_root="" && \ + mkdir "$wasm_module_root" && \ + wget -O "$wasm_module_root/replay.wasm" && \ + wget -O "$wasm_module_root/machine.wavm.br" && \ + echo "$wasm_module_root" > "$wasm_module_root/module-root.txt" && \ + ln -sfT "$wasm_module_root" latest +``` + +Replace the `` with the WASM module root you got earlier, +the `` with the direct link to the `replay.wasm` file (it must be a direct link to the file and not just a download site), +and the `` with the direct link to the `machine.wavm.br` file. + +### Step 5. Verify the fraud proofs + +In theory, fraud proofs should now be working with your newly built Docker images. +Make some transactions on your new blockchain, test out your modifications to the State Transition Function, +wait for a batch to be posted, and you should be seeing "validation succeeded" log lines! + +If you see "Error during validation", then the replay binary is likely not up-to-date with your modifications to the State Transition Function. +Ensure that the replay binary is freshly built and is not missing any modifications, and that the WASM module root set in the rollup contract matches your replay binary. diff --git a/arbitrum-docs/node-running/how-tos/build-nitro-locally.md b/arbitrum-docs/node-running/how-tos/build-nitro-locally.md index d3f5d3f38..614bf7343 100644 --- a/arbitrum-docs/node-running/how-tos/build-nitro-locally.md +++ b/arbitrum-docs/node-running/how-tos/build-nitro-locally.md @@ -11,13 +11,76 @@ import PublicPreviewBannerPartial from '../../partials/_public-preview-banner-pa +Arbitrum Nitro is the software that powers all Arbitrum chains. This how-to shows how you can build a docker image, or binaries, directly from Nitro's source code. If you want to run a node for one of the Arbitrum chains, however, it is recommended that you use the docker image available on DockerHub, as explained in [How to run a full node](/node-running/how-tos/running-a-full-node.mdx). + This how-to assumes that you're running one of the following operating systems: - [Debian 11.7 (arm64)](https://cdimage.debian.org/cdimage/archive/11.7.0/arm64/iso-cd/debian-11.7.0-arm64-netinst.iso) - [Ubuntu 22.04 (amd64)](https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-desktop-amd64.iso) - [MacOS Ventura 13.4](https://developer.apple.com/documentation/macos-release-notes/macos-13_4-release-notes). -### 1. Configure prerequisites +## Build a Docker image + +### Step 1. Configure [Docker](https://docs.docker.com/engine/install) + +#### For [Debian](https://docs.docker.com/engine/install/debian)/[Ubuntu](https://docs.docker.com/engine/install/ubuntu) + +```bash +for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove $pkg; done +# Add Docker's official GPG key: +sudo apt-get update +sudo apt-get install ca-certificates curl gnupg +sudo install -m 0755 -d /etc/apt/keyrings +curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg +sudo chmod a+r /etc/apt/keyrings/docker.gpg + +# Add the repository to Apt sources: +echo \ + "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \ + "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \ + sudo tee /etc/apt/sources.list.d/docker.list > /dev/null +sudo apt-get update +sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin +sudo service docker start +``` + +#### For [MacOS](https://docs.docker.com/desktop/install/mac-install/) + +Depending on whether your Mac has an Intel processor or Apple silicon, download the corresponding disk image from [Docker](https://docs.docker.com/desktop/install/mac-install/), and move it into your Applications folder. + +#### [Optional] Run docker from a different user + +After installing docker, you might want to be able to run it with your current user instead of root. You can run the following commands to do so. + +```bash +sudo groupadd docker +sudo usermod -aG docker $USER +newgrp docker +``` + +For troubleshooting, check Docker's section in [their documentation](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user) + +### Step 2. Download the Nitro source code + +```bash +git clone --branch @nitroVersionTag@ https://github.com/OffchainLabs/nitro.git +cd nitro +git submodule update --init --recursive --force +``` + +### Step 3. Build the Nitro node docker image + +```bash +docker build . --tag nitro-node +``` + +That command will build a Docker image called `nitro-node` from the local source. + +## Build Nitro's binaries natively + +If you want to build the node binaries natively, execute steps 1-3 of the [Build a Docker image](#build-a-docker-image) section and continue with the steps described here. Notice that even though we are building the binaries outside of Docker, it is still used to help build some WebAssembly components. + +### Step 4. Configure prerequisites #### For Debian/Ubuntu @@ -47,15 +110,7 @@ sudo mkdir -p /usr/local/bin sudo ln -s /opt/homebrew/opt/llvm/bin/wasm-ld /usr/local/bin/wasm-ld ``` -### 2. Configure Nitro - -```bash -git clone https://github.com/OffchainLabs/nitro.git -cd nitro -git submodule update --init --recursive --force -``` - -### 3. Configure Node [16.19](https://github.com/nvm-sh/nvm) +### Step 5. Configure Node [16.19](https://github.com/nvm-sh/nvm) #### For Debian/Ubuntu @@ -76,7 +131,7 @@ nvm install 16.19 nvm use 16.19 ``` -### 4. Configure Rust [1.72.1](https://www.rust-lang.org/tools/install) +### Step 6. Configure Rust [1.72.1](https://www.rust-lang.org/tools/install) ```bash curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh @@ -88,34 +143,7 @@ rustup target add wasm32-wasi --toolchain 1.72.1 cargo install cbindgen ``` -### 5. Configure [Docker](https://docs.docker.com/engine/install) - -#### For [Debian](https://docs.docker.com/engine/install/debian)/[Ubuntu](https://docs.docker.com/engine/install/ubuntu) - -```bash -for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove $pkg; done -# Add Docker's official GPG key: -sudo apt-get update -sudo apt-get install ca-certificates curl gnupg -sudo install -m 0755 -d /etc/apt/keyrings -curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg -sudo chmod a+r /etc/apt/keyrings/docker.gpg - -# Add the repository to Apt sources: -echo \ - "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \ - "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \ - sudo tee /etc/apt/sources.list.d/docker.list > /dev/null -sudo apt-get update -sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -sudo service docker start -``` - -#### For [MacOS](https://docs.docker.com/desktop/install/mac-install/) - -Depending on whether your Mac has an Intel processor or Apple silicon, download the corresponding disk image from [Docker](https://docs.docker.com/desktop/install/mac-install/), and move it into your Applications folder. - -### 6. Configure Go [1.20](https://github.com/moovweb/gvm) +### Step 7. Configure Go [1.20](https://github.com/moovweb/gvm) #### Install Bison @@ -143,8 +171,22 @@ curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/insta If you use zsh, replace `bash` with `zsh`. -### 7. Start build +### Step 8. Start build ```bash make ``` + +### Step 9. Produce binaries + +```bash +make build +``` + +### Step 10. Run your node + +To run your node using the generated binaries, use the following command from the `nitro` folder, with your desired parameters + +```bash +./target/bin/nitro +```