diff --git a/README.md b/README.md index 0eee8038..2b3457da 100644 --- a/README.md +++ b/README.md @@ -15,16 +15,117 @@ Minimalist, fast and modular implementation of a Beacon Chain and Lean Chain cli Follows Ethereum mainnet from a checkpoint-synced anchor and serves the standard Beacon API on port 5052, paired with an [ethrex](https://github.com/lambdaclass/ethrex) execution client that validates -each block's execution payload. Both run from pre-built images -(`ghcr.io/lambdaclass/ethlambda:beacon` and `ghcr.io/lambdaclass/ethrex`), so -only [Docker](https://www.docker.com/get-started) is needed. +each block's execution payload. The two talk over the Engine API, authenticated +with a shared JWT secret. Both ship release binaries for Linux (x86_64, +aarch64) and macOS (Apple silicon): ethlambda's are listed under +[Releases](#releases) below and ethrex's on +[its releases page](https://github.com/lambdaclass/ethrex/releases/latest). +To run both from pre-built Docker images instead, see +[Running with Docker](#running-with-docker) below. > **Warning:** this uses a lot of disk. ethrex needs at least 500 GB for > mainnet (1 TB recommended; see its > [hardware requirements](https://docs.ethrex.xyz/getting-started/hardware_requirements.html)). ```sh -docker pull ghcr.io/lambdaclass/ethlambda:beacon +# Pick the assets for your platform: linux-x86_64, linux-aarch64 or macos-aarch64 +curl -LO https://github.com/lambdaclass/ethlambda/releases/latest/download/ethlambda-linux-x86_64 +curl -LO https://github.com/lambdaclass/ethrex/releases/latest/download/ethrex-linux-x86_64 +chmod +x ethlambda-linux-x86_64 ethrex-linux-x86_64 +mv ethlambda-linux-x86_64 ethlambda && mv ethrex-linux-x86_64 ethrex +./ethlambda --version && ./ethrex --version + +mkdir -p beacon-data ethrex-data +# A persisted key keeps the node's identity, and so its custody set, stable across restarts +openssl rand -hex 32 > beacon-data/node-key +# The secret both clients authenticate the Engine API with +openssl rand -hex 32 > jwt.hex + +# In one terminal: ethrex, serving the Engine API on 127.0.0.1:8551 +./ethrex \ + --network mainnet \ + --datadir ethrex-data \ + --authrpc.jwtsecret jwt.hex + +# In another: the beacon node, pointed at ethrex +./ethlambda beacon \ + --network mainnet \ + --checkpoint-sync-url https://beaconstate.ethstaker.cc \ + --node-key beacon-data/node-key \ + --data-dir beacon-data/db \ + --discovery.advertise-ip "$(curl -s https://ifconfig.me)" \ + --execution-endpoint http://127.0.0.1:8551 \ + --execution-jwt-secret jwt.hex +``` + +`--discovery.advertise-ip` is the address published in the node's ENR. Without +it the node advertises `0.0.0.0` and logs a warning. It still dials out and +follows the chain, but peers cannot reach it until discovery learns the address +from their replies. The Beacon API binds `127.0.0.1:5052` by default, so +`--http-address` is only needed to reach it from another host. Stop the node +with `Ctrl+C`; it shuts down gracefully on SIGINT only, so a systemd unit needs +`KillSignal=SIGINT`. + +- `ethlambda-linux-x86_64` targets x86-64-v3, so it needs a CPU with AVX2 + (Haswell or later), and glibc 2.35 or newer. +- The macOS binaries are not notarized. A browser download is quarantined, which + `xattr -d com.apple.quarantine ` lifts; a `curl` download is not. +- [`RELEASE.md`](./RELEASE.md#verifying-a-release) shows how to verify an + ethlambda binary's provenance attestation. + +To skip payload validation, run the node without ethrex: drop it, the JWT +secret and the two `--execution-*` flags. The node then follows the chain and +serves the Beacon API without validating execution payloads, but it cannot +serve a validator client, since it answers attestation data, aggregation and +block production requests with `503`. + +The first start downloads the finalized state (several hundred MB) and logs +`Beacon checkpoint sync complete`, then a few minutes of +`Block parent missing, storing as pending` before `Block imported successfully` +lines start. Later starts resume from `beacon-data/db`. + +ethrex logs `No messages from the consensus layer` until the beacon node's +first fork choice update, then starts snap sync. Until that finishes, which +takes hours on mainnet, ethrex answers each payload `SYNCING` and the beacon +node imports blocks optimistically. + +Check progress from another terminal: + +```sh +curl -s localhost:5052/eth/v1/node/syncing +``` + +The node has caught up once `sync_distance` (the chain's current slot minus +`head_slot`) is near 0. Don't rely on `is_syncing` yet: it currently reads +`false` during catch-up as well. + +For Sepolia or Hoodi, change `--network mainnet` to `--network sepolia` or +`--network hoodi` in both the ethrex and the beacon node arguments, and point +`--checkpoint-sync-url` at that network's provider. See +[`ethlambda beacon`](#ethlambda-beacon--the-ethereum-beacon-chain) below for +the remaining flags. + +`--network plataberget` (the Glamsterdam testnet) is built in as well. It has +run Gloas since its `GLOAS_FORK_EPOCH`, which the node follows: a Gloas +checkpoint anchor is accepted, payload envelopes are fetched and imported, and +the validator-client endpoints serve Gloas duties, the payload envelope and the +payload timeliness vote included. + +The same binary runs [`ethlambda node`](#ethlambda-node--the-lean-consensus-chain) +and [`ethlambda validator`](#ethlambda-validator--the-validator-client); their +flags are below. + +### Running with Docker + +The same two nodes from pre-built images (`ghcr.io/lambdaclass/ethlambda:latest` +and `ghcr.io/lambdaclass/ethrex`), so only +[Docker](https://www.docker.com/get-started) is needed. Follow the logs with +`docker logs -f ethlambda-beacon` and `docker logs -f ethrex`. For another +network, change `--network mainnet` in both containers' arguments, not Docker's +own `--network ethereum`. + +```sh +docker pull ghcr.io/lambdaclass/ethlambda:latest docker pull ghcr.io/lambdaclass/ethrex mkdir -p beacon-data ethrex-data @@ -32,7 +133,7 @@ mkdir -p beacon-data ethrex-data openssl rand -hex 32 > beacon-data/node-key # The secret both clients authenticate the Engine API with openssl rand -hex 32 > jwt.hex -# This host's public address, published in the node's ENR (see below) +# This host's public address, published in the node's ENR (see above) PUBLIC_IP=$(curl -s https://ifconfig.me) # A private network, so the beacon node reaches ethrex's Engine API by name @@ -55,7 +156,7 @@ docker run -d --name ethlambda-beacon --network ethereum --stop-signal SIGINT \ -p 127.0.0.1:5052:5052 \ -v "$PWD/beacon-data:/data" \ -v "$PWD/jwt.hex:/jwt.hex:ro" \ - ghcr.io/lambdaclass/ethlambda:beacon beacon \ + ghcr.io/lambdaclass/ethlambda:latest beacon \ --network mainnet \ --checkpoint-sync-url https://beaconstate.ethstaker.cc \ --node-key /data/node-key \ @@ -69,19 +170,16 @@ docker run -d --name ethlambda-beacon --network ethereum --stop-signal SIGINT \
Running without an execution client -To run the consensus layer only, drop ethrex, the shared network, the JWT secret -and the two `--execution-*` flags. The node then follows the chain without -validating execution payloads. It cannot serve a validator client, since it -answers attestation data, aggregation and block production requests with -`503`: +The node without ethrex, as described above: no shared network, no JWT secret +and no `--execution-*` flags. ```sh -docker pull ghcr.io/lambdaclass/ethlambda:beacon +docker pull ghcr.io/lambdaclass/ethlambda:latest mkdir -p beacon-data # A persisted key keeps the node's identity, and so its custody set, stable across restarts openssl rand -hex 32 > beacon-data/node-key -# This host's public address, published in the node's ENR (see below) +# This host's public address, published in the node's ENR (see above) PUBLIC_IP=$(curl -s https://ifconfig.me) # The node shuts down gracefully on SIGINT only, so this makes `docker stop` flush its state @@ -89,7 +187,7 @@ docker run -d --name ethlambda-beacon --stop-signal SIGINT \ -p 9000:9000/udp -p 9001:9001/udp -p 9001:9001/tcp \ -p 127.0.0.1:5052:5052 \ -v "$PWD/beacon-data:/data" \ - ghcr.io/lambdaclass/ethlambda:beacon beacon \ + ghcr.io/lambdaclass/ethlambda:latest beacon \ --network mainnet \ --checkpoint-sync-url https://beaconstate.ethstaker.cc \ --node-key /data/node-key \ @@ -100,46 +198,6 @@ docker run -d --name ethlambda-beacon --stop-signal SIGINT \
-`--discovery.advertise-ip` is the address published in the node's ENR. Behind -Docker's port mapping the node cannot see its own public address, so without -the flag it advertises `0.0.0.0` and logs a warning. It still dials out and -follows the chain, but peers cannot reach it until discovery learns the address -from their replies. - -Follow the logs with `docker logs -f ethlambda-beacon`. The first start -downloads the finalized state (several hundred MB) and logs -`Beacon checkpoint sync complete`, then a few minutes of -`Block parent missing, storing as pending` before `Block imported successfully` -lines start. Later starts resume from `beacon-data/db`. - -ethrex logs `No messages from the consensus layer` until the beacon node's -first fork choice update, then starts snap sync (`docker logs -f ethrex`). -Until that finishes, which takes hours on mainnet, ethrex answers each payload -`SYNCING` and the beacon node imports blocks optimistically. - -Check progress from another terminal: - -```sh -curl -s localhost:5052/eth/v1/node/syncing -``` - -The node has caught up once `sync_distance` (the chain's current slot minus -`head_slot`) is near 0. Don't rely on `is_syncing` yet: it currently reads -`false` during catch-up as well. - -For Sepolia or Hoodi, change `--network mainnet` to `--network sepolia` or -`--network hoodi` in both the ethrex and the beacon node arguments (not Docker's -own `--network ethereum`), and point `--checkpoint-sync-url` at that network's -provider. See -[`ethlambda beacon`](#ethlambda-beacon--the-ethereum-beacon-chain) below for -the remaining flags. - -`--network plataberget` (the Glamsterdam testnet) is built in as well. It has -run gloas since its `GLOAS_FORK_EPOCH`, which the follower follows: a gloas -checkpoint anchor is accepted, and payload envelopes are fetched and imported. -It stays a follower, so the validator-client endpoints still refuse a gloas -epoch. - ### Lean consensus devnet To run a local lean devnet with ethlambda, follow the instructions on the @@ -151,8 +209,9 @@ To run a local lean devnet with ethlambda, follow the instructions on the - [Rust](https://rust-lang.org/tools/install) - [Git](https://git-scm.com/install) -- [Docker](https://www.docker.com/get-started) -- [yq](https://github.com/mikefarah/yq#install) +- For `make run-devnet` and `make docker-build` only: + [Docker](https://www.docker.com/get-started) and + [yq](https://github.com/mikefarah/yq#install) ### Building and testing @@ -173,14 +232,16 @@ Run `make help` or take a look at our [`Makefile`](./Makefile) for other useful ### Running the node -The binary follows one of two chains, chosen by a sub-command: +The binary carries both chains and the validator client as sub-commands: ```sh cargo build --release -./target/release/ethlambda [flags] +./target/release/ethlambda [flags] ``` -[`docs/cli.md`](./docs/cli.md) is the full flag reference for both. +[`docs/cli.md`](./docs/cli.md) is the full flag reference for every sub-command, +`benchmark` and `replay-era` included. The +[quickstart](#beacon-chain-follower) runs a release binary with the same arguments. #### `ethlambda node` — the Lean consensus chain @@ -217,13 +278,16 @@ under `lean-quickstart/local-devnet/genesis/`. See #### `ethlambda beacon` — the Ethereum Beacon Chain -A beacon chain follower. It anchors at a finalized checkpoint fetched from a -Beacon API, then follows the chain to its tip: blocks arriving over gossip and -range sync are imported through fork choice, and it custodies its slice of the -fulu data column matrix. What it stores it serves back, to peers over +A beacon node. It anchors at a finalized checkpoint fetched from a Beacon API, +then follows the chain to its tip: blocks arriving over gossip and range sync +are imported through fork choice, and it custodies its slice of the fulu data +column matrix. What it stores it serves back, to peers over `beacon_blocks_by_{range,root}` and to anyone over the standard Beacon API on -`--api-port`. It has no validator duties: it publishes nothing and subscribes to -no attestation or sync committee subnet. +`--api-port`. It performs no validator duties of its own: those belong to +[`ethlambda validator`](#ethlambda-validator--the-validator-client), which it +serves over that API once paired with an execution client, joining the +attestation and sync committee subnets its validators ask for and publishing +what they sign. ```sh ./target/release/ethlambda beacon \ @@ -260,6 +324,43 @@ See [`docs/beacon_wire.md`](./docs/beacon_wire.md) for what goes on the wire, chosen and verified, and [`docs/rpc.md`](./docs/rpc.md) for the Beacon API endpoints. +#### `ethlambda validator` — the validator client + +A separate process that holds the keys and performs the duties: attestations, +block proposals, aggregation, sync committee messages and, from Gloas on, the +payload envelope and the payload timeliness vote. It talks to a beacon node over +the standard Beacon API, so it works against `ethlambda beacon` (run with the +`--execution-*` flags) or any other conformant beacon node. + +> **Warning:** the validator client keeps no slashing-protection record yet. +> Slashing protection is a work in progress and will be added soon. Until then, +> do not run the same keys in another client while this one runs, and treat a +> restart with the same care as a manual key move. Read +> [Spec deviations](./docs/spec_deviations.md#the-validator-client-keeps-no-slashing-protection-record) +> before using keys that hold real stake. The client repeats this warning at +> startup. + +```sh +./target/release/ethlambda validator \ + --beacon-nodes http://127.0.0.1:5052 \ + --validators-dir ./validators \ + --secrets-dir ./secrets \ + --suggested-fee-recipient 0x +``` + +| Flag | Meaning | +|---|---| +| `--beacon-nodes` | Base URLs of the beacon nodes to use, comma-separated or repeated. Tried in order, so the order is a preference | +| `--validators-dir` | Directory of EIP-2335 keystores and `validator_definitions.yml`. Without that file, keystores in the Lighthouse layout (`<0xpubkey>/voting-keystore.json`) are discovered and the file is written | +| `--secrets-dir` | One password file per keystore, named after the validator's public key | +| `--suggested-fee-recipient` | Execution address for block rewards, `0x`-prefixed. Optional, but without it the beacon node picks an address that is not yours | +| `--graffiti` | Text for proposed blocks, at most 32 bytes | +| `--enable-keymanager` | Serve the keymanager API on `--keymanager-port` (default `5062`). Off by default, since it mutates key material | + +Metrics are served on `--metrics-port` (default `5064`). See +[`docs/cli.md`](./docs/cli.md#validator-flags) for the remaining flags and what +the client does each slot. + ### Running in a devnet To run a local devnet with multiple clients using [lean-quickstart](https://github.com/blockblaz/lean-quickstart): @@ -318,9 +419,10 @@ Full documentation is available in the [`docs/`](./docs) directory. Please refer ethlambda is a Beacon Chain and Lean Chain client. On the Beacon Chain (`ethlambda beacon`), it follows a network from a checkpoint-synced anchor and serves -the standard Beacon API; see the [Beacon Chain follower](#beacon-chain-follower) -quickstart. On the Lean Chain (`ethlambda node`), it implements the core features of a -Lean consensus client: +the standard Beacon API; paired with an execution client it validates payloads and +serves the [validator client](#ethlambda-validator--the-validator-client). See the +[Beacon Chain follower](#beacon-chain-follower) quickstart. On the Lean Chain +(`ethlambda node`), it implements the core features of a Lean consensus client: - **Networking** — libp2p peer connections, STATUS message handling, gossipsub for blocks and attestations - **State management** — genesis state generation, state transition function, block processing @@ -373,9 +475,8 @@ We wrote a [blogpost](https://blog.lambdaclass.com/ethlambda-devnet-5-and-beyond Some features we are looking to implement in the near future, in order of priority: - [Optimize block building](https://github.com/lambdaclass/ethlambda/issues/465) -- [Use state-diffs for storing states in the database](https://github.com/lambdaclass/ethlambda/issues/238) - [Prototype Goldfish + RLMD GHOST + BFT — devnet-6](https://github.com/lambdaclass/ethlambda/pull/434) -- [Integrate with execution clients](https://github.com/lambdaclass/ethlambda/pull/367), in particular [ethrex](https://github.com/lambdaclass/ethrex) — devnet-7 +- Integrate the Lean Chain with execution clients, in particular [ethrex](https://github.com/lambdaclass/ethrex) — devnet-7 ([first prototype](https://github.com/lambdaclass/ethlambda/pull/367)). The beacon node already pairs with one over the Engine API; see [`docs/beacon_engine.md`](./docs/beacon_engine.md) - Replace libp2p with the experimental [ethp2p](https://github.com/ethp2p/ethp2p), which we are porting to Rust - [Add a guest program and ZK proving of the STF](https://github.com/lambdaclass/ethlambda/issues/156) - Rewrite the STF in the concrete programming language to enable formal verification