`)
- so relative bind mounts (`./configs`, `./data`, `./genvm-module-*.yaml`) resolve.
+ so relative bind mounts (`./configs`, `./data`, `./genvm-module-*.yaml`) resolve, with
+ `-p genlayer-node` (rc.1 compose has no fixed project name).
- **Full node**: set `node.mode: "full"` in config.yaml — no wallet/operator/keystore/staking
needed (logs: "full node mode detected, skipping validator setup").
- **Custom image (e.g. ECR)**: override the node service `image:` (edit the compose or add a
`docker-compose.override.yml`). If the instance role lacks ECR permissions, authenticate
Docker with an admin token:
`aws ecr get-login-password --profile --region | docker login --username AWS --password-stdin .dkr.ecr..amazonaws.com`
-- **LLM in Docker**: mount a `genvm-module-llm.yaml` (built from the release's
- `genvm-modules-llm-release.yaml`) onto `/app/opt/genvm//config/genvm-module-llm.yaml`
- so your provider is enabled, and set `GENVM_VERSION` in `.env`.
+- **rc.1 Docker mounts**: the rc.1 compose file mounts only `config.yaml`, `data/` and the web
+ module. Set `GENVM_VERSION` in `.env` (the web-module mount path uses it), and mount the
+ protocol manifest (else the node refuses to start) and your `genvm-module-llm.yaml` (built from
+ the release's `genvm-modules-llm-release.yaml`; else the image's default LLM config is used).
+ Compose loads an override file automatically:
+ ```bash
+ cd /opt/genlayer-node/${VERSION}
+ GENVM_REL=$(sed -n 's/^ *root_dir: *\.\///p' configs/node/config.yaml) # e.g. opt/genvm/v0.6.0-rc10
+ GENVM_VERSION=$(basename "$GENVM_REL")
+ sudo grep -q '^GENVM_VERSION=' .env || echo "GENVM_VERSION=${GENVM_VERSION}" | sudo tee -a .env >/dev/null
+ sudo -u genlayer tee docker-compose.override.yml >/dev/null < Local / Remote SSH / GCP / AWS / Azure / Other cloud
-2. **New validator?** -> Staking wizard or skip
+2. **Node mode?** -> Full node / Validator (new: staking wizard, or existing)
3. **.env source?** -> Copy from previous / Create new
4. **Config source?** -> Use example (recommended) / Copy from previous
5. **Operator key?** -> Upload keystore.json / Upload keystore directory / Copy from previous / Generate new
@@ -917,7 +1002,7 @@ When you use the word "latest", the skill will automatically fetch and install t
**IMPORTANT**: When collecting user inputs, follow these rules:
-### .env Configuration (Step 3) - Ask via Wizard
+### config.yaml Rollup URLs (Step 4) - Ask via Wizard
Use AskUserQuestion with these options (tool requires minimum 2 options):
- **RPC URL** - Options: ["Skip for now", "I'll enter my URL"]
- **WebSocket URL** - Options: ["Skip for now", "I'll enter my URL"]
@@ -927,7 +1012,7 @@ Use AskUserQuestion with these options (tool requires minimum 2 options):
Example flow:
```
Step 1: AskUserQuestion
- Question: "Do you have the GenLayer Chain RPC URL?"
+ Question: "Do you have the GenLayer Chain RPC URL?" (from your network's docs)
Options:
- "Skip for now" (will need to configure manually later)
- "I'll enter my URL"
@@ -944,8 +1029,10 @@ These values are configured in config.yaml during the Node Identity group:
### Sensitive Values - NEVER Ask for These
These values must NEVER be collected by Claude. Generate .env with placeholders, then ask user to update manually on the VM:
-- **LLM API Key** - Generate placeholder: `YOUR_API_KEY_HERE`
-- **Node Password** - Generate placeholder: `YOUR_PASSWORD_HERE`
+- **RPC Provider Key** - Leave empty: `GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCKEY=`
+- **LLM API Key** - Leave empty (e.g. `OPENROUTERKEY=`)
+- **Node Password** - Leave empty: `GENLAYERNODE_PASSWORD=` (Docker: `NODE_PASSWORD=`);
+ the user sets `NODE_PASSWORD` (rc.1 compose needs it non-empty — full node: any value)
After generating .env, instruct user to:
1. SSH to the VM
@@ -958,7 +1045,8 @@ After generating .env, instruct user to:
### What Qualifies as a Secret
- **LLM API Keys**: `HEURISTKEY`, `COMPUT3KEY`, `IOINTELLIGENCEKEY`, and other provider keys
-- **Passwords**: `NODE_PASSWORD`, export passphrases
+- **RPC Key**: `GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCKEY` (RPC provider, e.g. Conduit)
+- **Passwords**: `GENLAYERNODE_PASSWORD`, `NODE_PASSWORD`, export passphrases
- **Auth Tokens**: Monitoring credentials, API tokens
- **Private Keys**: Never displayed at all
@@ -1015,8 +1103,11 @@ This will show the key value on your secure terminal."
**Reading .env file:**
```bash
-# Before displaying, mask secrets using sed
-cat /opt/genlayer-node/.env | sed -E 's/(KEY|PASSWORD|TOKEN)=([^[:space:]]{12}).*/\1=\2********.../'
+# Before displaying, mask secrets using sed (passwords fully, keys first 8 + last 4)
+sudo sed -E \
+ -e 's/^([A-Z_0-9]*(PASSWORD|PASSPHRASE)[A-Z_0-9]*=).+/\1***MASKED***/' \
+ -e '/\*\*\*MASKED\*\*\*$/!s/^([A-Z_0-9]*(KEY|TOKEN)[A-Z_0-9]*=)(.{8}).*(.{4})$/\1\3********...\4/' \
+ /opt/genlayer-node/.env
```
**Verifying environment variables:**
@@ -1026,9 +1117,9 @@ env | grep -E 'KEY|PASSWORD|TOKEN' | sed -E 's/=.*/=***MASKED***/'
```
**Creating systemd service:**
-- NEVER include plain-text passwords in service files
+- NEVER include plain-text passwords in service files or on the command line
- Use `EnvironmentFile` directive to load from .env
-- Example: `ExecStart=/opt/genlayer-node/bin/genlayernode run --password "${NODE_PASSWORD}"`
+- Example: `ExecStart=/opt/genlayer-node/bin/genlayernode run -c /opt/genlayer-node/configs/node/config.yaml` (the node reads `GENLAYERNODE_PASSWORD` from the environment)
### When User Needs to See Full Value
If user needs the actual secret value:
@@ -1038,7 +1129,7 @@ If user needs the actual secret value:
### Can Offer Options
These can be presented as multiple choice:
-- New validator or existing?
+- Full node, new validator or existing?
- LLM provider selection (Heurist, Comput3, io.net, etc.) - select provider, NOT the key
- Operator key source (import, copy, generate)
- Deployment method (systemd, docker, manual)
diff --git a/plugins/genlayernode/skills/genlayernode/collaboration.yaml b/plugins/genlayernode/skills/genlayernode/collaboration.yaml
index 99de612..de52fdc 100644
--- a/plugins/genlayernode/skills/genlayernode/collaboration.yaml
+++ b/plugins/genlayernode/skills/genlayernode/collaboration.yaml
@@ -5,23 +5,23 @@ version: "1.0.0"
dependencies:
- name: nodejs
- description: "Node.js v18+ must be installed"
- check: "node --version | grep -E 'v(1[89]|[2-9][0-9])'"
+ description: "Node.js v20+ must be installed on the machine that runs the GenLayer CLI (not needed on the node server)"
+ check: "node --version | grep -E 'v(2[0-9]|[3-9][0-9])'"
install: "curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - && sudo apt install -y nodejs"
- name: docker
description: "Docker must be installed and running"
check: "docker info"
- install: "curl -fsSL https://get.docker.com | sh && sudo usermod -aG docker $USER"
+ install: "curl -fsSL https://get.docker.com | env -u VERSION sh && sudo usermod -aG docker $USER"
- name: docker-compose
- description: "Docker Compose must be available"
- check: "docker compose version"
+ description: "Docker Compose v2.24+ must be available (release docker-compose.yaml uses env_file required: false)"
+ check: "docker compose version --short | sed 's/^v//' | awk -F. 'NR==1{ok=($1>2||($1==2&&$2>=24))} END{exit !ok}'"
install: "Usually included with Docker. If missing, install docker-compose-plugin"
- name: python3
- description: "Python 3 must be installed"
- check: "python3 --version"
+ description: "Python 3.10+ must be installed"
+ check: "python3 -c 'import sys; sys.exit(sys.version_info < (3, 10))'"
install: "sudo apt install -y python3"
- name: python3-pip
@@ -31,9 +31,14 @@ dependencies:
- name: python3-venv
description: "python3-venv must be installed for GenVM setup"
- check: "python3 -m venv --help"
+ check: "python3 -c 'import venv, ensurepip'"
install: "sudo apt install -y python3-venv"
+ - name: cosign
+ description: "cosign must be installed to verify the release signature"
+ check: "cosign version"
+ install: "curl -fsSLo /tmp/cosign.deb https://github.com/sigstore/cosign/releases/download/v2.4.1/cosign_2.4.1_amd64.deb && sudo dpkg -i /tmp/cosign.deb && rm /tmp/cosign.deb"
+
- name: wget
description: "wget must be installed for downloading node binaries"
check: "wget --version"
@@ -57,7 +62,7 @@ dependencies:
- name: genlayer-cli
description: "GenLayer CLI is needed for staking wizard and validator management"
check: "genlayer --version"
- install: "npm install -g genlayer"
+ install: "npm install -g genlayer@"
note: "Only required on local machine for staking wizard, not on the validator server"
composition:
diff --git a/plugins/genlayernode/skills/genlayernode/common-procedures.md b/plugins/genlayernode/skills/genlayernode/common-procedures.md
index 65607fa..bba8ba0 100644
--- a/plugins/genlayernode/skills/genlayernode/common-procedures.md
+++ b/plugins/genlayernode/skills/genlayernode/common-procedures.md
@@ -5,36 +5,46 @@ Shared commands and procedures used by both install and update workflows.
## Download & Extract Node Software
```bash
-# Fetch latest version automatically
-VERSION=$(curl -s "https://storage.googleapis.com/storage/v1/b/gh-af/o?prefix=genlayer-node/bin/amd64" | \
- grep -o '"name": *"[^"]*"' | sed -n 's/.*\/\(v[^/]*\)\/.*/\1/p' | sort -Vr | head -1)
-
-# Or set specific version manually
-# VERSION=v0.4.4
-
-# Download tarball to /tmp
-wget https://storage.googleapis.com/gh-af/genlayer-node/bin/amd64/${VERSION}/genlayer-node-linux-amd64-${VERSION}.tar.gz \
- -O /tmp/genlayer-node-${VERSION}.tar.gz
-
-# Create version directory
-sudo mkdir -p /opt/genlayer-node/${VERSION}
-
-# Extract
-sudo tar -xzvf /tmp/genlayer-node-${VERSION}.tar.gz \
- -C /opt/genlayer-node/${VERSION} --strip-components=1
-
-# Set ownership
-sudo chown -R $USER:$USER /opt/genlayer-node/${VERSION}
-```
+# Set the exact version approved for your network (release announcement).
+# Do not take the newest listed version automatically: it may not be approved yet.
+VERSION=v0.6.0
+
+# Download tarball, checksums, signature and install.sh to a temp dir
+# (stop on any 404 — never fall back to an older version)
+base=https://storage.googleapis.com/gh-af/genlayer-node/bin/amd64/${VERSION}
+DL=$(mktemp -d /tmp/genlayer-${VERSION}.XXXX) && chmod 755 "$DL" && cd "$DL"
+wget -q ${base}/genlayer-node-linux-amd64-${VERSION}.tar.gz ${base}/genlayer-node_${VERSION}_SHA256SUMS \
+ ${base}/genlayer-node_${VERSION}_SHA256SUMS.sigstore.json ${base}/install.sh
+
+# Verify before extracting (stop on any failure; the tarball must be listed in SHA256SUMS)
+cosign verify-blob --bundle genlayer-node_${VERSION}_SHA256SUMS.sigstore.json \
+ --certificate-identity-regexp '^https://github\.com/genlayerlabs/genlayer-node/\.github/workflows/release-package\.yml@' \
+ --certificate-oidc-issuer https://token.actions.githubusercontent.com genlayer-node_${VERSION}_SHA256SUMS \
+&& grep -q "[ *]genlayer-node-linux-amd64-${VERSION}.tar.gz$" genlayer-node_${VERSION}_SHA256SUMS \
+&& sha256sum -c genlayer-node_${VERSION}_SHA256SUMS --ignore-missing \
+|| { echo "VERIFICATION FAILED - stop, do not install" >&2; rm -f install.sh; false; } # no install.sh: the install below cannot run
+
+# Create version directory, extract and run the GenVM setup (install.sh), as the service user
+sudo install -d -o genlayer -g genlayer /opt/genlayer-node
+chmod +x install.sh
+sudo -u genlayer -H ./install.sh --install-dir /opt/genlayer-node/${VERSION} \
+ --tarball genlayer-node-linux-amd64-${VERSION}.tar.gz
+```
+
+Details and expected output: `install-procedure.md` Phase 2.
## GenVM Setup
-Downloads GenVM binaries (genvm and genvm-modules).
+`install.sh` runs it. To run it by hand (for example after a failed setup), use the GenVM tree
+that `genvm.root_dir` names, **in the final directory** (it writes absolute paths into the binaries):
```bash
-python3 /opt/genlayer-node/${VERSION}/third_party/genvm/bin/setup.py
+cd /opt/genlayer-node/${VERSION}
+GENVM_ROOT=$(readlink -f "$(sed -n 's/^ *root_dir: *//p' configs/node/config.yaml)")
+sudo -u genlayer python3 ${GENVM_ROOT}/bin/setup.py
```
+It verifies the runners, patches the binaries' interpreter path and precompiles the runners.
**Wait for completion before proceeding.**
## LLM Strategy Selection
@@ -48,19 +58,22 @@ The release tarball includes two LLM strategies:
### Apply LLM Strategy
```bash
+# GenVM config dir of the version being configured (genvm.root_dir in its config.yaml)
+GENVM_CFG=$(cd /opt/genlayer-node/${VERSION} && readlink -f "$(sed -n 's/^ *root_dir: *//p' configs/node/config.yaml)")/config
+
# Step 1: Overlay the release LLM config (includes all backends: openrouter, morpheus, etc.)
-cp /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-modules-llm-release.yaml \
- /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml
+cp ${GENVM_CFG}/genvm-modules-llm-release.yaml \
+ ${GENVM_CFG}/genvm-module-llm.yaml
# Step 2: If greybox strategy, switch the Lua script path
# (skip this step for default strategy)
sed -i 's/genvm-llm-default\.lua/genvm-llm-greybox.lua/' \
- /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml
+ ${GENVM_CFG}/genvm-module-llm.yaml
```
**Verify:**
```bash
-grep lua_script_path /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml
+grep lua_script_path ${GENVM_CFG}/genvm-module-llm.yaml
# default: genvm-llm-default.lua
# greybox: genvm-llm-greybox.lua
```
@@ -96,12 +109,12 @@ For **default** strategy, enable at least one provider.
```bash
# Replace with your provider name from the table above
sed -i '/^ :/,/^ [a-z]/ s/enabled: false/enabled: true/' \
- /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml
+ ${GENVM_CFG}/genvm-module-llm.yaml
```
**Verify it's enabled:**
```bash
-grep -A2 ':' /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml
+grep -A2 ':' ${GENVM_CFG}/genvm-module-llm.yaml
# Should show: enabled: true
```
@@ -117,7 +130,7 @@ The LLM module must be restarted per GenVM instance — there is no atomic resta
```bash
# Each validator runs its own GenVM manager on a different port
# Check the deployed configs for bind addresses:
-grep -r bind_address /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml
+grep -r bind_address ${GENVM_CFG}/genvm-module-llm.yaml
# Or check active processes:
ss -tlnp | grep genvm
```
@@ -125,7 +138,7 @@ ss -tlnp | grep genvm
### Step 2: Update the files
```bash
-GENVM_CONFIG="/opt/genlayer-node/${VERSION}/third_party/genvm/config"
+GENVM_CONFIG="${GENVM_CFG}"
# Update the Lua script (source of truth: release/genvm-llm-greybox.lua)
# Copy the new version to ALL genvm instance config directories
@@ -173,10 +186,13 @@ grep "greybox" /opt/genlayer-node/data/node/logs/node.log | tail -5
## Create Directory Structure
+The tarball ships `configs/node/`. The data directory is shared per minor line and linked from
+each version (see `install-procedure.md` Phase 2):
+
```bash
-# Create data and config directories
-mkdir -p /opt/genlayer-node/${VERSION}/data/node
-mkdir -p /opt/genlayer-node/${VERSION}/configs/node
+MINOR=$(echo ${VERSION} | grep -oE '^v[0-9]+\.[0-9]+')
+sudo -u genlayer mkdir -p /opt/genlayer-node/${MINOR}/data/node/keystore /opt/genlayer-node/${MINOR}/data/node/logs
+sudo -u genlayer ln -sfn ../${MINOR}/data /opt/genlayer-node/${VERSION}/data
```
## Setup Symlinks
@@ -185,14 +201,10 @@ Symlinks provide a stable path (`/opt/genlayer-node/bin`) that points to the cur
```bash
cd /opt/genlayer-node
-ln -sfn ${VERSION}/bin bin
-ln -sfn ${VERSION}/third_party third_party
-ln -sfn ${VERSION}/data data
-ln -sfn ${VERSION}/configs configs
-ln -sfn ${VERSION}/docker-compose.yaml docker-compose.yaml
-ln -sfn ${VERSION}/.env .env
-ln -sfn ${VERSION}/alloy-config.river alloy-config.river
-ln -sfn ${VERSION}/genvm-module-web-docker.yaml genvm-module-web-docker.yaml
+for p in bin opt configs data docker-compose.yaml alloy-config.river alloy-healthcheck.sh genvm-module-web-docker.yaml .env; do
+ sudo -u genlayer ln -sfn ${VERSION}/$p $p
+done
+# 'opt' makes the config's relative genvm.root_dir (./opt/genvm/...) resolve from /opt/genlayer-node
```
> **Note:** `alloy-config.river` ships with the node tarball. It must be symlinked like all other
@@ -200,8 +212,8 @@ ln -sfn ${VERSION}/genvm-module-web-docker.yaml genvm-module-web-docker.yaml
> during an upgrade, restart Alloy to pick up the new config: `docker restart genlayer-node-alloy`
**After symlinks, you can use:**
-- `/opt/genlayer-node/bin/genlayernode` instead of `/opt/genlayer-node/v0.4.4/bin/genlayernode`
-- `/opt/genlayer-node/.env` instead of `/opt/genlayer-node/v0.4.4/.env`
+- `/opt/genlayer-node/bin/genlayernode` instead of `/opt/genlayer-node/v0.6.0/bin/genlayernode`
+- `/opt/genlayer-node/.env` instead of `/opt/genlayer-node/v0.6.0/.env`
## Start WebDriver
@@ -209,14 +221,16 @@ WebDriver is required for web-based contract operations.
```bash
cd /opt/genlayer-node
-docker compose up -d
-
-# Wait for healthy status
-until docker inspect --format='{{.State.Health.Status}}' genlayer-node-webdriver 2>/dev/null | grep -q 'healthy'; do
- echo "Waiting for WebDriver..."
- sleep 2
+sudo docker compose up -d # sudo: .env is mode 600 (secrets)
+sudo docker compose up -d docker-socket-proxy # optional: WebDriver metrics for a binary node
+
+# Wait for healthy status (at most 3 minutes)
+for i in $(seq 1 36); do
+ s=$(sudo docker inspect -f '{{.State.Health.Status}}' genlayer-node-webdriver 2>/dev/null)
+ [ "$s" = healthy ] && break
+ echo "Waiting for WebDriver (${s:-not created})..."; sleep 5
done
-echo "WebDriver is healthy!"
+[ "$s" = healthy ] && echo "WebDriver is healthy!" || sudo docker logs --tail 30 genlayer-node-webdriver
```
**If WebDriver fails to start:**
@@ -228,7 +242,7 @@ docker ps -a | grep webdriver
docker rm -f genlayer-node-webdriver
# Retry
-docker compose up -d
+sudo docker compose up -d
```
## Doctor Check
@@ -236,21 +250,27 @@ docker compose up -d
Verifies all configuration is correct before starting the node.
```bash
-cd /opt/genlayer-node/${VERSION}
-set -a && source .env && set +a
-./bin/genlayernode doctor
+cd /opt/genlayer-node
+sudo -u genlayer bash -c 'set -a; . ./.env; set +a; ./bin/genlayernode doctor'
```
-**Expected output:**
+**Expected output** (example: Testnet Bradbury):
```
-✓ GenLayer Chain RPC: Connected
-✓ GenLayer Chain WebSocket: Connected
-✓ Validator Wallet Configuration: OK
-✓ GenVM Binaries: Found
-✓ WebDriver: Successfully rendered test page
+ ✓ GenLayer Chain RPC: Connected to https://rpc.testnet-chain.genlayer.com
+ ✓ GenLayer Chain WebSocket: Connected to wss://rpc.testnet-chain.genlayer.com
+ ✓ All required config fields are present
+ ✓ Consensus Deployment (admission):
+ ✓ registry 0x3F71AE080BD6270A91dBcc7960d88797828538E0 is sealed
+ ✓ chain 4221, 97 contracts match the trusted manifest
+ ✓ GenVM Binaries: Found at ./opt/genvm/v0.6.0-rc10
+ ✓ LLM Providers: 1 of 6 enabled backends usable
+ ✓ WebDriver: Successfully rendered test page at http://127.0.0.1:4444
All configuration checks passed!
```
+(Validator mode also checks the validator wallet / operator key. `ℹ interpreter outside the genvm
+tree` lines are informational. "you have no fallback" warns that only one LLM provider works.)
+
**Note:** Use `set -a && source .env && set +a` to properly export environment variables. Plain `source .env` does not export them.
## Verification Commands
@@ -267,7 +287,7 @@ curl -s http://localhost:9153/health | jq '.node_version'
**Check sync status:**
```bash
-curl -s http://localhost:9153/health | jq '.checks.validating'
+curl -s http://localhost:9153/metrics | grep -E '^genlayer_node_(synced|blocks_behind|synced_block)'
```
**View logs:**
@@ -299,11 +319,13 @@ Requires=docker.service
[Service]
Type=simple
-User=root
+User=genlayer
+Group=genlayer
WorkingDirectory=/opt/genlayer-node
EnvironmentFile=/opt/genlayer-node/.env
-ExecStart=/opt/genlayer-node/bin/genlayernode run --password ${NODE_PASSWORD}
-ExecStartPost=-/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy 2>/dev/null || true'
+Environment=DOCKER_HOST=tcp://127.0.0.1:2375
+ExecStart=/opt/genlayer-node/bin/genlayernode run -c /opt/genlayer-node/configs/node/config.yaml
+ExecStartPost=-+/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy >/dev/null 2>&1 || true'
Restart=always
RestartSec=10
@@ -314,7 +336,12 @@ WantedBy=multi-user.target
> **Note:** The `ExecStartPost` line automatically restarts the Alloy telemetry container
> whenever the node starts. This is required because Alloy's bind mount to the log directory
> becomes stale when symlinks change during upgrades. The `-` prefix means systemd will
-> ignore failures (e.g., if Alloy isn't installed). See `sharp-edges.yaml` -> `alloy-stale-bind-mount`.
+> ignore failures (e.g., if Alloy isn't installed); the `+` runs it as root, because the
+> `genlayer` service user is not in the docker group. See `sharp-edges.yaml` -> `alloy-stale-bind-mount`.
+>
+> No password is on the command line: in validator mode the node reads `GENLAYERNODE_PASSWORD`
+> from `EnvironmentFile`. `DOCKER_HOST` points at the read-only docker-socket-proxy (WebDriver
+> metrics); remove the line if that proxy is not started.
**Commands:**
```bash
@@ -381,13 +408,14 @@ docker rm -f genlayer-node-webdriver
docker compose up -d
```
-### Database Symlink Error (Fresh Install)
-**Symptom:** "mkdir genlayer.db: file exists" error.
-**Fix:** Remove dangling symlink:
-```bash
-rm /opt/genlayer-node/${VERSION}/data/node/genlayer.db
-```
-Fresh installs don't have a shared database to link to.
+### Database Error: "mkdir data/node/genvm.db: file exists"
+**Symptom:** The node stops at start with `error opening database at "data/node/genvm.db": mkdir ... file exists`.
+**Cause:** `data/node/genvm.db` is a link whose target does not exist from where the node runs —
+typically a pre-v0.6 `genlayer.db` link (the v0.6 node renames `genlayer.db` to `genvm.db` on
+start, link included), seen inside a Docker container where the target is not mounted.
+**Fix:** Use the v0.6 layout: the whole `data/` is the shared link and `data/node/genvm.db` is a
+real directory inside it (`common-procedures.md` -> "Create Directory Structure"). With the node stopped, move the real
+database directory into place instead of linking it.
### Alloy Not Sending Logs After Upgrade
**Symptom:** Node is running and healthy, but Grafana shows no logs/metrics. Validator appears offline in monitoring.
diff --git a/plugins/genlayernode/skills/genlayernode/install-procedure.md b/plugins/genlayernode/skills/genlayernode/install-procedure.md
index 8523355..b10a3e9 100644
--- a/plugins/genlayernode/skills/genlayernode/install-procedure.md
+++ b/plugins/genlayernode/skills/genlayernode/install-procedure.md
@@ -1,7 +1,7 @@
# GenLayer Validator Node Installation Procedure
## Goal
-Install a GenLayer validator node from scratch on a Linux server, ready to sync and validate.
+Install a GenLayer validator node (or a full node) from scratch on a Linux server, ready to sync and validate.
## Process Overview
@@ -14,7 +14,7 @@ flowchart TD
end
subgraph SERVER["VALIDATOR SERVER"]
- D[Phase 2: Download & Extract]
+ D[Phase 2: Download, Verify & Install]
E[Phase 3: GenVM Setup]
F[Phase 4: Environment .env]
G[Phase 5: Node Config yaml]
@@ -38,61 +38,73 @@ Before starting, ensure:
- **RAM**: 16GB+ (32GB recommended)
- **CPU**: 8+ cores (16+ recommended)
- **Disk**: 128GB+ SSD (256GB+ recommended)
-- **OS**: Ubuntu 20.04+, Debian 11+, or compatible
+- **OS**: Ubuntu 22.04+, Debian 12+, or compatible (Python 3.10+)
### Required Software
```bash
-# Check all prerequisites
-node --version # v18+ required
+# Check all prerequisites (on the server)
docker --version # v20.10+ required
docker compose version # Compose plugin required
-python3 --version # Python 3.8+ required
+python3 --version # Python 3.10+ required
pip3 --version # pip3 required
-python3 -m venv --help # venv module required
+python3 -c 'import venv, ensurepip' # venv module required ('python3 -m venv --help' passes without python3-venv)
+cosign version # verifies the release signature
```
-Install missing dependencies:
-```bash
-# Node.js v20
-curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
-sudo apt install -y nodejs
+Node.js v20+ is needed only where the GenLayer CLI runs (the staking wizard, LOCAL machine),
+not on the server.
-# Docker
-curl -fsSL https://get.docker.com | sh
+Install missing dependencies (show the list and ask the user to confirm first):
+```bash
+# Docker ('env -u VERSION': the get.docker.com script reads $VERSION as the Docker version to install)
+curl -fsSL https://get.docker.com | env -u VERSION sh
sudo usermod -aG docker $USER
# Python dependencies
sudo apt install -y python3 python3-pip python3-venv
+
+# cosign (pinned release)
+curl -fsSLo /tmp/cosign.deb https://github.com/sigstore/cosign/releases/download/v2.4.1/cosign_2.4.1_amd64.deb
+sudo dpkg -i /tmp/cosign.deb && rm /tmp/cosign.deb
+
+# Service user for the node — NOT in the docker group (docker-group membership is root-equivalent)
+sudo useradd --system --create-home --home-dir /var/lib/genlayer --shell /usr/sbin/nologin genlayer
```
## Installation Phases
### Phase 1: Staking Wizard (Local Machine)
-**Skip if you already have a validator wallet address and operator keystore.**
+**Skip if you already have a validator wallet address and operator keystore, or for a full node
+(`node.mode: "full"`: no staking, no operator key).**
The staking wizard runs on your **local machine** (not the server) to create wallet keys securely.
```bash
-# Install GenLayer CLI
-npm install -g genlayer
+# Install GenLayer CLI — the exact version named in the release announcement for your network
+npm install -g "genlayer@${CLI_VERSION}"
# Create and unlock owner account
genlayer account create --name "My Validator Owner"
genlayer account unlock --account "My Validator Owner"
-# Fund the owner account with 42,000+ GEN, then run wizard
-genlayer staking wizard --account "My Validator Owner" --network testnet-asimov
+# Fund the owner account with the deployed self-stake minimum + gas, then run wizard
+genlayer staking epoch-info --network testnet-bradbury # deployed minimum (42,000 GEN is the protocol default)
+genlayer staking wizard --account "My Validator Owner" --network testnet-bradbury
+# (--network: your network's CLI alias; testnet-bradbury is the testnet documented today)
```
+Phase 2 registration signs a proof of possession with the operator key: the operator must be an
+account the CLI holds (create it in the wizard). An operator address alone fails.
+
**Save these outputs from the wizard:**
| Item | Example | Used For |
|------|---------|----------|
| Validator Wallet | `0x1451c990...` | config.yaml |
| Operator Address | `0x992fff41...` | config.yaml |
| Keystore File | `Operator-keystore.json` | Upload to server |
-| Export Passphrase | (you set it) | Keystore import |
+| Export Passphrase | (you set it) | `.env` on the server (`GENLAYERNODE_PASSWORD`) |
**Fund the operator after wizard:**
```bash
@@ -103,54 +115,65 @@ genlayer account show --account "My Validator Operator"
# Send GEN from owner to operator (10 GEN minimum recommended)
genlayer account send 0xOPERATOR_ADDRESS 10gen \
--account "My Validator Owner" \
- --network testnet-asimov
+ --network testnet-bradbury
```
**See `staking-wizard-procedure.md` for detailed wizard walkthrough.**
---
-### Phase 2: Download & Extract (Server)
+### Phase 2: Download, Verify & Install (Server)
All remaining steps run on the **validator server**.
> **See `common-procedures.md`** for detailed commands.
```bash
-VERSION=v0.4.4
+VERSION=v0.6.0 # the exact version approved for your network (release announcement)
-# Download, extract, set permissions
+# Download, verify (cosign + SHA256), install with the release install.sh
# See: common-procedures.md -> "Download & Extract Node Software"
-# Create directories
-mkdir -p /opt/genlayer-node/${VERSION}/{data/node,configs/node}
+# Shared data directory (all v0.6.x) and the version's data/ link
+# See: common-procedures.md -> "Create Directory Structure"
# Setup symlinks
# See: common-procedures.md -> "Setup Symlinks"
```
+Check first that the tarball, `genlayer-node_${VERSION}_SHA256SUMS`, its `.sigstore.json` and
+`install.sh` all exist (`curl -fI`): on any 404, stop — never fall back to an older version.
+Release files supplied by the user (a pre-release build) have no `.sigstore.json`: run only the
+`sha256sum -c` check and tell the user the provenance was not verified.
+
### Phase 3: GenVM Setup (Server)
> **See `common-procedures.md` -> "GenVM Setup"**
+`install.sh` (Phase 2) runs it in the final directory. Re-run it only after a failure:
+
```bash
-python3 /opt/genlayer-node/${VERSION}/third_party/genvm/bin/setup.py
+cd /opt/genlayer-node/${VERSION}
+sudo -u genlayer python3 "$(sed -n 's/^ *root_dir: *//p' configs/node/config.yaml)/bin/setup.py"
```
-This downloads GenVM binaries (~2 minutes). Wait for completion.
+This verifies the runners and precompiles them (~2 minutes). Wait for completion. It writes the
+directory's absolute path into the GenVM binaries: never move the directory afterwards.
### Phase 4: Environment Configuration (Server)
```bash
-# Create .env from example
-cp /opt/genlayer-node/${VERSION}/.env.example /opt/genlayer-node/${VERSION}/.env
+# Create .env from example (mode 600: it holds secrets)
+sudo -u genlayer cp /opt/genlayer-node/${VERSION}/.env.example /opt/genlayer-node/${VERSION}/.env
+sudo chmod 600 /opt/genlayer-node/${VERSION}/.env
```
-**Edit .env to add these values:**
+**The user edits .env to add these values** (`sudo nano /opt/genlayer-node/${VERSION}/.env`):
```bash
-# Required: RPC URLs (get from GenLayer team or docs)
-GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCURL=https://your-rpc-url/rpc
-GENLAYERNODE_ROLLUP_GENLAYERCHAINWEBSOCKETURL=wss://your-rpc-url/ws
+# API key of your network's RPC provider, if it needs one (Testnet Bradbury: Conduit,
+# https://app.conduit.xyz/rpc-keys — its WebSocket refuses connections without it).
+# The RPC URLs are in config.yaml (Phase 5).
+GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCKEY=your-rpc-provider-key
# Required: LLM API key (set ONE based on your provider)
# HEURISTKEY=your-api-key-here
@@ -159,86 +182,123 @@ GENLAYERNODE_ROLLUP_GENLAYERCHAINWEBSOCKETURL=wss://your-rpc-url/ws
# OPENROUTERKEY=your-api-key-here
# (see common-procedures.md for full provider list)
-# Required: Node password (min 8 characters)
-NODE_PASSWORD=your-secure-password
+# Validator only: the operator keystore's export passphrase.
+# Binary (systemd) reads GENLAYERNODE_PASSWORD (add the line); Docker reads NODE_PASSWORD.
+GENLAYERNODE_PASSWORD=your-export-passphrase
+
+# Required by the rc.1 docker-compose.yaml for EVERY docker compose command (also the
+# WebDriver in Phase 7): it refuses to run while NODE_PASSWORD is empty.
+# Validator: the same export passphrase. Full node: any non-empty value (no keystore is unlocked).
+NODE_PASSWORD=your-export-passphrase
+
+# Docker only: the exact version, never latest
+NODE_VERSION=v0.6.0
```
+
**Select LLM strategy and enable provider:**
> **See `common-procedures.md` -> "LLM Strategy Selection"** for strategy details.
```bash
+# GenVM config dir = /config
+GENVM_CFG=$(cd /opt/genlayer-node/${VERSION} && readlink -f "$(sed -n 's/^ *root_dir: *//p' configs/node/config.yaml)")/config
+
# Apply release LLM config (includes all backends)
-cp /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-modules-llm-release.yaml \
- /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml
+sudo -u genlayer cp ${GENVM_CFG}/genvm-modules-llm-release.yaml \
+ ${GENVM_CFG}/genvm-module-llm.yaml
# If using greybox strategy (deterministic via OpenRouter), switch lua script:
-sed -i 's/genvm-llm-default\.lua/genvm-llm-greybox.lua/' \
- /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml
+sudo -u genlayer sed -i 's/genvm-llm-default\.lua/genvm-llm-greybox.lua/' \
+ ${GENVM_CFG}/genvm-module-llm.yaml
```
> **See `common-procedures.md` -> "Enable LLM Provider"** for provider mapping table.
```bash
# Enable your provider (replace with provider name)
-sed -i '/^ :/,/^ [a-z]/ s/enabled: false/enabled: true/' \
- /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml
+sudo -u genlayer sed -i '/^ :/,/^ [a-z]/ s/enabled: false/enabled: true/' \
+ ${GENVM_CFG}/genvm-module-llm.yaml
```
### Phase 5: Node Configuration (Server)
+The tarball ships `configs/node/config.yaml` (install.sh already set its `genvm.root_dir`);
+edit it in place. The network values are per-deployment: take them from
+your network's section in the docs ("Network-Specific Consensus Configuration"); for your own network, from its operator.
+Since v0.6 the node also needs the network's **protocol manifest**. Example — Testnet Bradbury:
+
```bash
-# Create config from example
-cp /opt/genlayer-node/${VERSION}/configs/node/config.yaml.example \
- /opt/genlayer-node/${VERSION}/configs/node/config.yaml
+sudo -u genlayer curl -fsSL https://docs.genlayer.com/validators/bradbury.protocol-manifest.json \
+ -o /opt/genlayer-node/${VERSION}/configs/node/protocol-manifest.json
+python3 -c "import json; m=json.load(open('/opt/genlayer-node/${VERSION}/configs/node/protocol-manifest.json')); print(m['chainId'], m['registry'])"
+# Must match consensus.chainid and consensus.consensusaddress (Testnet Bradbury: 4221 0x3F71AE080BD6270A91dBcc7960d88797828538E0)
```
-**Edit config.yaml to set validator addresses:**
+**Edit config.yaml to set the network values, the mode and validator addresses**
+(replace each key in its section; never add a second `rollup:` or `consensus:`). Example — Testnet Bradbury:
```yaml
+rollup:
+ genlayerchainrpcurl: "https://rpc.testnet-chain.genlayer.com/"
+ genlayerchainwebsocketurl: "wss://rpc.testnet-chain.genlayer.com/"
+ provider: "conduit"
+
+consensus:
+ consensusaddress: "0x3F71AE080BD6270A91dBcc7960d88797828538E0"
+ chainid: "4221"
+ manifestpath: "configs/node/protocol-manifest.json"
+ genesis: 47930
+
node:
+ mode: "validator" # or "full" (no addresses needed)
validatorWalletAddress: "0xYOUR_VALIDATOR_WALLET_ADDRESS"
operatorAddress: "0xYOUR_OPERATOR_ADDRESS"
```
-### Phase 6: Operator Key Import (Server)
+
+### Phase 6: Operator Key Import (Server, validator only)
**Option A: Upload keystore.json from wizard**
1. Create keystore directory:
```bash
- mkdir -p /opt/genlayer-node/${VERSION}/data/node/keystore
+ sudo install -d -m 700 -o genlayer -g genlayer /opt/genlayer-node/${VERSION}/data/node/keystore
```
2. Upload the keystore file (from local machine):
```bash
# For GCP:
- gcloud compute scp ./Operator-keystore.json INSTANCE:/opt/genlayer-node/${VERSION}/data/node/ \
+ gcloud compute scp ./Operator-keystore.json INSTANCE:/tmp/ \
--project=PROJECT --zone=ZONE
# For SSH:
- scp ./Operator-keystore.json user@host:/opt/genlayer-node/${VERSION}/data/node/
+ scp ./Operator-keystore.json user@host:/tmp/
+
+ # For AWS with SSM only: upload to an S3 bucket the instance role can read, then on the server
+ # aws s3 cp s3://BUCKET/Operator-keystore.json /tmp/ (delete the S3 object after)
```
-3. Import the keystore (on server):
+3. Install the keystore as-is (on server). It stays encrypted; no password on any command line —
+ the user sets `GENLAYERNODE_PASSWORD` in .env to its export passphrase:
```bash
cd /opt/genlayer-node/${VERSION}
- set -a && source .env && set +a
- ./bin/genlayernode account import \
- --password "$NODE_PASSWORD" \
- --path ./data/node/Operator-keystore.json
- # Enter export passphrase when prompted
+ # Stop if operator.json exists: the keystore is shared by all v0.6.x versions (never overwrite it)
+ sudo test ! -e ./data/node/keystore/operator.json && \
+ sudo install -m 600 -o genlayer -g genlayer /tmp/Operator-keystore.json ./data/node/keystore/operator.json && \
+ sudo shred -u /tmp/Operator-keystore.json
```
4. Verify import:
```bash
- ls -la /opt/genlayer-node/${VERSION}/data/node/keystore/
- # Should show UTC--* file
+ sudo ls -la /opt/genlayer-node/${VERSION}/data/node/keystore/
+ # Should show the keystore file (doctor in Phase 8 checks that it unlocks)
```
-**Option B: Copy from previous installation**
+**Option B: Copy from previous installation** (not needed between v0.6.x versions: the data
+directory, keystore included, is shared)
```bash
-cp -r /opt/genlayer-node/OLD_VERSION/data/node/keystore \
- /opt/genlayer-node/${VERSION}/data/node/
+sudo -u genlayer cp -a /opt/genlayer-node/OLD_VERSION/data/node/keystore/. \
+ /opt/genlayer-node/${VERSION}/data/node/keystore/
```
### Phase 7: Start WebDriver (Server)
@@ -247,12 +307,14 @@ cp -r /opt/genlayer-node/OLD_VERSION/data/node/keystore \
```bash
cd /opt/genlayer-node
-docker compose up -d
+sudo docker compose up -d # sudo: .env is mode 600 (secrets)
+sudo docker compose up -d docker-socket-proxy # optional: WebDriver metrics for a binary node
-# Wait for healthy status
-until docker inspect --format='{{.State.Health.Status}}' genlayer-node-webdriver 2>/dev/null | grep -q 'healthy'; do
+# Wait for healthy status (at most 3 minutes)
+for i in $(seq 1 36); do
+ sudo docker inspect --format='{{.State.Health.Status}}' genlayer-node-webdriver 2>/dev/null | grep -q '^healthy$' && break
echo "Waiting for WebDriver..."
- sleep 2
+ sleep 5
done
```
@@ -262,11 +324,12 @@ done
```bash
cd /opt/genlayer-node/${VERSION}
-set -a && source .env && set +a
-./bin/genlayernode doctor
+sudo -u genlayer bash -c 'set -a && . ./.env && set +a && ./bin/genlayernode doctor'
```
-All checks should pass before proceeding.
+All checks should pass before proceeding: GenLayer Chain RPC + WebSocket, the consensus
+deployment against the manifest (registry sealed, chain id matches), GenVM, at least one LLM provider,
+WebDriver. A full node without an LLM key fails only the LLM check (it still syncs).
### Phase 9: Start Node Service (Server)
@@ -282,11 +345,13 @@ Requires=docker.service
[Service]
Type=simple
-User=root
+User=genlayer
+Group=genlayer
WorkingDirectory=/opt/genlayer-node
EnvironmentFile=/opt/genlayer-node/.env
-ExecStart=/opt/genlayer-node/bin/genlayernode run --password ${NODE_PASSWORD}
-ExecStartPost=-/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy 2>/dev/null || true'
+Environment=DOCKER_HOST=tcp://127.0.0.1:2375
+ExecStart=/opt/genlayer-node/bin/genlayernode run -c /opt/genlayer-node/configs/node/config.yaml
+ExecStartPost=-+/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy >/dev/null 2>&1 || true'
Restart=always
RestartSec=10
@@ -297,7 +362,15 @@ EOF
> **Note:** The `ExecStartPost` automatically restarts the Alloy telemetry container when the
> node starts. This prevents stale bind mount issues where Alloy stops sending logs after
-> upgrades. The `-` prefix means failures are ignored (e.g., if monitoring isn't enabled).
+> upgrades. The `-` prefix means failures are ignored (e.g., if monitoring isn't enabled); the `+`
+> runs it as root, because the `genlayer` service user is not in the docker group.
+>
+> No password is on the command line: the node reads `GENLAYERNODE_PASSWORD` from the
+> `EnvironmentFile`. `DOCKER_HOST` points at the read-only docker-socket-proxy (Phase 7).
+>
+> **Docker Compose instead of systemd:** see SKILL.md -> "Docker Compose Deployment" (run from the
+> version directory; `sudo chown -R 10001:10001 ./data/`; mount the LLM config with
+> `docker-compose.override.yml`).
**Enable and start:**
```bash
@@ -312,16 +385,18 @@ sudo systemctl start genlayer-node
# Check service status
sudo systemctl status genlayer-node
-# Check sync progress (look for "GenVM synced blockNumber=")
-sudo journalctl -u genlayer-node -n 20 --no-pager | grep "GenVM synced"
+# Check sync progress (look for "Node is synced")
+sudo journalctl -u genlayer-node -n 20 --no-pager | grep -E "Node is synced|blockNumber="
# Check health endpoint
-curl -s http://localhost:9153/health | jq .
+curl -s http://localhost:9153/health | jq '{status, network, node_version}'
+curl -s http://localhost:9153/metrics | grep -E '^genlayer_node_(synced|blocks_behind)'
```
**Sync complete when:**
-- `checks.validating.status` = `"up"`
-- No "node is not synced" error
+- `status` = `"healthy"` (rc.1 reports an empty `network` label; the network is confirmed by doctor's
+ "Consensus Deployment (admission)" check)
+- `genlayer_node_synced` = 1 and `genlayer_node_blocks_behind` = 0
---
@@ -331,29 +406,33 @@ curl -s http://localhost:9153/health | jq .
|-------|---------|----------|
| Service running | `systemctl status genlayer-node` | Active (running) |
| WebDriver healthy | `docker ps \| grep webdriver` | healthy |
-| RPC connected | `curl localhost:9153/health` | zksync: up |
-| Syncing | `journalctl -u genlayer-node -f` | "GenVM synced" logs |
+| RPC connected | `curl localhost:9153/health` | status: healthy, network: your network |
+| Syncing | `journalctl -u genlayer-node -f` | "Node is synced" logs |
## Directory Structure After Installation
```
/opt/genlayer-node/
-├── v0.4.4/ # Version directory
+├── v0.6/data/node/ # Shared by all v0.6.x versions
+│ ├── keystore/ # Operator key
+│ ├── genvm.db/ # Chain history (PebbleDB directory)
+│ ├── ops.db/ # This node's state (vote salts, nonces)
+│ └── logs/
+├── v0.6.0/ # Version directory
│ ├── bin/genlayernode # Node binary
-│ ├── third_party/genvm/ # GenVM binaries
-│ ├── data/node/
-│ │ ├── keystore/ # Operator key
-│ │ ├── genlayer.db/ # Database (PebbleDB directory)
-│ │ └── logs/
+│ ├── opt/genvm/ # GenVM binaries
+│ ├── data -> ../v0.6/data # Link to the shared data directory
│ ├── configs/node/config.yaml # Node config
+│ ├── configs/node/protocol-manifest.json
│ ├── .env # Environment variables
│ └── docker-compose.yaml # WebDriver compose
-├── bin -> v0.4.4/bin # Symlinks
-├── data -> v0.4.4/data
-├── configs -> v0.4.4/configs
-├── .env -> v0.4.4/.env
-├── alloy-config.river -> v0.4.4/alloy-config.river # Ships with tarball
-└── genvm-module-web-docker.yaml -> v0.4.4/genvm-module-web-docker.yaml
+├── bin -> v0.6.0/bin # Symlinks
+├── opt -> v0.6.0/opt
+├── data -> v0.6.0/data
+├── configs -> v0.6.0/configs
+├── .env -> v0.6.0/.env
+├── alloy-config.river -> v0.6.0/alloy-config.river # Ships with tarball
+└── genvm-module-web-docker.yaml -> v0.6.0/genvm-module-web-docker.yaml
```
## Common Issues
@@ -365,7 +444,9 @@ curl -s http://localhost:9153/health | jq .
| "module_failed_to_start" | Enable LLM provider in GenVM config |
| WebDriver not starting | `docker rm -f genlayer-node-webdriver && docker compose up -d` |
| Env vars not loaded | Use `set -a && source .env && set +a` |
-| DB symlink error | `rm /opt/genlayer-node/${VERSION}/data/node/genlayer.db` |
+| DB error `mkdir data/node/genvm.db: file exists` | A link inside `data/node` points outside the mount — see sharp-edge `docker-db-symlink` |
+| WebSocket 401 / node exits at start | RPC provider key `GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCKEY` empty or wrong (sharp-edge `conduit-websocket-key-missing`) |
+| Node refuses to start: manifest / chain id | Phase 5 (sharp-edge `protocol-manifest-missing-or-mismatch`) |
## Useful Commands
@@ -379,13 +460,13 @@ sudo journalctl -u genlayer-node -f --no-hostname
curl -s http://localhost:9153/health | jq '.node_version'
# Check sync status
-curl -s http://localhost:9153/health | jq '.checks.validating'
+curl -s http://localhost:9153/metrics | grep -E '^genlayer_node_(synced|blocks_behind)'
```
## Next Steps
After installation completes:
1. Wait for sync to complete (check `journalctl` for progress)
-2. Verify validator is active: `genlayer staking validator-info --validator 0xYOUR_WALLET`
+2. Verify validator is active: `genlayer staking validator-info 0xYOUR_WALLET --network testnet-bradbury`
3. Optional: Enable monitoring — see `monitoring-procedure.md`
4. Optional: Set validator identity: `genlayer staking set-identity`
diff --git a/plugins/genlayernode/skills/genlayernode/monitoring-procedure.md b/plugins/genlayernode/skills/genlayernode/monitoring-procedure.md
index b86751e..9f4e6e0 100644
--- a/plugins/genlayernode/skills/genlayernode/monitoring-procedure.md
+++ b/plugins/genlayernode/skills/genlayernode/monitoring-procedure.md
@@ -8,7 +8,7 @@ Enable centralized monitoring by pushing metrics and logs to GenLayer Foundation
```mermaid
flowchart TD
subgraph Prerequisites
- A[Node Running & Synced
Port 9153 up] --> B[Request Creds
Discord #testnet-asimov]
+ A[Node Running & Synced
Port 9153 up] --> B[Request Creds
GenLayer Foundation team]
B --> C[Receive 4 credentials
Prom + Loki user/pass]
end
@@ -28,14 +28,8 @@ flowchart TD
Before enabling monitoring:
1. **Node must be running** and accessible on ops port (default 9153)
-2. **Metrics endpoint enabled** in config.yaml:
- ```yaml
- node:
- ops:
- port: 9153
- endpoints:
- metrics: true
- ```
+2. **Ops port reachable** (`node.ops.port`, default 9153). In v0.6 the ops server always serves
+ `/metrics`; the `node.ops.endpoints.metrics` switch is compatibility metadata only.
3. **Verify local metrics available:**
```bash
curl -s http://localhost:9153/metrics | head -20
@@ -44,10 +38,12 @@ Before enabling monitoring:
## What Ships With the Node Tarball
-The node tarball (v0.4.4+) **already includes** everything needed for monitoring:
+The node tarball (v0.6) **already includes** everything needed for monitoring:
- **`alloy-config.river`** — Full Alloy configuration for log collection and metrics forwarding
-- **`docker-compose.yaml`** — Includes the Alloy service under the `monitoring` profile
+- **`docker-compose.yaml`** — Includes the Alloy service (and the read-only `docker-socket-proxy`
+ Alloy uses for container discovery) under the `monitoring` profile
+- **`alloy-healthcheck.sh`** — Alloy container health check
- **`.env.example`** — Includes all monitoring environment variable placeholders
**You do NOT need to create these files manually.** The `alloy-config.river` symlink is created
@@ -59,9 +55,9 @@ automatically during installation/upgrade as part of the standard symlink setup
**Credentials must be requested from the GenLayer Foundation team.**
-1. Join the GenLayer Discord
-2. Go to `#testnet-asimov` channel
-3. Request monitoring credentials
+1. Contact the GenLayer Foundation team (GenLayer Discord, validator channels)
+2. Request monitoring endpoints and credentials for your network
+3. See also https://docs.genlayer.com/validators/monitoring
4. You will receive **4 credential values** (Prometheus and Loki have separate auth):
- **Prometheus** instance ID and API key (for metrics)
- **Loki** instance ID and API key (for logs)
@@ -73,13 +69,15 @@ automatically during installation/upgrade as part of the standard symlink setup
Edit `.env` on the server to set the monitoring variables:
```bash
-nano /opt/genlayer-node/.env
+sudo nano /opt/genlayer-node/.env
```
**Set these values:**
```bash
-# Monitoring URLs (usually pre-filled in .env.example)
+# Monitoring URLs — empty in .env.example on purpose (no default endpoint); use the values
+# the Foundation gives you (the two URLs below are an example). Alloy stops with
+# "ERROR: not set: ..." if any of the six is empty.
CENTRAL_MONITORING_URL=https://prometheus-prod-66-prod-us-east-3.grafana.net/api/prom/push
CENTRAL_LOKI_URL=https://logs-prod-042.grafana.net/loki/api/v1/push
@@ -97,8 +95,9 @@ VALIDATOR_NAME=your-validator-name
```
**Important notes:**
-- `NODE_ID` — Use your validator wallet address (e.g., `0x1451c990fa6Fa23Bc2773266Fa022cBb369cE165`)
-- `VALIDATOR_NAME` — A short identifier for your validator (no spaces, e.g., `edgars-asimov`)
+- `NODE_ID` — Use your validator wallet address (e.g., `0x1451c990fa6Fa23Bc2773266Fa022cBb369cE165`);
+ a full node has none — use a stable unique name (e.g., `fullnode-eu-1`)
+- `VALIDATOR_NAME` — A short identifier for your validator (no spaces, e.g., `my-validator-1`)
- URLs should NOT have trailing slashes
- `CENTRAL_MONITORING_USERNAME` and `CENTRAL_LOKI_USERNAME` are typically **different** numeric IDs
- The password (API key) **may** be the same for both, but verify with your Foundation credentials
@@ -117,26 +116,22 @@ ALLOY_SELF_MONITORING_INTERVAL=60s
**CRITICAL**: Always test credentials before starting Alloy to avoid spamming error logs.
```bash
-# Load env vars
-set -a && source /opt/genlayer-node/.env && set +a
+# Reads .env as the service user; credentials go to curl on stdin (-K -), never on a
+# command line (visible in the process list). Prints only HTTP codes.
+sudo -u genlayer bash -c '
+set -a; . /opt/genlayer-node/.env; set +a
# Test Prometheus (metrics) — expect HTTP 400 (auth OK, empty body rejected)
echo "=== Prometheus ==="
-curl -s -o /dev/null -w "HTTP %{http_code}" \
- -X POST \
- -u "${CENTRAL_MONITORING_USERNAME}:${CENTRAL_MONITORING_PASSWORD}" \
- "${CENTRAL_MONITORING_URL}"
-echo ""
+printf "user = \"%s:%s\"\n" "$CENTRAL_MONITORING_USERNAME" "$CENTRAL_MONITORING_PASSWORD" | \
+ curl -s -K - -o /dev/null -w "HTTP %{http_code}\n" -X POST "$CENTRAL_MONITORING_URL"
# Test Loki (logs) — expect HTTP 204 (auth OK, no content)
echo "=== Loki ==="
-curl -s -o /dev/null -w "HTTP %{http_code}" \
- -X POST \
- -u "${CENTRAL_LOKI_USERNAME}:${CENTRAL_LOKI_PASSWORD}" \
- "${CENTRAL_LOKI_URL}" \
- -H "Content-Type: application/json" \
- --data-raw "{}"
-echo ""
+printf "user = \"%s:%s\"\n" "$CENTRAL_LOKI_USERNAME" "$CENTRAL_LOKI_PASSWORD" | \
+ curl -s -K - -o /dev/null -w "HTTP %{http_code}\n" -X POST "$CENTRAL_LOKI_URL" \
+ -H "Content-Type: application/json" --data-raw "{}"
+'
```
**Expected results:**
@@ -298,8 +293,8 @@ CENTRAL_MONITORING_URL=https://prometheus-prod-66.../api/prom/push/
| `CENTRAL_MONITORING_PASSWORD` | Yes | Prometheus API key | `glc_xxxxx...` |
| `CENTRAL_LOKI_USERNAME` | Yes | Loki instance ID (numeric, **different** from Prometheus) | `9876543` |
| `CENTRAL_LOKI_PASSWORD` | Yes | Loki API key | `glc_xxxxx...` |
-| `NODE_ID` | Yes | Validator wallet address | `0x1451c990...` |
-| `VALIDATOR_NAME` | Yes | Short validator name (no spaces) | `edgars-asimov` |
+| `NODE_ID` | Yes | Validator wallet address (full node: a stable unique name) | `0x1451c990...` |
+| `VALIDATOR_NAME` | Yes | Short validator name (no spaces) | `my-validator-1` |
| `NODE_METRICS_ENDPOINT` | No | Node metrics address | `host.docker.internal:9153` |
| `LOG_FILE_PATTERN` | No | Log file glob pattern | `/var/log/genlayer/node*.log` |
| `METRICS_SCRAPE_INTERVAL` | No | How often to scrape metrics | `15s` |
@@ -336,7 +331,7 @@ log path, and **no new logs are sent to Grafana**.
**Solution:** The systemd service should include `ExecStartPost` to auto-restart Alloy:
```ini
-ExecStartPost=-/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy 2>/dev/null || true'
+ExecStartPost=-+/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy 2>/dev/null || true'
```
If your existing systemd service doesn't have this line, add it:
@@ -345,7 +340,7 @@ If your existing systemd service doesn't have this line, add it:
sudo nano /etc/systemd/system/genlayer-node.service
# Add after ExecStart line:
-# ExecStartPost=-/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy 2>/dev/null || true'
+# ExecStartPost=-+/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy 2>/dev/null || true'
# Reload and restart
sudo systemctl daemon-reload
diff --git a/plugins/genlayernode/skills/genlayernode/pre-update-checklist.md b/plugins/genlayernode/skills/genlayernode/pre-update-checklist.md
index eed6cd5..b167fff 100644
--- a/plugins/genlayernode/skills/genlayernode/pre-update-checklist.md
+++ b/plugins/genlayernode/skills/genlayernode/pre-update-checklist.md
@@ -8,8 +8,8 @@ This checklist MUST be executed before starting any validator node update to cat
**Purpose:** Detect if LLM provider is configured but not enabled (edge: llm-provider-not-enabled)
```bash
-# Check which LLM key is set in .env
-grep -E '(HEURISTKEY|COMPUT3KEY|IOINTELLIGENCEKEY|LIBERTAI_API_KEY|ANTHROPICKEY|GEMINIKEY|OPENROUTERKEY|MORPHEUS_API_KEY)=' /opt/genlayer-node/.env | grep -v '^#' | grep -v '=$'
+# Check which LLM key is set in .env — NAMES ONLY, never the values
+sudo awk -F= '/^(HEURISTKEY|COMPUT3KEY|IOINTELLIGENCEKEY|LIBERTAI_API_KEY|ANTHROPICKEY|GEMINIKEY|OPENROUTERKEY|MORPHEUS_API_KEY|OPENAIKEY|XAIKEY)=/ && length($2)>0 {print $1": set"}' /opt/genlayer-node/.env
# Provider mapping:
# HEURISTKEY -> heurist
@@ -22,32 +22,34 @@ grep -E '(HEURISTKEY|COMPUT3KEY|IOINTELLIGENCEKEY|LIBERTAI_API_KEY|ANTHROPICKEY|
# MORPHEUS_API_KEY -> morpheus
# For each non-empty key, check if that provider is enabled
+# GenVM config dir of the running version (genvm.root_dir in config.yaml):
+GENVM_CFG=$(cd /opt/genlayer-node && readlink -f "$(sed -n 's/^ *root_dir: *//p' configs/node/config.yaml)")/config
# Replace with the provider name from mapping above:
-grep -A 2 ':' /opt/genlayer-node/third_party/genvm/config/genvm-module-llm.yaml | grep 'enabled'
+grep -A 2 ':' ${GENVM_CFG}/genvm-module-llm.yaml | grep 'enabled'
# If shows "enabled: false" -> FLAG FOR FIX
```
**Action if detected:** Note which provider needs to be enabled in new version.
-### 1.2 Check Current Database Structure
-**Purpose:** Detect if database is not shared between patch versions (edge: database-not-shared-patch-versions)
+### 1.2 Check Current Data Directory Structure
+**Purpose:** Detect if the data directory is not shared between patch versions (edge: database-not-shared-patch-versions)
```bash
-# Check if current installation has separate databases
-ls -la /opt/genlayer-node/v0.4.*/data/node/genlayer.db 2>/dev/null
-readlink /opt/genlayer-node/v0.4.*/data/node/genlayer.db 2>/dev/null
+# v0.6: each version's data/ should be a link to the shared /opt/genlayer-node//data
+for d in /opt/genlayer-node/v0.6.*/data; do printf '%s -> %s\n' "$d" "$(readlink "$d" || echo 'REAL DIRECTORY')"; done
+ls /opt/genlayer-node/v0.6/data/node/ # genvm.db ops.db keystore logs
-# If both show as directories (not symlinks) -> FLAG FOR FIX
+# If a version has a real data/ directory (not a link) -> FLAG FOR FIX
```
-**Action if detected:** Plan to create shared database structure during update.
+**Action if detected:** Plan to move that data directory to the shared location (node stopped) during the update.
### 1.3 Record Current Sync State
**Purpose:** Baseline to verify new version syncs correctly
```bash
# Record current version and sync block
-curl -s http://localhost:9153/health | jq -r '.node_version, .checks.validating.error'
+curl -s http://localhost:9153/health | jq -r '.status, .network, .node_version'
curl -s http://localhost:9153/metrics | grep 'genlayer_node_synced_block'
```
@@ -76,7 +78,8 @@ grep 'genlayer-node-alloy' /etc/systemd/system/genlayer-node.service
```bash
sudo nano /etc/systemd/system/genlayer-node.service
# Add after ExecStart line:
-# ExecStartPost=-/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy 2>/dev/null || true'
+# ExecStartPost=-+/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy >/dev/null 2>&1 || true'
+# ('+' runs it as root: the node's service user is not in the docker group)
sudo systemctl daemon-reload
```
@@ -97,11 +100,11 @@ df -h /opt/genlayer-node
**Purpose:** Ensure system hasn't degraded since initial install
```bash
-# Python with pip and venv
-python3 --version && pip3 --version && python3 -m venv --help
+# Python 3.10+ with pip and venv (python3-venv: ensurepip must import)
+python3 -c 'import sys; sys.exit(sys.version_info < (3,10))' && python3 -m pip --version && python3 -c 'import venv, ensurepip'
-# Docker
-docker --version && docker compose version
+# Docker, cosign (release signature check)
+docker --version && docker compose version && cosign version | grep GitVersion
```
**Action if missing:** Reinstall prerequisites before proceeding.
@@ -112,26 +115,24 @@ docker --version && docker compose version
**Purpose:** Ensure slow GenVM setup completed successfully (edge: setup-py-failed)
```bash
-ls -la /opt/genlayer-node/${NEW_VERSION}/third_party/genvm/bin/genvm
-ls -la /opt/genlayer-node/${NEW_VERSION}/third_party/genvm/bin/genvm-modules
+NEW_GENVM=$(cd /opt/genlayer-node/${NEW_VERSION} && readlink -f "$(sed -n 's/^ *root_dir: *//p' configs/node/config.yaml)")
+ls -la ${NEW_GENVM}/bin/genvm-modules ${NEW_GENVM}/executor/*/bin/genvm
```
-**Action if missing:** Re-run setup.py, don't proceed to switch.
+**Action if missing:** Re-run setup.py in that directory (`python3 ${NEW_GENVM}/bin/setup.py`, as the service user), don't proceed to switch.
### 3.2 Preserve LLM Strategy in New Version
**Purpose:** Apply release LLM config and preserve current strategy (default or greybox)
```bash
# Apply release LLM config to new version
-cp /opt/genlayer-node/${NEW_VERSION}/third_party/genvm/config/genvm-modules-llm-release.yaml \
- /opt/genlayer-node/${NEW_VERSION}/third_party/genvm/config/genvm-module-llm.yaml
+sudo -u genlayer cp ${NEW_GENVM}/config/genvm-modules-llm-release.yaml ${NEW_GENVM}/config/genvm-module-llm.yaml
# Check current strategy and carry it forward
-grep lua_script_path /opt/genlayer-node/third_party/genvm/config/genvm-module-llm.yaml
+grep lua_script_path ${GENVM_CFG}/genvm-module-llm.yaml
# If shows genvm-llm-greybox.lua, apply greybox to new version:
-grep -q 'genvm-llm-greybox.lua' /opt/genlayer-node/third_party/genvm/config/genvm-module-llm.yaml && \
- sed -i 's/genvm-llm-default\.lua/genvm-llm-greybox.lua/' \
- /opt/genlayer-node/${NEW_VERSION}/third_party/genvm/config/genvm-module-llm.yaml
+grep -q 'genvm-llm-greybox.lua' ${GENVM_CFG}/genvm-module-llm.yaml && \
+ sudo -u genlayer sed -i 's/genvm-llm-default\.lua/genvm-llm-greybox.lua/' ${NEW_GENVM}/config/genvm-module-llm.yaml
```
### 3.3 Check LLM Provider Enabled in New Version
@@ -140,25 +141,24 @@ grep -q 'genvm-llm-greybox.lua' /opt/genlayer-node/third_party/genvm/config/genv
```bash
# Check if your provider is enabled in the new version
# Replace with your provider name (heurist, comput3, ionet, libertai, anthropic, google, openrouter):
-grep -A 2 ':' /opt/genlayer-node/${NEW_VERSION}/third_party/genvm/config/genvm-module-llm.yaml | grep 'enabled'
+grep -A 2 ':' ${NEW_GENVM}/config/genvm-module-llm.yaml | grep 'enabled'
# If disabled, enable it NOW before switching:
-sed -i '/^ :/,/^ [a-z]/ s/enabled: false/enabled: true/' \
- /opt/genlayer-node/${NEW_VERSION}/third_party/genvm/config/genvm-module-llm.yaml
+sudo -u genlayer sed -i '/^ :/,/^ [a-z]/ s/enabled: false/enabled: true/' ${NEW_GENVM}/config/genvm-module-llm.yaml
```
**Action:** Fix BEFORE switching versions.
-### 3.3 Verify Shared Database Structure
-**Purpose:** Ensure new version uses shared DB (edge: database-not-shared-patch-versions)
+### 3.4 Verify Shared Data Directory
+**Purpose:** Ensure new version uses the shared data directory (edge: database-not-shared-patch-versions)
```bash
-# Check that symlink was created correctly
-readlink /opt/genlayer-node/${NEW_VERSION}/data/node/genlayer.db
-# Should point to /opt/genlayer-node/v0.4/data/node/genlayer.db
+# Check that the link was created correctly
+readlink /opt/genlayer-node/${NEW_VERSION}/data
+# Should be ../v0.6/data
-# Verify shared DB exists
-ls -la /opt/genlayer-node/v0.4/data/node/genlayer.db
+# Verify the shared data exists (genvm.db, ops.db, keystore)
+ls -la /opt/genlayer-node/v0.6/data/node/
```
**Action if wrong:** Fix structure before switching.
@@ -179,10 +179,10 @@ docker inspect --format='{{.State.Health.Status}}' genlayer-node-webdriver
**Purpose:** Test new version without committing to it
```bash
-# Temporarily point to new version (without stopping old node)
+# Run from the new version directory (without stopping old node).
+# 'set -a' exports the .env values to doctor (plain 'source' does not).
cd /opt/genlayer-node/${NEW_VERSION}
-source .env
-./bin/genlayernode doctor
+sudo -u genlayer bash -c 'set -a; . ./.env; set +a; ./bin/genlayernode doctor'
```
**Action if fails:** Fix issues before switching.
diff --git a/plugins/genlayernode/skills/genlayernode/sharp-edges.yaml b/plugins/genlayernode/skills/genlayernode/sharp-edges.yaml
index 7a0ac0e..59db89b 100644
--- a/plugins/genlayernode/skills/genlayernode/sharp-edges.yaml
+++ b/plugins/genlayernode/skills/genlayernode/sharp-edges.yaml
@@ -54,13 +54,14 @@ edges:
severity: high
- id: nodejs-not-installed
- description: "Node.js is not installed or version too old"
+ description: "Node.js is not installed or too old on the machine that runs the GenLayer CLI"
detect: |
+ # LOCAL machine (validator mode: staking wizard). The node SERVER does not need Node.js.
node --version
- # Should be v18 or higher
- impact: "Cannot install genlayer CLI."
+ # Should be v20 or higher (22 for a fresh install)
+ impact: "Cannot install the GenLayer CLI or run the staking wizard."
fix: |
- Install Node.js 18+:
+ Install Node.js 20+ (on the LOCAL machine):
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt install -y nodejs
severity: critical
@@ -72,23 +73,47 @@ edges:
docker compose version
impact: "Cannot run WebDriver or Docker-based deployment."
fix: |
- Install Docker:
- curl -fsSL https://get.docker.com | sh
+ Install Docker ('env -u VERSION': the script reads $VERSION as the Docker version):
+ curl -fsSL https://get.docker.com | env -u VERSION sh
sudo usermod -aG docker $USER
# Log out and back in
severity: critical
+ - id: compose-node-password-empty
+ description: "rc.1 docker-compose.yaml refuses every command while NODE_PASSWORD is empty"
+ detect: |
+ cd /opt/genlayer-node && sudo docker compose config -q
+ # "required variable NODE_PASSWORD is missing a value: NODE_PASSWORD is not set ..."
+ impact: "WebDriver (binary node) and the Docker node do not start; the error also blocks a full node, which has no keystore."
+ fix: |
+ The user sets NODE_PASSWORD in .env (sudo nano /opt/genlayer-node/.env):
+ validator = the operator keystore's export passphrase; full node = any non-empty value.
+ Never put the value on a command line. Fixed in the node after rc.1 (genlayer-node#2271).
+ severity: high
+
+ - id: compose-too-old
+ description: "Docker Compose is older than v2.24"
+ detect: |
+ docker compose version --short
+ # Must be 2.24 or higher
+ impact: "The release docker-compose.yaml uses 'env_file: required: false'; older Compose rejects the file, so WebDriver and Docker deployment do not start."
+ fix: |
+ Update the Compose plugin:
+ sudo apt-get update && sudo apt-get install -y docker-compose-plugin
+ severity: high
+
# ============================================================================
# WALLET AND STAKING
# ============================================================================
- id: insufficient-gen-balance
- description: "Wallet has less than 42,000 GEN for staking"
+ description: "Owner wallet has less than the deployed self-stake minimum (+ gas)"
detect: |
- genlayer staking balance --address YOUR_ADDRESS
- impact: "Cannot create validator. Staking wizard will fail."
+ genlayer staking epoch-info --network testnet-bradbury # deployed minimum
+ genlayer account show --account "Owner Name" # owner balance
+ impact: "Cannot create validator (or it never activates). Staking wizard stops at the balance check."
fix: |
- Acquire at least 42,000 GEN tokens:
+ Acquire at least the deployed minimum (42,000 GEN is only the protocol default) plus gas:
- Purchase on supported exchanges
- Transfer from another wallet
- Use testnet faucet (for testnet only)
@@ -108,8 +133,8 @@ edges:
- id: wrong-network-selected
description: "Selected wrong network in staking wizard"
detect: |
- Check the RPC URL in your config matches the network
- you selected during wallet creation.
+ Check the RPC URL, consensusaddress and chainid in config.yaml match the network
+ you selected during wallet creation (values from that network's section in the docs).
impact: "Validator won't be recognized on the correct network."
fix: |
Re-run staking wizard with correct network selection
@@ -130,39 +155,65 @@ edges:
Check network connectivity:
curl -I https://storage.googleapis.com
- Retry download or use alternative mirror.
+ Retry the download. If a release file returns 404, STOP: the version is not published
+ (yet). Never fall back to an older version — it may belong to another network.
severity: high
+ - id: release-verification-failed
+ description: "cosign verify-blob or sha256sum -c fails for the downloaded release"
+ detect: |
+ cosign verify-blob --bundle genlayer-node_${VERSION}_SHA256SUMS.sigstore.json \
+ --certificate-identity-regexp '^https://github\.com/genlayerlabs/genlayer-node/\.github/workflows/release-package\.yml@' \
+ --certificate-oidc-issuer https://token.actions.githubusercontent.com genlayer-node_${VERSION}_SHA256SUMS
+ sha256sum -c genlayer-node_${VERSION}_SHA256SUMS --ignore-missing
+ impact: |
+ The checksum file was not produced by GenLayer's release workflow, or the tarball does not
+ match it. Extraction feeds the GenVM setup script, so a tampered tarball runs code on the server.
+ fix: |
+ Do NOT extract. Delete the download directory, download again, verify again.
+ If it still fails, stop and report it. A locally built (pre-release) tarball has no
+ .sigstore.json: verify sha256 only and tell the user provenance was not verified.
+ severity: critical
+
- id: extraction-failed
description: "Tar extraction failed or incomplete"
detect: |
ls -la ${VERSION}/bin/genlayernode
- ls -la ${VERSION}/third_party/genvm/bin/
+ ls -la ${VERSION}/opt/genvm/
impact: "Missing binaries, node won't start."
fix: |
- Re-download and extract:
- rm -rf ${VERSION}
- tar -xzvf genlayer-node-linux-amd64-${VERSION}.tar.gz -C ./${VERSION}
+ Re-install from the verified tarball (install.sh refuses a non-empty dir; --force wipes it):
+ ./install.sh --install-dir /opt/genlayer-node/${VERSION} --tarball --force
severity: high
- id: setup-py-failed
description: "GenVM setup.py failed to download dependencies"
detect: |
- ls ${VERSION}/third_party/genvm/bin/genvm
- ls ${VERSION}/third_party/genvm/bin/genvm-modules
+ cd /opt/genlayer-node/${VERSION}
+ R=$(sed -n 's/^ *root_dir: *//p' configs/node/config.yaml)
+ ls $R/bin/genvm-modules $R/executor/*/bin/genvm
impact: "GenVM cannot execute intelligent contracts."
fix: |
- Check Python and pip are installed:
- python3 --version
- pip3 --version
-
- Re-run setup:
- # v0.6+: setup.py lives under opt/genvm//bin (there is NO third_party/)
- cd ${VERSION} && python3 ./opt/genvm//bin/setup.py --interactive false --os linux --arch amd64
- # older layout:
- cd ${VERSION} && python3 ./third_party/genvm/bin/setup.py
+ Check Python 3.10+ with pip and venv (ensurepip) are installed:
+ python3 --version; python3 -m pip --version; python3 -c 'import venv, ensurepip'
+
+ Re-run setup IN THE FINAL DIRECTORY, as the service user:
+ cd /opt/genlayer-node/${VERSION} && sudo -u genlayer python3 $R/bin/setup.py
+ (older layout: python3 ./third_party/genvm/bin/setup.py)
severity: critical
+ - id: setup-py-final-path
+ description: "GenVM set up in one directory, then the directory was moved -> exec fails ENOENT"
+ detect: |
+ # 'fork/exec .../genvm-modules: no such file or directory' although the file exists:
+ # the ENOENT is the binary's INTERPRETER path, which setup.py baked in absolute.
+ readelf -l opt/genvm/*/bin/genvm-modules | grep interpreter
+ impact: "Node/doctor cannot start GenVM; the error points at the binary, not at the stale interpreter path."
+ fix: |
+ Never move a set-up version directory. Re-run setup.py in place (setup-py-failed fix);
+ install.sh always runs it in the final --install-dir.
+ severity: high
+
- id: genvm-macos-executor-in-linux-tarball
description: "linux-amd64 tarball ships a macOS (Mach-O) GenVM executor -> Exec format error"
detect: |
@@ -224,20 +275,61 @@ edges:
grep -E "GENLAYERNODE_CONSENSUS_CONSENSUSADDRESS|GENLAYERNODE_CONSENSUS_GENESIS" .env 2>/dev/null
impact: "Node cannot interact with consensus, or scans from block 0 (very slow) if genesis is wrong/missing."
fix: |
- The AddressManager address AND genesis block are PER-DEPLOYMENT — there is no
- single correct value. Get them from the deployment you are joining:
- - asimov / bradbury: from official docs or a working peer's config
- - custom / pre-release deployments: from the release's shipped config.yaml
- v0.6+ sets them in configs/node/config.yaml under `consensus:`
+ The AddressManager address, chain id and genesis block are PER-DEPLOYMENT — there is no
+ single correct value. Take them from the official docs for the network you join
+ ("Network-Specific Consensus Configuration"); for your own network, from its operator.
+ v0.6+ sets them in configs/node/config.yaml under `consensus:`. Example — Testnet Bradbury:
consensus:
- consensusaddress: "0x...." # AddressManager for THIS deployment
- genesis: # AddressManager deployment block (pin it!)
- A missing/wrong `genesis` makes the node auto-scan from block 0 of a ~14M-block
- chain (synced_block stuck at 0). Confirm the address resolves at runtime:
- /health should report the intended `network`, and `genlayer_node_info` should show
- the expected address_manager + staking_address.
+ consensusaddress: "0x3F71AE080BD6270A91dBcc7960d88797828538E0"
+ chainid: "4221" # DECIMAL, not hex
+ manifestpath: "configs/node/protocol-manifest.json"
+ genesis: 47930 # AddressManager deployment block
+ v0.6 refuses to start when the manifest's registry/chain do not match these values.
+ A missing/wrong `genesis` makes the node auto-scan from block 0. Confirm at runtime:
+ doctor "Consensus Deployment (admission)" passes (rc.1 /health reports an empty `network`).
+ severity: critical
+
+ - id: protocol-manifest-missing-or-mismatch
+ description: "v0.6+ node refuses to start: protocol manifest missing, or its registry/chain do not match config"
+ detect: |
+ ls -la /opt/genlayer-node/configs/node/protocol-manifest.json
+ python3 -c "import json; m=json.load(open('/opt/genlayer-node/configs/node/protocol-manifest.json')); print(m['chainId'], m['registry'])"
+ grep -E '^ (consensusaddress|chainid|manifestpath):' /opt/genlayer-node/configs/node/config.yaml
+ # doctor: "Consensus Deployment (admission)" fails; logs: manifest / admission errors at start
+ impact: "The node does not start (deployment admission check). The release package ships NO network files."
+ fix: |
+ Download the network's manifest from its docs section to the path manifestpath names,
+ e.g. Testnet Bradbury:
+ curl -fsSL https://docs.genlayer.com/validators/bradbury.protocol-manifest.json \
+ -o /opt/genlayer-node//configs/node/protocol-manifest.json
+ Its chainId and registry must match consensus.chainid (decimal) and consensus.consensusaddress
+ (Testnet Bradbury: 4221, 0x3F71AE080BD6270A91dBcc7960d88797828538E0). Never hand-write a manifest.
+ Docker: the rc.1 compose file does not mount it — mount it with docker-compose.override.yml
+ (SKILL.md -> Docker Compose Deployment).
severity: critical
+ - id: conduit-websocket-key-missing
+ description: "The network's RPC provider refuses the WebSocket without its API key (HTTP 401) — e.g. Conduit on Testnet Bradbury"
+ detect: |
+ sudo awk -F= '/^GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCKEY=/{print (length($2)>0?"set":"EMPTY")}' /opt/genlayer-node/.env
+ # doctor: "GenLayer Chain WebSocket" fails; node exits at start (the WebSocket dial is fatal, in every mode)
+ impact: "Full nodes and validators both fail to start. The HTTP RPC answers without a key, which hides the problem."
+ fix: |
+ The USER creates a key with the RPC provider (Testnet Bradbury: https://app.conduit.xyz/rpc-keys)
+ and sets it in the node's .env:
+ GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCKEY=
+ The node sends it as X-AccessKey (header name: rollup.genlayerchainrpckeyheader).
+ severity: critical
+
+ - id: rpc-key-in-url
+ description: "The RPC provider API key is put in the RPC/WebSocket URL or in config.yaml"
+ detect: |
+ grep -nE 'genlayerchain(rpc|websocket)url:.*(key|token|[?&]|/[0-9a-f]{24,})' /opt/genlayer-node/configs/node/config.yaml
+ impact: "A key in a URL ends up in logs, metrics labels and error texts."
+ fix: |
+ Keep the URLs key-free; set only GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCKEY in .env (mode 600).
+ severity: high
+
- id: rpc-url-unreachable
description: "RPC or WebSocket URL is not accessible"
detect: |
@@ -254,12 +346,12 @@ edges:
- id: shared-rpc-outage
description: "Configured rollup RPC or WebSocket endpoint is experiencing a shared outage"
detect: |
- # HTTP RPC smoke tests against the configured endpoint
- source /opt/genlayer-node/.env 2>/dev/null || source .env
- curl -sS -m 10 -X POST "$GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCURL" \
+ # HTTP RPC smoke tests against the configured endpoint (v0.6: URL is in config.yaml)
+ U=$(sed -n 's/^ genlayerchainrpcurl: *"\([^"]*\)".*/\1/p' /opt/genlayer-node/configs/node/config.yaml)
+ curl -sS -m 10 -X POST "$U" \
-H "Content-Type: application/json" \
-d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}'
- curl -sS -m 10 -X POST "$GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCURL" \
+ curl -sS -m 10 -X POST "$U" \
-H "Content-Type: application/json" \
-d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":2}'
@@ -337,14 +429,14 @@ edges:
- id: no-llm-key-set
description: "No LLM provider API key configured"
detect: |
- env | grep -E "(HEURISTKEY|COMPUT3KEY|IOINTELLIGENCE_API_KEY)"
- impact: "Node cannot execute intelligent contracts."
+ # Presence only — never print the values
+ sudo awk -F= '/^(HEURISTKEY|COMPUT3KEY|IOINTELLIGENCEKEY|OPENROUTERKEY|ANTHROPICKEY|GEMINIKEY|XAIKEY|OPENAIKEY|MORPHEUS_API_KEY|LIBERTAI_API_KEY|ATOMAKEY)=/{print $1": "(length($2)>0?"set":"EMPTY")}' /opt/genlayer-node/.env
+ impact: |
+ A validator cannot execute LLM contracts (votes Timeout on them). A full node still syncs
+ (replay never calls the LLM), but doctor fails and gen_call on LLM contracts fails.
fix: |
- Set at least one LLM API key:
- export HEURISTKEY='your_key'
-
- Add to shell profile for persistence:
- echo "export HEURISTKEY='your_key'" >> ~/.bashrc
+ The USER sets at least one provider key in /opt/genlayer-node/.env (mode 600), e.g.
+ `sudo nano /opt/genlayer-node/.env`. Never in a shell profile, never on a command line.
severity: critical
- id: llm-key-invalid
@@ -361,8 +453,8 @@ edges:
description: "LLM provider is configured in .env but not enabled in GenVM config"
detect: |
# PROACTIVE CHECK (before switching versions):
- # 1. Check which LLM key is set in .env
- grep -E '(HEURISTKEY|COMPUT3KEY|IOINTELLIGENCEKEY|LIBERTAI_API_KEY|ANTHROPICKEY|GEMINIKEY|OPENROUTERKEY|MORPHEUS_API_KEY)=' .env | grep -v '^#' | grep -v '=$'
+ # 1. Check which LLM key is set in .env — NAMES ONLY, never the values
+ sudo awk -F= '/^(HEURISTKEY|COMPUT3KEY|IOINTELLIGENCEKEY|LIBERTAI_API_KEY|ANTHROPICKEY|GEMINIKEY|OPENROUTERKEY|MORPHEUS_API_KEY)=/ && length($2)>0 {print $1}' .env
# 2. For each set key, check if that provider is enabled in GenVM config
# Provider name mapping:
@@ -374,13 +466,19 @@ edges:
# GEMINIKEY -> google
# OPENROUTERKEY -> openrouter
# MORPHEUS_API_KEY -> morpheus
- grep -A 2 ':' third_party/genvm/config/genvm-module-llm.yaml | grep 'enabled'
+ GENVM_CFG=$(readlink -f "$(sed -n 's/^ *root_dir: *//p' configs/node/config.yaml)")/config
+ grep -A 2 ':' ${GENVM_CFG}/genvm-module-llm.yaml | grep 'enabled'
+ # Or simply: doctor lists every enabled backend and whether its key is set.
# REACTIVE DETECTION (after node fails to bootstrap):
# Node logs show: "failed to start module via manager error=\"starting module Llm: manager request failed: 500"
journalctl -u genlayer-node | grep -i "module_failed_to_start\|starting module Llm"
impact: |
- CRITICAL BOOTSTRAP FAILURE - NODE NEVER BECOMES OPERATIONAL
+ v0.6: the node starts in DEGRADED mode — the LLM module may start with no usable backend —
+ and votes Timeout on every transaction that needs an LLM (doctor: "0 of N enabled backends
+ are usable"). A validator then misses those votes until fixed.
+
+ v0.5 and older: CRITICAL BOOTSTRAP FAILURE - NODE NEVER BECOMES OPERATIONAL
This is NOT a runtime failure - this is a COMPLETE BOOTSTRAP FAILURE:
- Node NEVER successfully starts
@@ -426,16 +524,16 @@ edges:
================================================================
After extracting new version, BEFORE switching symlinks:
- # 1. Identify which provider is configured in .env
- grep -E '(HEURISTKEY|COMPUT3KEY|IOINTELLIGENCEKEY|LIBERTAI_API_KEY|ANTHROPICKEY|GEMINIKEY|OPENROUTERKEY|MORPHEUS_API_KEY)=' /opt/genlayer-node/.env | grep -v '^#' | grep -v '=$'
+ # 1. Identify which provider is configured in .env (names only)
+ sudo awk -F= '/^(HEURISTKEY|COMPUT3KEY|IOINTELLIGENCEKEY|LIBERTAI_API_KEY|ANTHROPICKEY|GEMINIKEY|OPENROUTERKEY|MORPHEUS_API_KEY)=/ && length($2)>0 {print $1}' /opt/genlayer-node/.env
- # 2. Enable the corresponding provider in GenVM config
- # Replace with the provider name (heurist, comput3, ionet, libertai, anthropic, google)
- sed -i '/^ :/,/^ [a-z]/ s/enabled: false/enabled: true/' \
- /opt/genlayer-node/${NEW_VERSION}/third_party/genvm/config/genvm-module-llm.yaml
+ # 2. Enable the corresponding provider in the NEW version's GenVM config
+ # Replace with the provider name (heurist, comput3, ionet, libertai, anthropic, google, openrouter, morpheus)
+ NEW_CFG=$(cd /opt/genlayer-node/${NEW_VERSION} && readlink -f "$(sed -n 's/^ *root_dir: *//p' configs/node/config.yaml)")/config
+ sudo -u genlayer sed -i '/^ :/,/^ [a-z]/ s/enabled: false/enabled: true/' ${NEW_CFG}/genvm-module-llm.yaml
# 3. Verify it's enabled
- grep -A 2 ':' /opt/genlayer-node/${NEW_VERSION}/third_party/genvm/config/genvm-module-llm.yaml
+ grep -A 2 ':' ${NEW_CFG}/genvm-module-llm.yaml
# Should show: enabled: true
REACTIVE FIX (after node failed to bootstrap):
@@ -444,11 +542,11 @@ edges:
sudo systemctl stop genlayer-node
2. Enable the provider in GenVM config:
- sed -i '/^ :/,/^ [a-z]/ s/enabled: false/enabled: true/' \
- /opt/genlayer-node/third_party/genvm/config/genvm-module-llm.yaml
+ CFG=$(cd /opt/genlayer-node && readlink -f "$(sed -n 's/^ *root_dir: *//p' configs/node/config.yaml)")/config
+ sudo -u genlayer sed -i '/^ :/,/^ [a-z]/ s/enabled: false/enabled: true/' ${CFG}/genvm-module-llm.yaml
3. Verify the change:
- grep -A 2 ':' /opt/genlayer-node/third_party/genvm/config/genvm-module-llm.yaml
+ grep -A 2 ':' ${CFG}/genvm-module-llm.yaml
# Should show: enabled: true
4. Restart the node:
@@ -461,7 +559,7 @@ edges:
6. Verify node is operational:
curl -s http://localhost:9153/health | jq '.status'
- # Should show: "up" or "down" (syncing) but NOT connection refused
+ # v0.6: "healthy" (or a degraded state while catching up) — NOT connection refused
severity: critical
when_to_check: |
MANDATORY CHECKS:
@@ -498,10 +596,10 @@ edges:
impact: "Web module cannot execute, contracts fail."
fix: |
Start WebDriver:
- docker compose up -d
+ cd /opt/genlayer-node && sudo docker compose up -d
Check logs:
- docker logs genlayer-node-webdriver
+ sudo docker logs genlayer-node-webdriver
severity: high
- id: node-not-syncing
@@ -512,9 +610,9 @@ edges:
impact: "Validator cannot participate in consensus."
fix: |
Check:
- 1. RPC URL is correct and accessible
- 2. Network has sufficient peers
- 3. Firewall allows outbound connections
+ 1. RPC URL is correct and accessible; the RPC provider key is set if needed (WebSocket 401 otherwise)
+ 2. consensus.genesis is pinned (the deployment's AddressManager block; Testnet Bradbury: 47930)
+ 3. Firewall allows outbound HTTPS/WSS
Check logs for errors.
severity: high
@@ -555,6 +653,56 @@ edges:
See create-systemd-service pattern.
severity: high
+ - id: node-user-in-docker-group
+ description: "The node runs as root or as a user in the docker group, or gets the Docker socket"
+ detect: |
+ systemctl show -p User genlayer-node
+ id -nG genlayer | grep -w docker
+ impact: "Docker-group membership is root-equivalent: a node compromise becomes host root."
+ fix: |
+ Run the node as the system user genlayer (not in the docker group). For WebDriver metrics,
+ start docker-socket-proxy (read-only, loopback 2375) and set DOCKER_HOST=tcp://127.0.0.1:2375
+ only in the node's unit — not in a shell you also use for docker commands.
+ severity: high
+
+ - id: two-writers-on-shared-data
+ description: "Two node processes open the same data directory, or two nodes use the same operator"
+ detect: |
+ pgrep -af 'genlayernode run'
+ sudo docker ps --format '{{.Names}}' | grep -x genlayer-node
+ # the second process fails to open data/node/genvm.db or ops.db
+ impact: |
+ Pebble allows one writer: the second process cannot open the databases. ops.db holds this
+ node's vote salts and its operator's nonces and is stamped to node.operatorAddress: two nodes
+ with the same operator, or a copied ops.db, break that record (missed reveals).
+ fix: |
+ Stop the old node before starting the new one (update Phase 2). Never copy ops.db to another
+ node; never run two nodes with the same operator.
+ severity: critical
+
+ - id: docker-default-llm-config
+ description: "Docker node uses the image's default LLM config, not the one configured on the host"
+ detect: |
+ cd /opt/genlayer-node/${VERSION} && sudo docker compose -p genlayer-node --profile node run --rm --entrypoint /app/bin/genlayernode genlayer-node doctor | grep -A8 'LLM Providers'
+ # "0 of 4 enabled backends are usable" with only (anthropic, google, heurist, xai) listed
+ impact: "Doctor fails; the node starts degraded and votes Timeout on LLM transactions."
+ fix: |
+ Mount the host's genvm-module-llm.yaml into the container with docker-compose.override.yml
+ (SKILL.md -> Docker Compose Deployment; rc.1 path /app/opt/genvm//config/),
+ or set a key for one of the image defaults.
+ severity: high
+
+ - id: docker-data-not-owned-by-10001
+ description: "Docker node cannot write ./data (the container runs as uid 10001)"
+ detect: |
+ sudo docker logs genlayer-node 2>&1 | grep -i 'permission denied'
+ stat -c '%u' /opt/genlayer-node/${VERSION}/data/node
+ impact: "The node container exits at start."
+ fix: |
+ sudo chown -R 10001:10001 /opt/genlayer-node/${VERSION}/data/
+ (trailing slash: data is a link; chown -R on the link name changes only the link)
+ severity: high
+
# ============================================================================
# SYSTEMD SERVICE (from infrastructure patterns)
# ============================================================================
@@ -579,9 +727,9 @@ edges:
[ -z "$GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCURL" ] && echo "not set"
impact: "Node fails to connect to rollup or consensus contracts."
fix: |
- For manual runs, source the .env file first:
- source /opt/genlayer-node/.env
- ./bin/genlayernode run --password "$NODE_PASSWORD"
+ For manual runs, EXPORT the .env values first (plain `source` does not export):
+ cd /opt/genlayer-node && sudo -u genlayer bash -c 'set -a; . ./.env; set +a; ./bin/genlayernode run -c $(pwd)/configs/node/config.yaml'
+ (validator mode reads GENLAYERNODE_PASSWORD from the environment — never --password)
For systemd, ensure EnvironmentFile is set in service file.
severity: high
@@ -607,15 +755,13 @@ edges:
- id: data-directory-permissions
description: "Data directory has wrong ownership"
detect: |
- ls -la /opt/genlayer-node/data/
- # Should be owned by the user running the node
+ ls -la /opt/genlayer-node/data/node/
+ # binary/systemd: owned by genlayer; Docker: owned by uid 10001 (the container user)
impact: "Node cannot write to data directory, fails to start."
fix: |
- Fix ownership:
- sudo chown -R $USER:$USER /opt/genlayer-node/data/
-
- Or for systemd service user:
- sudo chown -R ansible:ansible /opt/genlayer-node/data/
+ Fix ownership (trailing slash: data/ is a link and chown -R on a link name changes only the link):
+ sudo chown -R genlayer:genlayer /opt/genlayer-node/data/ # binary/systemd
+ sudo chown -R 10001:10001 /opt/genlayer-node/${VERSION}/data/ # Docker
severity: high
- id: webdriver-not-healthy
@@ -629,22 +775,25 @@ edges:
docker logs genlayer-node-webdriver
Restart if needed:
- docker compose down
- docker compose up -d
+ sudo docker compose down
+ sudo docker compose up -d
- Wait for healthy:
- until docker inspect --format='{{.State.Health.Status}}' genlayer-node-webdriver | grep -q 'healthy'; do sleep 2; done
+ Wait for healthy (at most 3 minutes):
+ for i in $(seq 1 36); do sudo docker inspect -f '{{.State.Health.Status}}' genlayer-node-webdriver | grep -q '^healthy$' && break; sleep 5; done
severity: high
- id: missing-python-packages
- description: "Missing python3-pip or python3-venv packages"
+ description: "Python older than 3.10, or python3-pip / python3-venv missing"
detect: |
- dpkg -l | grep -E "python3-pip|python3-venv"
- impact: "GenVM setup.py fails to install dependencies."
+ python3 -c 'import sys; print(sys.version); sys.exit(sys.version_info < (3,10))'
+ python3 -m pip --version
+ python3 -c 'import venv, ensurepip' # 'python3 -m venv --help' passes WITHOUT python3-venv
+ impact: "GenVM setup.py fails to create its venv / install dependencies."
fix: |
Install required Python packages:
sudo apt update
sudo apt install -y python3 python3-pip python3-venv
+ Python < 3.10 (Ubuntu 20.04, Debian 11): upgrade the OS or install a newer python3.
severity: high
- id: new-operator-not-funded
@@ -655,12 +804,12 @@ edges:
journalctl -u genlayer-node | grep -i "insufficient funds"
impact: "Node cannot submit transactions (activate, propose, vote). Validator will miss duties."
fix: |
- 1. Get operator address from node logs or keystore
- 2. Send ETH to the operator address for gas fees
- 3. Recommended minimum: 0.1 ETH for initial operations
+ 1. Get the operator address (config.yaml node.operatorAddress, or the keystore's "address" field)
+ 2. Send GEN to the operator address for gas fees
+ 3. Recommended minimum: 10 GEN for initial operations
To find operator address:
- cat /opt/genlayer-node/data/node/keystore/*/address
+ grep operatorAddress /opt/genlayer-node/configs/node/config.yaml
Or check logs:
journalctl -u genlayer-node | grep "operator"
@@ -681,8 +830,9 @@ edges:
- Find your validator
- Update operator address to the new one
- Option 2: Using genlayer CLI
- genlayer staking update-operator --validator YOUR_VALIDATOR_ADDRESS --new-operator NEW_OPERATOR_ADDRESS
+ Option 2: Using genlayer CLI (signed by the OWNER; Phase 2 also needs the new operator's
+ proof of possession: --operator-account names its CLI account, CLI v0.40.0-rc.3+)
+ genlayer staking set-operator YOUR_VALIDATOR_WALLET NEW_OPERATOR_ADDRESS --account "Owner Name" --operator-account "Operator Name" --network testnet-bradbury
severity: critical
- id: consensus-genesis-not-set
@@ -690,17 +840,17 @@ edges:
detect: |
# v0.6+: in config.yaml under consensus:
grep -E "^\s*genesis:" configs/node/config.yaml
- # Symptom: synced_block stays 0 while latest_block is ~14M
+ # Symptom: synced_block stays 0 while the chain head is far ahead
curl -s http://localhost:9153/metrics | grep -E "genlayer_node_synced|blocks_behind"
impact: |
Without a pinned genesis the node auto-detects by scanning from block 0 of the
- settlement chain (~14M blocks); synced_block stays 0 and sync never catches up.
+ settlement chain; on a long chain synced_block stays 0 for a long time.
NOTE: genesis is NOT "0" — it is the AddressManager DEPLOYMENT block for the
specific deployment.
fix: |
Pin the deployment's AddressManager deploy block in config.yaml:
consensus:
- genesis: # the AddressManager deployment block for THIS deployment
+ genesis: # the AddressManager deployment block for THIS deployment (Testnet Bradbury: 47930)
Restart. With genesis pinned the node catches up to head in seconds.
severity: high
@@ -711,34 +861,28 @@ edges:
ls /opt/genlayer-node/genlayer-node-*/
impact: "Binaries at wrong path, commands fail."
fix: |
- Use --strip-components=1 when extracting:
- sudo tar -xzvf genlayer-node.tar.gz -C /opt/genlayer-node --strip-components=1
+ Use install.sh (it extracts with --strip-components=1), or when extracting by hand:
+ sudo tar -xzvf genlayer-node.tar.gz -C /opt/genlayer-node/${VERSION} --strip-components=1
Or move files to correct location:
sudo mv /opt/genlayer-node/genlayer-node-*/* /opt/genlayer-node/
severity: medium
- id: docker-db-symlink
- description: "Docker deploy fails with PebbleDB error because DB symlink points outside container mount"
+ description: "Docker node fails to open its DB because a link INSIDE data/node points outside the mount"
detect: |
- # Check if docker node is crash-looping with genlayer.db error
- docker logs genlayer-node 2>&1 | grep "genlayer.db"
- # Check if genlayer.db is a symlink (shouldn't be for docker deploys)
- ls -la /opt/genlayer-node/${VERSION}/data/node/genlayer.db
+ # v0.6 message (the node renames a genlayer.db link to genvm.db on start, link included):
+ sudo docker logs genlayer-node 2>&1 | grep -E 'error opening database at "data/node/(genvm|genlayer).db"'
+ ls -la /opt/genlayer-node/${VERSION}/data/node/ # genvm.db / genlayer.db must not be links
impact: |
- Docker container cannot follow symlinks that point outside the mounted volume.
- The node will crash with "mkdir data/node/genlayer.db: file exists" or similar
- PebbleDB error because the symlink target is not accessible inside the container.
+ Docker cannot follow a link whose target is outside the mounted volume. The node stops
+ with "mkdir data/node/genvm.db: file exists" and restarts in a loop.
fix: |
- Replace the symlink with a real directory:
- rm /opt/genlayer-node/${VERSION}/data/node/genlayer.db
- mkdir -p /opt/genlayer-node/${VERSION}/data/node/genlayer.db
-
- Then restart the container:
- docker compose --profile node up -d
-
- PREVENTION: When deploying as docker, skip the DB symlink creation step.
- The database should live directly in the version directory for docker deploys.
+ With the node stopped, put the real database directory in place of the link
+ (mv the link's target over it; do NOT create an empty directory — that resyncs and loses
+ ops.db). Then use the v0.6 layout (common-procedures.md -> Create Directory Structure): the whole data/ is the
+ shared link — a link as the mount SOURCE is resolved by Docker on the host and works;
+ links inside the mounted tree do not.
severity: high
- id: docker-webdriver-host-ssrf-address-forbidden
@@ -777,17 +921,15 @@ edges:
description: "`doctor check` and an unexported .env break the v0.6 doctor command"
detect: |
./bin/genlayernode doctor --help
- # v0.6: subcommand is just `doctor` (NO `check`); flags include -N and --expected-consensus-address
+ # v0.6: subcommand is just `doctor` (NO `check`)
impact: |
`doctor check` errors ('accepts 0 arg(s), received 1'). And plain `source .env` does NOT
- export vars, so doctor reports LLM keys as 'not set' even when present. For custom/pre-release
- deployments the built-in network registry won't recognize the consensus address.
+ export vars, so doctor reports LLM keys / the RPC key as 'not set' even when present.
fix: |
cd /opt/genlayer-node
- set -a; source .env; set +a # EXPORT env so doctor sees LLM keys
- ./bin/genlayernode doctor --expected-consensus-address 0x
- # -N asimov|bradbury only covers built-in networks; use --expected-consensus-address for
- # custom / pre-release deployments.
+ sudo -u genlayer bash -c 'set -a; . ./.env; set +a; ./bin/genlayernode doctor'
+ # Doctor checks the consensus deployment against consensus.consensusaddress / chainid and
+ # the protocol manifest from config.yaml (no extra flag needed).
severity: medium
- id: full-node-mode-vs-validator
@@ -798,13 +940,14 @@ edges:
impact: |
A full node syncs and serves RPC but does NOT participate in consensus, so it needs no
validator wallet, operator address, keystore, staking, or gas funding (logs: 'full node
- mode detected, skipping validator setup'). It DOES still need an LLM provider + genvm
- config (for gen_call). Running the staking wizard / importing an operator keystore for a
- full node is wasted effort.
+ mode detected, skipping validator setup'). It syncs without an LLM key (replay never calls
+ the LLM), but doctor fails and gen_call on LLM contracts fails without one. It DOES need the
+ RPC provider key when the network's provider requires one (the WebSocket dial at boot is fatal without it). Running the staking wizard /
+ importing an operator keystore for a full node is wasted effort.
fix: |
For a full node: set node.mode: "full", leave validatorWalletAddress/operatorAddress empty,
- and skip the staking wizard + keystore import entirely. Still configure RPC/WS, the
- consensus address + genesis, and one LLM provider key.
+ and skip the staking wizard + keystore import entirely. Still configure RPC/WS + the RPC
+ provider key (if needed), the consensus values + manifest, and (recommended) one LLM provider key.
severity: low
# ============================================================================
@@ -857,7 +1000,9 @@ edges:
Docker bind mounts are established at container start time and resolve symlinks
to their target inodes. When symlinks change (e.g., /opt/genlayer-node/data
switches from v0.4.3/data to v0.4.4/data), the container still sees the old
- directory's inode.
+ directory's inode. (v0.6 layout: every version's data/ resolves to the same shared
+ /opt/genlayer-node//data, so this mostly bites across minor versions — keep the
+ restart anyway.)
fix: |
IMMEDIATE FIX (after node restart):
=====================================
@@ -876,7 +1021,8 @@ edges:
[Service]
...
- ExecStartPost=-/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy 2>/dev/null || true'
+ ExecStartPost=-+/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy >/dev/null 2>&1 || true'
+ # '+' = run as root: the node's service user is not in the docker group
This ensures Alloy's bind mount is refreshed on every node restart.
@@ -921,6 +1067,20 @@ edges:
- But Grafana shows no data / validator appears offline
- Alloy container is running but log timestamps are old
+ - id: v05-to-v06-not-in-place
+ description: "Trying to update a v0.5 (Bradbury Phase 1 / Asimov) node in place to v0.6 (Bradbury Phase 2)"
+ detect: |
+ /opt/genlayer-node/bin/genlayernode version | grep -E 'Version: +v0\.5'
+ grep -E '^ genlayerchainrpcurl:.*(zksync|genlayer-testnet)' /opt/genlayer-node/configs/node/config.yaml
+ impact: |
+ Phase 2 is a new chain (Arbitrum Orbit, chain id 4221) with new contracts and a new validator
+ registration (proof of possession). Old data, config and registration do not carry over.
+ fix: |
+ Stop and disable the old node, then do a fresh install (install-procedure.md) in a new
+ version directory with a fresh data directory. Register the validator again with the
+ qualified CLI (staking-wizard-procedure.md).
+ severity: high
+
- id: high-downtime-update
description: "Stopping old node before preparing new version causes 3-4 minute downtime"
detect: |
@@ -943,14 +1103,12 @@ edges:
Phase 1: PREPARE WHILE OLD NODE RUNS (No Downtime)
===================================================
- 1. Download new version to /tmp
- 2. Create new version directory
- 3. Extract tarball
- 4. Run GenVM setup (THIS IS THE SLOW PART - 2+ minutes)
- 5. Copy config.yaml and .env
- 6. Enable LLM provider in GenVM config if needed
- 7. Create data directory and symlinks to shared DB
- 8. Copy keystore
+ 1. Download + verify new version (cosign + sha256)
+ 2. install.sh into the new version directory
+ 3. (install.sh) GenVM setup — THIS IS THE SLOW PART, 2+ minutes
+ 4. Copy config.yaml (keep the new genvm.root_dir) + protocol manifest + .env
+ 5. Apply LLM config / enable LLM provider if needed
+ 6. Link the shared data directory (genvm.db, ops.db, keystore — no copy)
Phase 2: QUICK SWITCH (10-15 seconds downtime)
==============================================
@@ -965,16 +1123,12 @@ edges:
validator_impact: "missed validations, lost rewards, priming delay, potential slashing"
- id: database-not-shared-patch-versions
- description: "Database copied instead of shared between v0.4.x patch versions"
+ description: "Data directory copied instead of shared between patch versions"
detect: |
- # Check if each patch version has its own database instead of shared
- ls -la /opt/genlayer-node/v0.4.3/data/node/genlayer.db
- ls -la /opt/genlayer-node/v0.4.4/data/node/genlayer.db
- # If both are directories (not symlinks), they're not shared
-
- # Correct structure should show:
- # v0.4.3/data/node/genlayer.db -> ../../../v0.4/data/node/genlayer.db (symlink)
- # v0.4.4/data/node/genlayer.db -> ../../../v0.4/data/node/genlayer.db (symlink)
+ # v0.6 layout: each version's data/ is a link to the shared /opt/genlayer-node//data
+ for d in /opt/genlayer-node/v0.6.*/data; do printf '%s -> %s\n' "$d" "$(readlink "$d" || echo REAL-DIR)"; done
+ # v0.4/v0.5 layout (older installs): data/node/genlayer.db per version
+ ls -la /opt/genlayer-node/v0.4.*/data/node/genlayer.db 2>/dev/null
impact: |
- Wasted disk space (duplicate databases)
- Each version has different sync state
@@ -1008,14 +1162,15 @@ edges:
6. Start the node:
sudo systemctl start genlayer-node
- STRUCTURE FOR NEW INSTALLATIONS:
-
- For major.minor versions (v0.4.x), use shared database:
- /opt/genlayer-node/v0.4/data/node/genlayer.db (shared DB)
- /opt/genlayer-node/v0.4.3/data/node/genlayer.db -> symlink to shared
- /opt/genlayer-node/v0.4.4/data/node/genlayer.db -> symlink to shared
+ STRUCTURE FOR NEW INSTALLATIONS (v0.6+):
- Keystore and logs remain version-specific (not shared).
+ Share the WHOLE data directory per minor line (v0.6 splits state into genvm.db =
+ chain history and ops.db = this node's vote salts / nonces / operator stamp — both must
+ persist across patch updates, and ops.db must never be copied to another node):
+ /opt/genlayer-node/v0.6/data/node/{genvm.db,ops.db,keystore,logs} (shared)
+ /opt/genlayer-node/v0.6.0/data -> ../v0.6/data
+ /opt/genlayer-node/v0.6.1/data -> ../v0.6/data
+ See common-procedures.md -> "Create Directory Structure".
severity: high
note: "This issue typically occurs when upgrading from old installations that weren't set up with shared storage"
@@ -1051,7 +1206,7 @@ edges:
# The validator wallet address is a SMART CONTRACT, not the owner account.
# Common mistake: using owner address in config.yaml instead of validator wallet.
# Check: validator-info should work with the address in config.yaml
- genlayer staking validator-info --validator
+ genlayer staking validator-info --network testnet-bradbury
# If it returns "Address is not a validator", the wrong address is being used.
impact: |
Node starts but cannot validate. It won't recognize itself as a validator
@@ -1090,24 +1245,24 @@ edges:
If the operator account was deleted:
- Create a new operator account
- - Update the validator to use the new operator:
- genlayer staking set-operator \
- --account "Owner Name" \
- --validator 0xVALIDATOR_WALLET \
- --operator 0xNEW_OPERATOR
- - Fund the new operator address with ETH for gas
+ - Update the validator to use the new operator (owner signs; Phase 2 also needs the
+ new operator's proof of possession: --operator-account, CLI v0.40.0-rc.3+):
+ genlayer staking set-operator 0xVALIDATOR_WALLET 0xNEW_OPERATOR \
+ --account "Owner Name" --operator-account "New Operator Name" --network testnet-bradbury
+ - Fund the new operator address with GEN for gas
severity: high
- id: wizard-insufficient-balance
description: "Running wizard without enough GEN tokens"
detect: |
+ genlayer staking epoch-info --network testnet-bradbury # deployed self-stake minimum
genlayer account show
- # Balance must be >= 42,000 GEN (plus small gas buffer)
+ # Balance must be >= the deployed minimum (42,000 GEN is the protocol default) plus gas
impact: |
Wizard exits at balance check step. No on-chain changes made.
fix: |
1. Check current balance: genlayer account show
- 2. Fund account with 42,000+ GEN
+ 2. Fund account with the deployed minimum + gas
3. Re-run the wizard
severity: medium
@@ -1147,7 +1302,7 @@ edges:
2. If balance is 0 or low, send GEN from owner:
genlayer account send 0xOPERATOR_ADDRESS 10gen \
--account "Owner Name" \
- --network testnet-asimov
+ --network testnet-bradbury
3. Verify operator now has funds:
genlayer account show --account "Operator Name"
diff --git a/plugins/genlayernode/skills/genlayernode/skill.yaml b/plugins/genlayernode/skills/genlayernode/skill.yaml
index f09d925..3b8b851 100644
--- a/plugins/genlayernode/skills/genlayernode/skill.yaml
+++ b/plugins/genlayernode/skills/genlayernode/skill.yaml
@@ -28,10 +28,10 @@ owns:
# Default configuration values (from infrastructure)
defaults:
- network: testnet-asimov
- # NOTE: rpc_url and websocket_url have NO defaults - user must provide
- consensus_contract_address: "0xe66B434bc83805f380509642429eC8e43AE9874a"
- consensus_genesis: 17326
+ network: testnet-bradbury # CLI network alias; the testnet documented today
+ # NOTE: rpc_url, websocket_url, the RPC provider key, consensus address, chain id, genesis and
+ # the protocol manifest are PER-DEPLOYMENT - NO defaults. Take them from the network's section
+ # in the docs ("Network-Specific Consensus Configuration"); for your own network, from its operator.
install_path: "/opt/genlayer-node"
data_dir: "./data/node"
node_rpc_port: 9151
@@ -43,8 +43,10 @@ defaults:
# Required user configuration (no defaults)
required_inputs:
- - rpc_url: "GenLayer Chain ZKSync HTTP RPC URL"
- - websocket_url: "GenLayer Chain ZKSync WebSocket URL"
+ - rpc_url: "GenLayer Chain HTTP RPC URL (from the network's docs)"
+ - websocket_url: "GenLayer Chain WebSocket URL (from the network's docs)"
+ - consensus_values: "consensusaddress, chainid, genesis and the protocol manifest (from the network's docs)"
+ - rpc_key: "RPC provider API key, if the provider needs one (set by the user in .env)"
- validator_wallet_address: "From staking wizard or existing wallet"
- operator_address: "From keystore import or generate new"
- llm_api_key: "From chosen LLM provider"
@@ -57,18 +59,16 @@ directory_structure:
description: "Version-specific directory containing all node files"
contents:
- "bin/genlayernode"
- - "third_party/genvm/"
+ - "opt/genvm//"
- "docker-compose.yaml"
- ".env"
- - "data/node/genlayer.db"
- - "data/node/keystore/"
- - "data/node/logs/"
- - "data/node/merkle/"
+ - "data -> ../{MINOR}/data (shared: genvm.db, ops.db, keystore/, logs/)"
- "configs/node/config.yaml"
+ - "configs/node/protocol-manifest.json"
- path: "bin -> {VERSION}/bin"
description: "Symlink to current version binary"
- - path: "third_party -> {VERSION}/third_party"
- description: "Symlink to current version third_party"
+ - path: "opt -> {VERSION}/opt"
+ description: "Symlink to current version GenVM tree"
- path: "data -> {VERSION}/data"
description: "Symlink to current version data"
- path: "configs -> {VERSION}/configs"
@@ -127,7 +127,7 @@ inputs_required:
options:
- label: "Yes, run staking wizard"
value: true
- description: "I need to create validator wallet and stake 42,000+ GEN"
+ description: "I need to create validator wallet and stake the deployed self-stake minimum"
- label: "No, I already have a validator wallet"
value: false
description: "I already completed staking and have my operator keystore"
@@ -185,6 +185,19 @@ inputs_required:
description: "Create a new operator key (requires updating validator settings)"
required: true
+ - id: node_mode
+ description: "Full node (no staking) or validator"
+ type: choice
+ question: "Will this node run as a full node or as a validator?"
+ options:
+ - label: "Full node"
+ value: full
+ description: "Syncs and serves RPC; no staking wizard, operator key or keystore password"
+ - label: "Validator"
+ value: validator
+ description: "Needs a registered validator wallet and the operator keystore"
+ required: true
+
- id: llm_provider
description: "LLM provider for intelligent contract execution"
type: choice
@@ -214,6 +227,9 @@ inputs_required:
- label: "Atoma"
value: atoma
env_var: "ATOMAKEY"
+ - label: "OpenRouter"
+ value: openrouter
+ env_var: "OPENROUTERKEY"
required: true
- id: llm_api_key
@@ -257,11 +273,11 @@ config_wizard:
- id: genlayerchainrpcurl
prompt: "GenLayer Chain RPC URL"
required: true
- validation: "Must be a valid HTTP(S) URL - user must provide ZKSync RPC endpoint"
+ validation: "Must be a valid HTTP(S) URL - user must provide the network's RPC endpoint (no API key in it)"
- id: genlayerchainwebsocketurl
prompt: "GenLayer Chain WebSocket URL"
required: true
- validation: "Must be a valid WSS URL - user must provide ZKSync WebSocket endpoint"
+ validation: "Must be a valid WSS URL - user must provide the network's WebSocket endpoint (no API key in it)"
- id: node_identity
name: "Node Identity"
@@ -290,7 +306,7 @@ config_wizard:
- id: rpc_groups
prompt: "Which RPC endpoint groups to enable?"
type: multi_choice
- default: ["genlayer", "ethereum", "zksync"]
+ default: ["genlayer", "ethereum"]
options:
- label: "genlayer"
description: "gen_* methods (call, getContractSchema, etc.)"
@@ -378,13 +394,12 @@ patterns:
Check all required software is installed before proceeding.
commands:
- "uname -m | grep -E '^(x86_64|amd64)$'"
- - "node --version"
- - "npm --version"
- "docker --version"
- "docker compose version"
- - "python3 --version"
+ - "python3 -c 'import sys; sys.exit(sys.version_info < (3,10))'"
- "pip3 --version"
- - "python3 -m venv --help"
+ - "python3 -c 'import venv, ensurepip'"
+ - "cosign version"
- id: install-prerequisites-ubuntu
description: |
@@ -392,20 +407,22 @@ patterns:
commands:
- "sudo apt update"
- "sudo apt install -y python3 python3-pip python3-venv wget curl jq tar"
+ - "sudo apt install -y docker.io docker-compose-v2 && sudo systemctl enable --now docker"
+ - "curl -fsSLo /tmp/cosign.deb https://github.com/sigstore/cosign/releases/download/v2.4.1/cosign_2.4.1_amd64.deb && sudo dpkg -i /tmp/cosign.deb"
- id: install-genlayer-cli
description: |
Install the GenLayer CLI globally via npm.
- command: "npm install -g genlayer"
+ command: 'npm install -g "genlayer@${CLI_VERSION}" # exact version from the release announcement for your network'
- id: create-validator-wallet
description: |
Run the interactive staking wizard to create validator wallet.
- User must have 42,000+ GEN tokens.
+ User must have the deployed self-stake minimum (genlayer staking epoch-info).
The wizard is FULLY INTERACTIVE (requires TTY) and cannot be automated.
It runs on the user's LOCAL machine, not the validator server.
See staking-wizard-procedure.md for the complete walkthrough.
- command: "genlayer staking wizard --network testnet-asimov"
+ command: "genlayer staking wizard --network testnet-bradbury"
notes:
- "The wizard creates THREE addresses: owner, operator, and validator wallet"
- "Owner = cold wallet that controls withdrawals (stays local)"
@@ -413,6 +430,7 @@ patterns:
- "Validator Wallet = smart contract deployed on-chain (used in config.yaml)"
- "The wizard handles: account selection, balance check, operator creation, keystore export, staking transaction, and identity setup"
- "If user cannot run wizard, use individual CLI commands (see staking-wizard-procedure.md)"
+ - "Phase 2: the operator must be a CLI account so it can sign the proof of possession; an address alone fails"
outputs:
- "validator_wallet_address (from on-chain transaction result)"
- "operator_address (from operator account creation)"
@@ -433,8 +451,8 @@ patterns:
- "genlayer account create --name 'Operator Name'"
- "genlayer account unlock --account 'Operator Name'"
- "genlayer account export --account 'Operator Name' --output './operator-keystore.json'"
- - "genlayer staking validator-join --account 'Owner Name' --amount 42000gen --operator 0xOPERATOR_ADDRESS --network testnet-asimov"
- - "genlayer staking set-identity --account 'Owner Name' --validator 0xVALIDATOR_WALLET"
+ - "genlayer staking validator-join --account 'Owner Name' --amount gen --operator 0xOPERATOR_ADDRESS --network testnet-bradbury"
+ - "genlayer staking set-identity 0xVALIDATOR_WALLET --account 'Operator Name' --moniker 'My Validator' --network testnet-bradbury"
notes:
- "account create, account unlock, and account export require interactive password input"
- "validator-join is non-interactive when all flags are provided"
@@ -453,7 +471,7 @@ patterns:
the validator misses validation rounds.
commands:
- "genlayer account show --account '${OPERATOR_ACCOUNT_NAME}'"
- - "genlayer account send ${OPERATOR_ADDRESS} 10gen --account '${OWNER_ACCOUNT_NAME}' --network testnet-asimov"
+ - "genlayer account send ${OPERATOR_ADDRESS} 10gen --account '${OWNER_ACCOUNT_NAME}' --network testnet-bradbury"
- "genlayer account show --account '${OPERATOR_ACCOUNT_NAME}'"
notes:
- "Recommended initial funding: 10 GEN"
@@ -463,12 +481,12 @@ patterns:
- id: verify-validator-info
description: |
Verify validator registration after staking wizard completes.
- command: "genlayer staking validator-info --validator 0xVALIDATOR_WALLET_ADDRESS"
+ command: "genlayer staking validator-info 0xVALIDATOR_WALLET_ADDRESS --network testnet-bradbury"
notes:
- "Use the validatorWallet address, NOT the owner address"
- "Check selfStakePendingDeposits for activation epoch"
- - "live: false until deposit activates (typically 2 epochs)"
- - "needsPriming may need to be resolved with: genlayer staking validator-prime"
+ - "genlayer staking validators shows status pending until the deposit activates (typically 2 epochs)"
+ - "needsPriming may need to be resolved with: genlayer staking validator-prime 0xVALIDATOR_WALLET_ADDRESS --network testnet-bradbury"
- id: list-available-versions
description: |
@@ -479,33 +497,37 @@ patterns:
- id: download-and-extract
description: |
- Download to /tmp, then extract to versioned directory structure.
+ Download to a temp dir, verify (cosign + SHA256), then install to the versioned directory
+ with the release install.sh (extract + GenVM setup). See install-procedure.md Phase 2.
commands:
- "export VERSION=${VERSION}"
- - "wget https://storage.googleapis.com/gh-af/genlayer-node/bin/amd64/${VERSION}/genlayer-node-linux-amd64-${VERSION}.tar.gz -O /tmp/genlayer-node-${VERSION}.tar.gz"
- - "sudo mkdir -p /opt/genlayer-node/${VERSION}"
- - "sudo tar -xzvf /tmp/genlayer-node-${VERSION}.tar.gz -C /opt/genlayer-node/${VERSION} --strip-components=1"
- - "sudo mkdir -p /opt/genlayer-node/${VERSION}/{data/node,configs/node}"
- - "sudo chown -R $USER:$USER /opt/genlayer-node"
- - "rm /tmp/genlayer-node-${VERSION}.tar.gz"
+ - "base=https://storage.googleapis.com/gh-af/genlayer-node/bin/amd64/${VERSION}"
+ - "wget ${base}/genlayer-node-linux-amd64-${VERSION}.tar.gz ${base}/genlayer-node_${VERSION}_SHA256SUMS ${base}/genlayer-node_${VERSION}_SHA256SUMS.sigstore.json ${base}/install.sh"
+ - "cosign verify-blob --bundle genlayer-node_${VERSION}_SHA256SUMS.sigstore.json --certificate-identity-regexp '^https://github\\.com/genlayerlabs/genlayer-node/\\.github/workflows/release-package\\.yml@' --certificate-oidc-issuer https://token.actions.githubusercontent.com genlayer-node_${VERSION}_SHA256SUMS"
+ - "grep -q \"[ *]genlayer-node-linux-amd64-${VERSION}.tar.gz$\" genlayer-node_${VERSION}_SHA256SUMS # stop if the tarball is not listed"
+ - "sha256sum -c genlayer-node_${VERSION}_SHA256SUMS --ignore-missing # stop on any failure: do not run install.sh"
+ - "sudo install -d -o genlayer -g genlayer /opt/genlayer-node"
+ - "chmod +x install.sh && sudo -u genlayer -H ./install.sh --install-dir /opt/genlayer-node/${VERSION} --tarball genlayer-node-linux-amd64-${VERSION}.tar.gz"
- id: setup-symlinks
description: |
Create symlinks for current version for easy access from /opt/genlayer-node.
commands:
- - "ln -sfn /opt/genlayer-node/${VERSION}/bin /opt/genlayer-node/bin"
- - "ln -sfn /opt/genlayer-node/${VERSION}/third_party /opt/genlayer-node/third_party"
- - "ln -sfn /opt/genlayer-node/${VERSION}/data /opt/genlayer-node/data"
- - "ln -sfn /opt/genlayer-node/${VERSION}/configs /opt/genlayer-node/configs"
- - "ln -sfn /opt/genlayer-node/${VERSION}/docker-compose.yaml /opt/genlayer-node/docker-compose.yaml"
- - "ln -sfn /opt/genlayer-node/${VERSION}/.env /opt/genlayer-node/.env"
- - "ln -sfn /opt/genlayer-node/${VERSION}/alloy-config.river /opt/genlayer-node/alloy-config.river"
- - "ln -sfn /opt/genlayer-node/${VERSION}/genvm-module-web-docker.yaml /opt/genlayer-node/genvm-module-web-docker.yaml"
+ - "sudo -u genlayer ln -sfn ../${MINOR}/data /opt/genlayer-node/${VERSION}/data"
+ - "sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/bin /opt/genlayer-node/bin"
+ - "sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/opt /opt/genlayer-node/opt"
+ - "sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/data /opt/genlayer-node/data"
+ - "sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/configs /opt/genlayer-node/configs"
+ - "sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/docker-compose.yaml /opt/genlayer-node/docker-compose.yaml"
+ - "sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/.env /opt/genlayer-node/.env"
+ - "sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/alloy-config.river /opt/genlayer-node/alloy-config.river"
+ - "sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/genvm-module-web-docker.yaml /opt/genlayer-node/genvm-module-web-docker.yaml"
- id: run-genvm-setup
description: |
- Run GenVM post-install script to download dependencies.
- command: "python3 /opt/genlayer-node/third_party/genvm/bin/setup.py"
+ Run GenVM post-install script (install.sh already runs it). Run it in the final
+ directory: it writes that absolute path into the GenVM binaries.
+ command: "cd /opt/genlayer-node/${VERSION} && sudo -u genlayer python3 \"$(sed -n 's/^ *root_dir: *//p' configs/node/config.yaml)/bin/setup.py\""
- id: create-env-file
description: |
@@ -518,11 +540,17 @@ patterns:
# Node identification
NODE_ID=${NODE_ID}
NODE_TYPE=validator
- NODE_ENV=testnet-asimov
+ NODE_ENV=${NETWORK}
NODE_VERSION=${VERSION}
- # Node password (for keystore encryption)
- NODE_PASSWORD=${PASSWORD}
+ # Keystore password (validator only; the user sets it, Claude leaves it empty)
+ GENLAYERNODE_PASSWORD=
+ # rc.1 compose refuses every docker compose command (also the WebDriver) while this is
+ # empty: validator = the export passphrase; full node = any non-empty value (user sets it)
+ NODE_PASSWORD=
+
+ # RPC provider API key, if the provider needs one (the user sets it)
+ GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCKEY=
# LLM API Key (only set the one you're using)
HEURISTKEY=${HEURIST_KEY}
@@ -533,19 +561,9 @@ patterns:
GEMINIKEY=${GEMINI_KEY}
XAIKEY=${XAI_KEY}
ATOMAKEY=${ATOMA_KEY}
+ OPENROUTERKEY=${OPENROUTER_KEY}
- # Rollup configuration
- GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCURL=${RPC_URL}
- GENLAYERNODE_ROLLUP_GENLAYERCHAINWEBSOCKETURL=${WEBSOCKET_URL}
-
- # Consensus configuration (AddressManager)
- GENLAYERNODE_CONSENSUS_CONSENSUSADDRESS=0xe66B434bc83805f380509642429eC8e43AE9874a
- GENLAYERNODE_CONSENSUS_GENESIS=17326
-
- # Node configuration
- GENLAYERNODE_NODE_VALIDATORWALLETADDRESS=${VALIDATOR_ADDRESS}
- GENLAYERNODE_NODE_OPERATORADDRESS=${OPERATOR_ADDRESS}
- GENLAYERNODE_NODE_MODE=validator
+ # Rollup, consensus and node values live in config.yaml (generate-config-yaml)
# Ports
NODE_RPC_PORT=${RPC_PORT:-9151}
@@ -566,15 +584,18 @@ patterns:
# GenLayer Node Configuration
# Generated by genlayernode skill
- # Rollup configuration
+ # Rollup configuration (the API key goes in .env, never here)
rollup:
genlayerchainrpcurl: "${RPC_URL}"
genlayerchainwebsocketurl: "${WEBSOCKET_URL}"
+ provider: "${RPC_PROVIDER}"
- # Consensus configuration (AddressManager)
+ # Consensus configuration (AddressManager) - per-deployment, from the network's docs
consensus:
- consensusaddress: "0xe66B434bc83805f380509642429eC8e43AE9874a"
- genesis: 17326
+ consensusaddress: "${CONSENSUS_ADDRESS}"
+ chainid: "${CHAIN_ID}"
+ manifestpath: "configs/node/protocol-manifest.json"
+ genesis: ${GENESIS_BLOCK}
# Data directory
datadir: "./data/node"
@@ -595,7 +616,7 @@ patterns:
# Node configuration
node:
- mode: "validator"
+ mode: "${NODE_MODE:-validator}"
validatorWalletAddress: "${VALIDATOR_ADDRESS}"
operatorAddress: "${OPERATOR_ADDRESS}"
admin:
@@ -607,7 +628,7 @@ patterns:
genlayer: ${RPC_GENLAYER:-true}
genlayer_debug: ${RPC_DEBUG:-false}
ethereum: ${RPC_ETHEREUM:-true}
- zksync: ${RPC_ZKSYNC:-true}
+ zksync: ${RPC_ZKSYNC:-false}
ops:
port: ${OPS_PORT:-9153}
endpoints:
@@ -619,7 +640,7 @@ patterns:
# GenVM configuration
genvm:
- root_dir: ./third_party/genvm
+ root_dir: ./opt/genvm/v0.6.0-rc10 # keep the value install.sh wrote (rc.1)
start_manager: true
manager_url: http://127.0.0.1:3999
permits:
@@ -656,14 +677,13 @@ patterns:
- id: import-operator-key-from-wizard
description: |
- Import operator keystore exported from staking wizard.
+ Install the operator keystore exported from the staking wizard as-is (encrypted).
+ No password or passphrase on any command line: the user sets GENLAYERNODE_PASSWORD
+ (Docker: NODE_PASSWORD) in .env to the export passphrase.
command: |
- cd /opt/genlayer-node && \
- ./bin/genlayernode account import \
- --password "${NODE_PASSWORD}" \
- --passphrase "${EXPORT_PASSPHRASE}" \
- --path "${KEYSTORE_PATH}" \
- --quiet
+ sudo install -d -m 700 -o genlayer -g genlayer /opt/genlayer-node/data/node/keystore && \
+ sudo test ! -e /opt/genlayer-node/data/node/keystore/operator.json && \
+ sudo install -m 600 -o genlayer -g genlayer "${KEYSTORE_PATH}" /opt/genlayer-node/data/node/keystore/operator.json
- id: copy-operator-key-from-previous
description: |
@@ -676,15 +696,13 @@ patterns:
Generate a new operator key.
command: |
cd /opt/genlayer-node && \
- ./bin/genlayernode account new \
- --setup \
- --password "${NODE_PASSWORD}"
+ sudo -u genlayer bash -c 'set -a; . ./.env; set +a; ./bin/genlayernode account new -c $(pwd)/configs/node/config.yaml --setup'
warnings:
- "You must update your validator's operator address after generating!"
- - "You must fund the new operator address with ETH for gas fees!"
+ - "You must fund the new operator address with GEN for gas fees!"
post_steps:
- "Note the new operator address from command output"
- - "Fund the operator address with ETH (for transaction gas)"
+ - "Fund the operator address with GEN (for transaction gas)"
- "Update validator settings to use new operator address"
- id: start-webdriver
@@ -699,15 +717,14 @@ patterns:
- id: run-doctor-check
description: |
Validate configuration before starting.
- command: "cd /opt/genlayer-node && source .env && ./bin/genlayernode doctor"
+ command: "cd /opt/genlayer-node && sudo -u genlayer bash -c 'set -a; . ./.env; set +a; ./bin/genlayernode doctor'"
- id: run-node-manual
description: |
Start node manually (use with screen/tmux).
commands:
- "cd /opt/genlayer-node"
- - "source .env"
- - "./bin/genlayernode run --password '${NODE_PASSWORD}'"
+ - "sudo -u genlayer bash -c 'set -a; . ./.env; set +a; ./bin/genlayernode run -c $(pwd)/configs/node/config.yaml'"
- id: create-systemd-service
description: |
@@ -722,12 +739,13 @@ patterns:
[Service]
Type=simple
- User=${USER}
- Group=${USER}
+ User=genlayer
+ Group=genlayer
WorkingDirectory=/opt/genlayer-node
EnvironmentFile=/opt/genlayer-node/.env
- ExecStart=/opt/genlayer-node/bin/genlayernode run --password ${NODE_PASSWORD}
- ExecStartPost=-/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy 2>/dev/null || true'
+ Environment=DOCKER_HOST=tcp://127.0.0.1:2375
+ ExecStart=/opt/genlayer-node/bin/genlayernode run -c /opt/genlayer-node/configs/node/config.yaml
+ ExecStartPost=-+/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy >/dev/null 2>&1 || true'
Restart=always
RestartSec=10
StandardOutput=journal
@@ -746,12 +764,14 @@ patterns:
- "ExecStartPost restarts Alloy to refresh its bind mount after symlink changes"
- "The '-' prefix means systemd ignores failures (e.g., if Alloy isn't installed)"
- "The 5-second sleep ensures the node has started before restarting Alloy"
+ - "The '+' runs ExecStartPost as root: the genlayer service user is not in the docker group"
+ - "The node reads GENLAYERNODE_PASSWORD from EnvironmentFile - no --password argument"
- id: run-node-docker
description: |
Start node using Docker Compose method.
commands:
- - "cd /opt/genlayer-node && source .env && docker compose --profile node up -d"
+ - "cd /opt/genlayer-node/${VERSION} && sudo chown -R 10001:10001 ./data/ && sudo docker compose -p genlayer-node --profile node up -d"
- id: verify-node-running
description: |
@@ -802,7 +822,7 @@ patterns:
Diagnose validator downtime before changing configuration or requesting unban.
Covers shared RPC outages, temporary validator bans, and LLM provider capability errors.
commands:
- - "cd /opt/genlayer-node && source .env && ./bin/genlayernode doctor"
+ - "cd /opt/genlayer-node && sudo -u genlayer bash -c 'set -a; . ./.env; set +a; ./bin/genlayernode doctor'"
- "curl -s http://localhost:9153/health | jq"
- "journalctl -u genlayer-node --since '1 hour ago' --no-pager | grep -Ei 'temporary ban|validator is banned|NO_PROVIDER_FOR_PROMPT|Gateway Timeout|504|get chain ID'"
decision_tree:
@@ -836,9 +856,9 @@ anti_patterns:
fix: "Use an AMD64 Linux server."
- id: insufficient-tokens
- description: "Creating validator with less than 42,000 GEN"
+ description: "Creating validator with less than the deployed self-stake minimum"
why_bad: "Cannot stake and validator won't be active."
- fix: "Acquire 42,000+ GEN before running staking wizard."
+ fix: "Acquire the deployed minimum (genlayer staking epoch-info; 42,000 GEN is the protocol default) before running staking wizard."
- id: missing-llm-key
description: "Not setting any LLM API key"
@@ -884,7 +904,7 @@ anti_patterns:
fix: |
Use EnvironmentFile directive:
- EnvironmentFile=/opt/genlayer-node/.env
- - ExecStart=.../genlayernode run --password "${NODE_PASSWORD}"
+ - ExecStart=.../genlayernode run -c /opt/genlayer-node/configs/node/config.yaml (reads GENLAYERNODE_PASSWORD from the environment)
- id: secrets-in-command-output
description: "Showing secrets in command output or verification steps"
@@ -928,14 +948,14 @@ procedure:
detail: |
Verify system meets requirements:
- AMD64 architecture (x86_64)
- - Node.js v18+, Docker, Python 3
+ - Docker, Python 3.10+ (pip, venv), cosign
- 16GB+ RAM, 8+ cores, 128GB+ SSD
commands:
- "uname -m"
- - "node --version"
- "docker compose version"
- "python3 --version"
- on_failure: "Install missing prerequisites"
+ - "cosign version"
+ on_failure: "Install missing prerequisites (after the user confirms)"
- step: "Decide: New validator or existing?"
decision_point: true
@@ -962,18 +982,20 @@ procedure:
- Validator wallet address: smart contract deployed on-chain
Prerequisites:
- - genlayer CLI installed (npm install -g genlayer)
- - Owner account created and unlocked with 42,000+ GEN
+ - genlayer CLI installed (exact version for your network: npm install -g "genlayer@${CLI_VERSION}")
+ - Owner account created and unlocked with the deployed self-stake minimum
- Terminal with TTY for interactive prompts
Wizard steps:
1. Account selection (or creation)
2. Network selection
- 3. Balance verification (42,000+ GEN)
- 4. Operator setup (create new operator, export keystore)
- 5. Stake amount confirmation
- 6. On-chain validator registration (returns validator wallet address)
- 7. Identity setup (optional: moniker, website, socials)
+ 3. Funding source (wallet or vesting)
+ 4. Balance verification (deployed minimum + gas)
+ 5. Operator setup (create new operator, export keystore)
+ 6. Stake amount confirmation
+ 7. On-chain validator registration (returns validator wallet address)
+ 8. Identity setup (optional: moniker, website, socials)
+ 9. Summary
If wizard cannot be used, individual CLI commands are available:
- genlayer account create/unlock (interactive)
@@ -981,13 +1003,13 @@ procedure:
- genlayer staking validator-join (non-interactive with all flags)
- genlayer staking set-identity (interactive)
commands:
- - "npm install -g genlayer"
- - "genlayer staking wizard --network testnet-asimov"
+ - 'npm install -g "genlayer@${CLI_VERSION}"'
+ - "genlayer staking wizard --network testnet-bradbury"
outputs:
- - "validator_wallet_address (from Step 6 on-chain result)"
- - "operator_address (from Step 4 operator creation)"
- - "operator-keystore.json (exported to current directory in Step 4)"
- - "export_passphrase (set by user in Step 4)"
+ - "validator_wallet_address (from Step 7 on-chain result)"
+ - "operator_address (from Step 5 operator creation)"
+ - "operator-keystore.json (exported to current directory in Step 5)"
+ - "export_passphrase (set by user in Step 5)"
notes:
- "The wizard runs on LOCAL machine, not the validator server"
- "Save the operator keystore file — needed for upload to server"
@@ -1008,13 +1030,13 @@ procedure:
If balance is 0, transfer GEN from the owner:
genlayer account send 0xOPERATOR_ADDRESS 10gen \
- --account "Owner Name" --network testnet-asimov
+ --account "Owner Name" --network testnet-bradbury
Recommended initial funding: 10 GEN (covers thousands of transactions).
Monitor and top up when operator balance drops below 1 GEN.
commands:
- "genlayer account show --account '${OPERATOR_ACCOUNT_NAME}'"
- - "genlayer account send ${OPERATOR_ADDRESS} 10gen --account '${OWNER_ACCOUNT_NAME}' --network testnet-asimov"
+ - "genlayer account send ${OPERATOR_ADDRESS} 10gen --account '${OWNER_ACCOUNT_NAME}' --network testnet-bradbury"
on_failure: "Fund operator manually via wallet or exchange"
notes:
- "Operator needs GEN for gas fees to sign validation transactions"
@@ -1040,69 +1062,53 @@ procedure:
- step: "Download node software"
detail: |
- Download and extract to versioned directory structure with shared storage for database.
+ Download, verify and install to versioned directory structure with shared storage for data.
Directory structure created:
/opt/genlayer-node/
- ├── v0.4/ # Major version (shared storage)
- │ └── data/node/ # Shared database directory
- │ └── genlayer.db # Database shared across v0.4.x
- ├── v0.4.3/ # Patch version
+ ├── v0.6/ # Major version (shared storage)
+ │ └── data/node/ # Shared data directory (all v0.6.x)
+ │ ├── genvm.db # Chain history
+ │ ├── ops.db # This node's state (vote salts, nonces)
+ │ ├── keystore/
+ │ └── logs/
+ ├── v0.6.0/ # Patch version
│ ├── bin/
- │ ├── third_party/
+ │ ├── opt/genvm/
│ ├── configs/
- │ ├── data/node/
- │ │ ├── genlayer.db -> ../../v0.4/data/node/genlayer.db (symlink)
- │ │ ├── keystore/ # Version-specific keystore
- │ │ └── logs/ # Version-specific logs
+ │ ├── data -> ../v0.6/data (symlink)
│ ├── docker-compose.yaml
│ └── .env
└── [convenience symlinks pointing to current version]
command: |
- # Extract major version for shared storage (e.g., v0.4 from v0.4.3)
+ # Extract major version for shared storage (e.g., v0.6 from v0.6.0)
MAJOR_VERSION=$(echo ${VERSION} | grep -oE 'v[0-9]+\.[0-9]+' || echo ${VERSION})
echo "Installing version: ${VERSION} (major: ${MAJOR_VERSION})"
- # Download to /tmp
- wget https://storage.googleapis.com/gh-af/genlayer-node/bin/amd64/${VERSION}/genlayer-node-linux-amd64-${VERSION}.tar.gz -O /tmp/genlayer-node-${VERSION}.tar.gz
+ # Download, verify (cosign + SHA256) and install with the release install.sh
+ # (extract + GenVM setup in the final directory) - see install-procedure.md Phase 2
+ sudo install -d -o genlayer -g genlayer /opt/genlayer-node
+ sudo -u genlayer -H ./install.sh --install-dir /opt/genlayer-node/${VERSION} \
+ --tarball genlayer-node-linux-amd64-${VERSION}.tar.gz
# Create major version directory for shared storage FIRST
- # genlayer.db is a PebbleDB directory, not a file
- sudo mkdir -p /opt/genlayer-node/${MAJOR_VERSION}/data/node/genlayer.db
- echo "Created major version directory: /opt/genlayer-node/${MAJOR_VERSION}/data/node/genlayer.db"
-
- # Create patch version directory structure
- sudo mkdir -p /opt/genlayer-node/${VERSION}
- sudo tar -xzvf /tmp/genlayer-node-${VERSION}.tar.gz -C /opt/genlayer-node/${VERSION} --strip-components=1
-
- # Create version-specific directories (keystore and logs are NOT shared)
- sudo mkdir -p /opt/genlayer-node/${VERSION}/data/node/keystore
- sudo mkdir -p /opt/genlayer-node/${VERSION}/data/node/logs
- sudo mkdir -p /opt/genlayer-node/${VERSION}/configs/node
- echo "Created version-specific directories"
-
- # Remove any existing genlayer.db in patch version (we'll symlink it)
- # NOTE: For binary (systemd) deploys only. Docker deploys cannot use
- # symlinks because the target is outside the container's mounted volume.
- sudo rm -rf /opt/genlayer-node/${VERSION}/data/node/genlayer.db
-
- # Create symlink from patch version to major version shared database
- sudo ln -sfn ../../../${MAJOR_VERSION}/data/node/genlayer.db /opt/genlayer-node/${VERSION}/data/node/genlayer.db
- echo "Created symlink: /opt/genlayer-node/${VERSION}/data/node/genlayer.db -> ../../../${MAJOR_VERSION}/data/node/genlayer.db"
+ sudo -u genlayer mkdir -p /opt/genlayer-node/${MAJOR_VERSION}/data/node/keystore /opt/genlayer-node/${MAJOR_VERSION}/data/node/logs
+ echo "Created major version directory: /opt/genlayer-node/${MAJOR_VERSION}/data/node"
- # Change ownership
- sudo chown -R $USER:$USER /opt/genlayer-node
+ # Link the patch version's data/ to the shared data directory (binary and Docker)
+ sudo -u genlayer ln -sfn ../${MAJOR_VERSION}/data /opt/genlayer-node/${VERSION}/data
+ echo "Created symlink: /opt/genlayer-node/${VERSION}/data -> ../${MAJOR_VERSION}/data"
# Setup convenience symlinks pointing to current version
- ln -sfn /opt/genlayer-node/${VERSION}/bin /opt/genlayer-node/bin
- ln -sfn /opt/genlayer-node/${VERSION}/third_party /opt/genlayer-node/third_party
- ln -sfn /opt/genlayer-node/${VERSION}/data /opt/genlayer-node/data
- ln -sfn /opt/genlayer-node/${VERSION}/configs /opt/genlayer-node/configs
- ln -sfn /opt/genlayer-node/${VERSION}/docker-compose.yaml /opt/genlayer-node/docker-compose.yaml
- ln -sfn /opt/genlayer-node/${VERSION}/.env /opt/genlayer-node/.env
- ln -sfn /opt/genlayer-node/${VERSION}/alloy-config.river /opt/genlayer-node/alloy-config.river
- ln -sfn /opt/genlayer-node/${VERSION}/genvm-module-web-docker.yaml /opt/genlayer-node/genvm-module-web-docker.yaml
+ sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/bin /opt/genlayer-node/bin
+ sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/opt /opt/genlayer-node/opt
+ sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/data /opt/genlayer-node/data
+ sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/configs /opt/genlayer-node/configs
+ sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/docker-compose.yaml /opt/genlayer-node/docker-compose.yaml
+ sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/.env /opt/genlayer-node/.env
+ sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/alloy-config.river /opt/genlayer-node/alloy-config.river
+ sudo -u genlayer ln -sfn /opt/genlayer-node/${VERSION}/genvm-module-web-docker.yaml /opt/genlayer-node/genvm-module-web-docker.yaml
# Verify structure
echo "Directory structure:"
@@ -1115,8 +1121,8 @@ procedure:
ls -la /opt/genlayer-node/${VERSION}/data/node/
- step: "Run GenVM setup"
- detail: "Download GenVM dependencies."
- command: "python3 /opt/genlayer-node/third_party/genvm/bin/setup.py"
+ detail: "install.sh already ran it. Re-run only after a failure, in the final directory."
+ command: "cd /opt/genlayer-node/${VERSION} && sudo -u genlayer python3 \"$(sed -n 's/^ *root_dir: *//p' configs/node/config.yaml)/bin/setup.py\""
- step: "Create .env file"
decision_point: true
@@ -1147,8 +1153,8 @@ procedure:
sub_steps:
- name: "Rollup URLs (Required - No Defaults)"
questions:
- - "GenLayer Chain RPC URL (user must provide ZKSync HTTP RPC URL)"
- - "GenLayer Chain WebSocket URL (user must provide ZKSync WebSocket URL)"
+ - "GenLayer Chain RPC URL (user must provide the network's HTTP RPC URL)"
+ - "GenLayer Chain WebSocket URL (user must provide the network's WebSocket URL)"
- name: "Node Identity"
questions:
- "Validator Wallet Address"
@@ -1160,7 +1166,7 @@ procedure:
- "Enable genlayer group? (default: yes)"
- "Enable genlayer_debug group? (default: no)"
- "Enable ethereum group? (default: yes)"
- - "Enable zksync group? (default: yes)"
+ - "Enable zksync group? (default: no)"
- name: "Ops Endpoints"
questions:
- "Enable ops endpoints? (default: yes)"
@@ -1181,17 +1187,17 @@ procedure:
question: "How would you like to configure the operator key?"
options:
- answer: "Import from staking wizard export"
- command: "./bin/genlayernode account import --password '${PASSWORD}' --passphrase '${PASSPHRASE}' --path '${KEYSTORE_PATH}' --quiet"
+ command: "sudo test ! -e /opt/genlayer-node/data/node/keystore/operator.json && sudo install -m 600 -o genlayer -g genlayer '${KEYSTORE_PATH}' /opt/genlayer-node/data/node/keystore/operator.json # stops if a key exists (shared keystore); passphrase goes in .env as GENLAYERNODE_PASSWORD"
- answer: "Copy from previous installation"
command: "cp -r ${PREV_KEYSTORE}/* /opt/genlayer-node/data/node/keystore/"
- answer: "Generate new operator key"
- command: "./bin/genlayernode account new --setup --password '${PASSWORD}'"
+ command: "sudo -u genlayer bash -c 'set -a; . ./.env; set +a; ./bin/genlayernode account new -c $(pwd)/configs/node/config.yaml --setup'"
warnings:
- "You must update your validator's operator address after generating!"
- - "You must fund the new operator address with ETH for gas fees!"
+ - "You must fund the new operator address with GEN for gas fees!"
post_steps:
- "Note the new operator address from command output"
- - "Send ETH to the new operator address for transaction gas"
+ - "Send GEN to the new operator address for transaction gas"
- "Update validator settings via staking UI or CLI to use new operator"
- step: "Configure LLM provider"
@@ -1214,7 +1220,7 @@ procedure:
- step: "Run doctor check"
detail: "Validate configuration before starting."
- command: "cd /opt/genlayer-node && source .env && ./bin/genlayernode doctor"
+ command: "cd /opt/genlayer-node && sudo -u genlayer bash -c 'set -a; . ./.env; set +a; ./bin/genlayernode doctor'"
- step: "Decide: Deployment method"
decision_point: true
diff --git a/plugins/genlayernode/skills/genlayernode/staking-wizard-procedure.md b/plugins/genlayernode/skills/genlayernode/staking-wizard-procedure.md
index 281c6bf..6b83962 100644
--- a/plugins/genlayernode/skills/genlayernode/staking-wizard-procedure.md
+++ b/plugins/genlayernode/skills/genlayernode/staking-wizard-procedure.md
@@ -21,15 +21,21 @@ Reference: https://docs.genlayer.com/validators/setup-guide#understanding-valida
Before running the wizard:
-- **genlayer CLI** installed (`npm install -g genlayer`)
-- An **owner account** with **42,000+ GEN** tokens on GenLayer Chain
-- The CLI must be configured for the correct **network** (e.g., `testnet-asimov`)
+- **genlayer CLI** — the exact version named in the release announcement for your network
+ (`npm install -g "genlayer@${CLI_VERSION}"`). The default npm tag can select an older CLI
+ without your network's addresses. Node.js 20+ (22 for a fresh install).
+- An **owner account** with GEN for the **deployed self-stake minimum** plus gas. Read the
+ minimum with `genlayer staking epoch-info --network testnet-bradbury` — 42,000 GEN is only the
+ protocol default, not a guaranteed network setting. Delegation does not replace self-stake.
+- The CLI must be configured for the correct **network** (e.g., `testnet-bradbury`)
- The owner account must be **unlocked**
+- **Back up the owner key** (private key or seed phrase). The wizard exports only the operator
+ key; a lost owner key means the stake can never be exited or claimed.
### Verify Prerequisites
```bash
-# Check CLI is installed
+# Check CLI is installed (must be the version for your network)
genlayer --version
# List existing accounts
@@ -58,12 +64,12 @@ genlayer account use "My Validator Owner"
# Show address to fund
genlayer account show
-# -> Send 42,000+ GEN to the displayed address
+# -> Send the deployed self-stake minimum + gas to the displayed address
```
**IMPORTANT**: `account create` is interactive — it prompts for an encryption password (min 8 characters). This cannot be automated via piped input.
-## Wizard Flow (7 Steps)
+## Wizard Flow (9 Steps)
The wizard is **fully interactive** and runs in the terminal. It does NOT open a browser. All prompts require keyboard input.
@@ -74,10 +80,10 @@ The wizard is **fully interactive** and runs in the terminal. It does NOT open a
genlayer staking wizard
# Skip account and network prompts
-genlayer staking wizard --account "My Validator Owner" --network testnet-asimov
+genlayer staking wizard --account "My Validator Owner" --network testnet-bradbury
# Skip identity setup
-genlayer staking wizard --account "My Validator Owner" --network testnet-asimov --skip-identity
+genlayer staking wizard --account "My Validator Owner" --network testnet-bradbury --skip-identity
```
### Step 1: Account Setup
@@ -105,7 +111,7 @@ Using account: My Validator Owner (0x01959d0ed126285cbed368a0d9e7c4eb14d2b77b)
**Prompt**: "Select network"
-**Options**: Available networks (excludes "studionet"). Default: `testnet-asimov`.
+**Options**: Available networks (excludes "studionet"). Choose your network (e.g., `testnet-bradbury`).
**Skipped if**: `--network` flag is provided.
@@ -113,29 +119,34 @@ Using account: My Validator Owner (0x01959d0ed126285cbed368a0d9e7c4eb14d2b77b)
```
Step 2: Network Selection
-------------------------
-Using network: Genlayer Asimov Testnet
+Using network: Genlayer Bradbury Testnet
```
-### Step 3: Balance Check
+### Step 3: Funding Source
+
+**What happens**: Selects where the self-stake comes from: the owner **wallet**, or a **vesting**
+contract the owner holds.
+
+### Step 4: Balance Check
**What happens**: Verifies the owner account has enough GEN to stake.
**Requirements**:
-- After Epoch 0: Minimum 42,000 GEN + gas buffer (~0.01 GEN)
-- During Epoch 0: No minimum enforced, but validator won't activate until minimum is met
+- The deployed self-stake minimum (read on-chain; 42,000 GEN is the protocol default) + gas
+- During Epoch 0 a registered, healthy validator may correctly wait for activation
**If balance insufficient**: Wizard displays error with required amount and exits. Fund the account and re-run.
**Output on success**:
```
-Step 3: Balance Check
+Step 4: Balance Check
---------------------
Balance: 110050 GEN
Minimum stake required: 42000 GEN
Balance sufficient!
```
-### Step 4: Operator Setup
+### Step 5: Operator Setup
**What happens**: Creates or selects the operator account and exports its keystore.
@@ -145,6 +156,11 @@ Balance sufficient!
**Three paths**:
+> **Phase 2 proof of possession**: registration needs the operator public key and a proof of
+> possession bound to the owner, chain and validator-wallet factory. The operator must be an
+> account the CLI holds (Path B or C) so it can sign the proof. An external operator address alone
+> is not enough, and the old `validatorJoin(address)` flow does not work.
+
#### Path A: Same as Owner (Not recommended)
- Operator = Owner address
- No keystore export
@@ -169,7 +185,7 @@ Balance sufficient!
**Output**:
```
-Step 4: Operator Setup
+Step 5: Operator Setup
----------------------
✔ Do you want to use a separate operator address? Yes
✔ How would you like to set up the operator? Create new operator account
@@ -198,11 +214,11 @@ into your validator node software.
**CRITICAL**: Save the exported keystore file and remember the export password. You need both to import the operator key on the validator server.
-### Step 5: Stake Amount
+### Step 6: Stake Amount
**What happens**: User specifies how much GEN to stake.
-**Prompt**: "Enter stake amount (min: 42000, max: {balance} GEN)"
+**Prompt**: "Enter stake amount (min: {deployed minimum}, max: {balance} GEN)"
**Input format**: Number with optional "GEN" suffix (e.g., `42000`, `100000gen`, `50000 GEN`)
@@ -210,23 +226,23 @@ into your validator node software.
**Output**:
```
-Step 5: Stake Amount
+Step 6: Stake Amount
--------------------
✔ Enter stake amount (min: 42000, max: 110050 GEN): 100000
✔ You will stake 100000. Continue? Yes
```
-### Step 6: Join as Validator (On-chain Transaction)
+### Step 7: Join as Validator (On-chain Transaction)
**What happens**: Submits the staking transaction to the blockchain. This:
1. Stakes the specified GEN amount
2. Registers you as a validator
3. Deploys a validator wallet smart contract
-4. Designates the operator address
+4. Designates the operator (with its proof of possession)
**Output**:
```
-Step 6: Join as Validator
+Step 7: Join as Validator
-------------------------
⠴ Creating validator with 100000 GEN stake...
Result:
@@ -243,7 +259,7 @@ Result:
**The `validatorWallet` address is what you'll configure in the node's config.yaml.**
-### Step 7: Identity Setup (Optional)
+### Step 8: Identity Setup (Optional)
**What happens**: Sets public metadata for your validator.
@@ -261,9 +277,9 @@ Result:
**Skipped if**: `--skip-identity` flag is provided, or user answers "no".
-**Can be done later**: `genlayer staking set-identity`
+**Can be done later**: `genlayer staking set-identity --account "" --network testnet-bradbury`
-### Completion Summary
+### Step 9: Summary
The wizard displays a summary with all addresses and next steps:
@@ -277,14 +293,14 @@ Summary:
Owner: 0x01959d0ed126285cbed368a0d9e7c4eb14d2b77b (My Validator Owner)
Operator: 0x992fff41a1f49af2043c2017b80f5a8d66f0a47b (My Validator Operator)
Staked Amount: 100000gen
- Network: Genlayer Asimov Testnet
+ Network: Genlayer Bradbury Testnet
Next Steps:
1. Transfer operator keystore to your validator server:
/current/dir/My Validator Operator-keystore.json
2. Import it into your validator node software
3. Monitor your validator:
- genlayer staking validator-info --validator 0x1451c990...
+ genlayer staking validator-info 0x1451c990... --network testnet-bradbury
4. Lock your account when done: genlayer account lock
```
@@ -293,9 +309,10 @@ Next Steps:
| Item | Where to Find | Needed For |
|------|--------------|------------|
| **Validator Wallet Address** | Wizard summary output | `config.yaml` → `node.validatorWalletAddress` |
-| **Operator Address** | Wizard Step 4 output | `config.yaml` → `node.operatorAddress` |
-| **Operator Keystore File** | Current directory (exported file) | Upload to server → import into node |
-| **Export Passphrase** | You set it in Step 4 | Importing keystore on the server |
+| **Operator Address** | Wizard Step 5 output | `config.yaml` → `node.operatorAddress` |
+| **Operator Keystore File** | Current directory (exported file) | Upload to server → installed into the node keystore (install-procedure.md Phase 6) |
+| **Export Passphrase** | You set it in Step 5 | `GENLAYERNODE_PASSWORD` (binary) / `NODE_PASSWORD` (Docker) in the server's `.env` |
+| **Owner key backup** | Your wallet / CLI keystore | Exiting and claiming stake — not recoverable from the node |
| **Owner Account Password** | You set it during account creation | Unlocking owner account for future operations |
| **Operator funded** | `genlayer account show --account "Operator"` | Operator needs GEN for gas fees to sign validation transactions |
@@ -351,15 +368,15 @@ The skill should present the user with options based on the owner's available ba
# Transfer GEN from owner to operator
genlayer account send gen \
--account "Owner Account Name" \
- --network testnet-asimov
+ --network testnet-bradbury
```
Example:
```bash
# Owner has 60,050 GEN available, sending 50,000 to operator
genlayer account send 0x992fff41a1F49AF2043c2017B80F5a8d66f0a47B 50000gen \
- --account "Edgars Asimov" \
- --network testnet-asimov
+ --account "My Validator Owner" \
+ --network testnet-bradbury
```
#### 4. Verify Transfer
@@ -406,14 +423,16 @@ This step should be performed:
```bash
# Check validator info on-chain
-genlayer staking validator-info --validator
+genlayer staking validator-info --network testnet-bradbury
+# Registered validators that are not active yet are listed by:
+genlayer staking validators --network testnet-bradbury
# Expected output includes:
# - owner, operator addresses
# - vDeposit (staked amount)
# - selfStakePendingDeposits (shows activation epoch)
# - needsPriming status
-# - live status (false until activated)
+# - status `pending` in `genlayer staking validators` until the deposit activates
```
**Note on activation**: After staking, your deposit enters a pending state. It activates after a waiting period (typically 2 epochs). The `selfStakePendingDeposits` field shows the `activatesAtEpoch` value.
@@ -423,12 +442,12 @@ genlayer staking validator-info --validator
After your deposit activates, you may need to prime the validator:
```bash
-genlayer staking validator-prime --validator
+genlayer staking validator-prime --network testnet-bradbury
```
Check if priming is needed:
```bash
-genlayer staking validator-info --validator
+genlayer staking validator-info --network testnet-bradbury
# Look for: needsPriming: true/false
```
@@ -462,16 +481,21 @@ genlayer account unlock --account "My Validator Operator"
genlayer account export --account "My Validator Operator" --output "./operator-keystore.json"
# 4. Join as validator (non-interactive)
+# Phase 2: the operator must be an account this CLI holds, so it can sign the proof of
+# possession — an address alone fails. Check the exact flags with
+# `genlayer staking validator-join --help` of the qualified CLI version.
genlayer staking validator-join \
--account "My Validator Owner" \
- --amount 42000gen \
+ --amount gen \
--operator 0x \
- --network testnet-asimov
+ --network testnet-bradbury
# 5. Set identity (non-interactive via individual fields, or interactive via wizard)
-genlayer staking set-identity \
- --account "My Validator Owner" \
- --validator 0x
+# The CLI signs set-identity with the validator's OPERATOR account (see --help).
+genlayer staking set-identity 0x \
+ --account "My Validator Operator" \
+ --moniker "My Validator" \
+ --network testnet-bradbury
```
**Note**: Steps 1-3 require interactive password prompts. Step 4 (`validator-join`) is non-interactive if all flags are provided. However, `account create`, `account unlock`, and `account export` always require interactive password input.
@@ -492,16 +516,17 @@ genlayer account use "Account Name"
genlayer account lock --account "My Validator Owner"
# Check epoch info (staking parameters)
-genlayer staking epoch-info --network testnet-asimov
+genlayer staking epoch-info --network testnet-bradbury
# List active validators
-genlayer staking active-validators --network testnet-asimov
+genlayer staking active-validators --network testnet-bradbury
-# Change operator address later
-genlayer staking set-operator \
+# Change operator address later (signed by the owner; Phase 2 also needs the new operator's
+# proof of possession: --operator-account names its CLI account, CLI v0.40.0-rc.3+)
+genlayer staking set-operator 0x 0x \
--account "My Validator Owner" \
- --validator 0x \
- --operator 0x
+ --operator-account "My Validator Operator" \
+ --network testnet-bradbury
```
## Common Issues
@@ -509,10 +534,10 @@ genlayer staking set-operator \
### "Address is not a validator"
- The account address is not the validator wallet address
- Use the `validatorWallet` from the wizard output, not the owner address
-- Correct: `genlayer staking validator-info --validator 0x`
+- Correct: `genlayer staking validator-info 0x --network testnet-bradbury`
### Balance Insufficient
-- Fund the owner account with 42,000+ GEN
+- Fund the owner account with the deployed self-stake minimum + gas (`genlayer staking epoch-info`)
- Re-run the wizard after funding
### Account Locked
@@ -523,7 +548,7 @@ genlayer staking set-operator \
- Re-run the wizard — no on-chain changes were made
### Export Password Forgotten
-- The export password is needed to import the keystore on the server
+- The export password unlocks the keystore on the server (`GENLAYERNODE_PASSWORD` / `NODE_PASSWORD`)
- If forgotten, export again: `genlayer account export --account "Operator Name"`
- This creates a new keystore file with a new password
diff --git a/plugins/genlayernode/skills/genlayernode/update-procedure.md b/plugins/genlayernode/skills/genlayernode/update-procedure.md
index 60d038c..d51382e 100644
--- a/plugins/genlayernode/skills/genlayernode/update-procedure.md
+++ b/plugins/genlayernode/skills/genlayernode/update-procedure.md
@@ -3,24 +3,25 @@
## Goal
Minimize validator downtime during updates by preparing the new version while the old version continues running.
+Conventions (`VERSION`, `INSTALL_ROOT`, `NODE_DIR`, `MINOR`, `NODE_USER`) are the ones defined at the
+top of `install-procedure.md`. `VERSION` is the **new** version; `OLD_VERSION` the running one.
+
## Process Overview
```mermaid
flowchart TD
subgraph PHASE1["PHASE 1: PREPARATION (Old version RUNNING)"]
- OLD[Old Node v0.4.3
RUNNING]
- P1[1. Download v0.4.4]
- P2[2. Extract to new dir]
- P3[3. GenVM Setup ~2min]
- P4[4. Copy config]
- P5[5. Copy .env]
- P6[6. Enable LLM provider]
- P7[7. Symlink shared DB]
- P8[8. Copy keystore]
+ OLD[Old Node v0.6.0
RUNNING]
+ P1[1. Download + verify v0.6.1]
+ P2[2. install.sh into new dir
incl. GenVM setup ~2min]
+ P3[3. Copy config.yaml + manifest]
+ P4[4. Copy .env]
+ P5[5. Apply LLM config]
+ P6[6. Link shared data dir]
READY[NEW VERSION READY]
OLD -.-> P1
- P1 --> P2 --> P3 --> P4 --> P5 --> P6 --> P7 --> P8 --> READY
+ P1 --> P2 --> P3 --> P4 --> P5 --> P6 --> READY
end
subgraph PHASE2["PHASE 2: QUICK SWITCH (10-15 sec downtime)"]
@@ -32,10 +33,10 @@ flowchart TD
end
subgraph PHASE3["PHASE 3: VERIFICATION"]
- VERIFY[Verify version + sync + logs]
+ VERIFY[Verify version + network + sync + logs]
SUCCESS[SUCCESS
Continue validating]
FAILED[FAILED?]
- ROLLBACK[ROLLBACK
Switch back to v0.4.3]
+ ROLLBACK[ROLLBACK
Switch back to v0.6.0]
VERIFY --> SUCCESS
VERIFY --> FAILED --> ROLLBACK
@@ -49,62 +50,62 @@ flowchart TD
- OLD WAY: STOP → Download → Extract → GenVM → START = **3-4 minutes**
- NEW WAY: [Prepare while running] → STOP → Switch → START = **10-15 seconds**
-## Update Sequence (for v0.4.x patch updates)
+## Update Sequence (for v0.6.x patch updates)
-> **Shared commands documented in `common-procedures.md`**
+> **Shared commands documented in `common-procedures.md` and `install-procedure.md`.**
+> Run `pre-update-checklist.md` first.
### Phase 1: Preparation (Old Version Running)
All preparation steps happen WHILE the validator is actively running and validating:
```bash
-# 1. Check current version
-curl -s http://localhost:9153/health | jq '.node_version'
-
-# 2-4. Download, extract, set permissions
-# See: common-procedures.md -> "Download & Extract Node Software"
-VERSION=v0.4.5
-wget https://storage.googleapis.com/gh-af/genlayer-node/bin/amd64/${VERSION}/genlayer-node-linux-amd64-${VERSION}.tar.gz \
- -O /tmp/genlayer-node-${VERSION}.tar.gz
-sudo mkdir -p /opt/genlayer-node/${VERSION}
-sudo tar -xzvf /tmp/genlayer-node-${VERSION}.tar.gz \
- -C /opt/genlayer-node/${VERSION} --strip-components=1
-sudo chown -R $USER:$USER /opt/genlayer-node/${VERSION}
-
-# 5. Run GenVM setup (slow part - do while old node runs!)
-# See: common-procedures.md -> "GenVM Setup"
-python3 /opt/genlayer-node/${VERSION}/third_party/genvm/bin/setup.py
-
-# 6-8. Copy config and .env from current version
-sudo mkdir -p /opt/genlayer-node/${VERSION}/configs/node
-sudo cp /opt/genlayer-node/configs/node/config.yaml \
- /opt/genlayer-node/${VERSION}/configs/node/config.yaml
-sudo cp /opt/genlayer-node/.env /opt/genlayer-node/${VERSION}/.env
-
-# 9a. Apply LLM strategy
-# See: common-procedures.md -> "LLM Strategy Selection"
-sudo cp /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-modules-llm-release.yaml \
- /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml
-# If greybox strategy (check current config to preserve existing choice):
-grep -q 'genvm-llm-greybox.lua' /opt/genlayer-node/third_party/genvm/config/genvm-module-llm.yaml && \
- sudo sed -i 's/genvm-llm-default\.lua/genvm-llm-greybox.lua/' \
- /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml
-
-# 9b. Enable LLM provider
-# See: common-procedures.md -> "Enable LLM Provider"
-sudo sed -i '/^ :/,/^ [a-z]/ s/enabled: false/enabled: true/' \
- /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml
-
-# 10-11. Create data dir and symlink to shared database
-sudo mkdir -p /opt/genlayer-node/${VERSION}/data/node
-ln -s /opt/genlayer-node/v0.4/data/node/genlayer.db \
- /opt/genlayer-node/${VERSION}/data/node/genlayer.db
-
-# 12. Copy keystore
-sudo cp -r /opt/genlayer-node/data/node/keystore \
- /opt/genlayer-node/${VERSION}/data/node/
+# 1. Check current version and note it
+curl -s http://localhost:9153/health | jq '{status, network, node_version}'
+OLD_VERSION=$(basename "$(dirname "$(readlink -f /opt/genlayer-node/bin)")") # e.g. v0.6.0
+case "$OLD_VERSION" in v[0-9]*) ;; *) echo "cannot determine OLD_VERSION ('$OLD_VERSION') - stop" >&2; exit 1 ;; esac
+VERSION=v0.6.1 # the approved new version
+INSTALL_ROOT=/opt/genlayer-node; NODE_DIR=${INSTALL_ROOT}/${VERSION}; NODE_USER=genlayer
+MINOR=$(echo ${VERSION} | grep -oE '^v[0-9]+\.[0-9]+')
+
+# 2. Download, verify and install the new version (GenVM setup is the slow part —
+# do it while the old node runs). Same commands as install-procedure.md Phase 2
+# (see: common-procedures.md -> "Download & Extract Node Software"), into ${NODE_DIR}.
+
+# 3. Copy config + protocol manifest from the current version, keep the new GenVM path.
+# install.sh wrote the right genvm.root_dir into the NEW packaged config; the old
+# config may name a GenVM version the new package does not ship.
+NEW_ROOT=$(sed -n 's/^ *root_dir: *//p' ${NODE_DIR}/configs/node/config.yaml)
+sudo -u ${NODE_USER} cp ${INSTALL_ROOT}/${OLD_VERSION}/configs/node/config.yaml ${NODE_DIR}/configs/node/config.yaml
+sudo -u ${NODE_USER} cp ${INSTALL_ROOT}/${OLD_VERSION}/configs/node/protocol-manifest.json ${NODE_DIR}/configs/node/
+sudo -u ${NODE_USER} sed -i "s#^\( *root_dir:\).*#\1 ${NEW_ROOT}#" ${NODE_DIR}/configs/node/config.yaml
+grep -n 'root_dir' ${NODE_DIR}/configs/node/config.yaml
+
+# 4. Copy .env (secrets stay on the server; never print it)
+sudo -u ${NODE_USER} cp -p ${INSTALL_ROOT}/${OLD_VERSION}/.env ${NODE_DIR}/.env
+# Docker only: set NODE_VERSION=${VERSION} in the new .env, then pre-pull the image:
+# sudo -u ${NODE_USER} sed -i "s/^NODE_VERSION=.*/NODE_VERSION=${VERSION}/" ${NODE_DIR}/.env
+# (cd ${NODE_DIR} && sudo docker compose --profile node pull genlayer-node)
+
+# 5. Apply the LLM config, preserving the current strategy and providers.
+# See install-procedure.md Phase 4 / common-procedures.md -> "LLM Strategy Selection".
+OLD_CFG=$(cd ${INSTALL_ROOT}/${OLD_VERSION} && readlink -f "$(sed -n 's/^ *root_dir: *//p' configs/node/config.yaml)")/config
+NEW_CFG=$(cd ${NODE_DIR} && readlink -f "${NEW_ROOT}")/config
+sudo -u ${NODE_USER} cp ${NEW_CFG}/genvm-modules-llm-release.yaml ${NEW_CFG}/genvm-module-llm.yaml
+grep -q 'genvm-llm-greybox.lua' ${OLD_CFG}/genvm-module-llm.yaml && \
+ sudo -u ${NODE_USER} sed -i 's/genvm-llm-default\.lua/genvm-llm-greybox.lua/' ${NEW_CFG}/genvm-module-llm.yaml
+# enable the same providers as before ( from the common-procedures.md mapping):
+sudo -u ${NODE_USER} sed -i '/^ :/,/^ [a-z]/ s/enabled: false/enabled: true/' ${NEW_CFG}/genvm-module-llm.yaml
+# Docker only: re-create docker-compose.override.yml in ${NODE_DIR} (SKILL.md -> Docker Compose Deployment).
+
+# 6. Link the shared data directory (genvm.db, ops.db, keystore/ — no copy, no resync)
+test ! -e ${NODE_DIR}/data -o -L ${NODE_DIR}/data || { echo "${NODE_DIR}/data is a real directory - stop" >&2; exit 1; }
+sudo -u ${NODE_USER} ln -sfn ../${MINOR}/data ${NODE_DIR}/data
+ls -la ${NODE_DIR}/data/node/
```
-**At this point, everything is ready for the new version.**
+**At this point, everything is ready for the new version.** Never start the new node while the
+old one runs: both would open the same databases (one writer only), and two nodes with the same
+operator must never run at once.
### Phase 2: Quick Switch (Minimize Downtime)
Now we do the fastest possible switch:
@@ -116,14 +117,9 @@ sudo systemctl stop genlayer-node
# 2. Update symlinks (very fast - <1 second)
# See: common-procedures.md -> "Setup Symlinks"
cd /opt/genlayer-node
-ln -sfn /opt/genlayer-node/${VERSION}/bin bin
-ln -sfn /opt/genlayer-node/${VERSION}/third_party third_party
-ln -sfn /opt/genlayer-node/${VERSION}/data data
-ln -sfn /opt/genlayer-node/${VERSION}/configs configs
-ln -sfn /opt/genlayer-node/${VERSION}/docker-compose.yaml docker-compose.yaml
-ln -sfn /opt/genlayer-node/${VERSION}/.env .env
-ln -sfn /opt/genlayer-node/${VERSION}/alloy-config.river alloy-config.river
-ln -sfn /opt/genlayer-node/${VERSION}/genvm-module-web-docker.yaml genvm-module-web-docker.yaml
+for p in bin opt configs data docker-compose.yaml alloy-config.river alloy-healthcheck.sh genvm-module-web-docker.yaml .env; do
+ sudo -u genlayer ln -sfn ${VERSION}/$p $p
+done
# 3. Start new version immediately
sudo systemctl start genlayer-node
@@ -132,12 +128,18 @@ sudo systemctl start genlayer-node
# See: sharp-edges.yaml -> "alloy-stale-bind-mount"
# The Alloy container's bind mount becomes stale when symlinks change.
# Without this restart, Alloy stops sending logs to Grafana.
+# (The systemd unit's ExecStartPost also does this.)
sleep 5 # Wait for node to initialize
-docker restart genlayer-node-alloy 2>/dev/null || true
+sudo docker restart genlayer-node-alloy 2>/dev/null || true
# Total downtime: 10-15 seconds (Alloy restart is async)
```
+**Docker deployment:** from the new version directory,
+`sudo docker compose -p genlayer-node --profile node up -d`. The rc.1 compose file has no fixed
+project name: `-p genlayer-node` keeps the same project, so this recreates the existing containers
+from the new directory (without it the container names conflict).
+
> **IMPORTANT**: The Alloy restart is critical! Without it, the telemetry container
> continues reading from the old log path (stale bind mount) and your validator
> will appear offline to monitoring systems, potentially leading to penalties.
@@ -148,11 +150,11 @@ docker restart genlayer-node-alloy 2>/dev/null || true
> **See `common-procedures.md` -> "Verification Commands"**
```bash
-# Check version
-curl -s http://localhost:9153/health | jq '.node_version'
+# Check version, network and health
+curl -s http://localhost:9153/health | jq '{status, network, node_version, genvm_version}'
# Check sync status
-curl -s http://localhost:9153/health | jq '.checks.validating'
+curl -s http://localhost:9153/metrics | grep -E '^genlayer_node_(synced|blocks_behind)'
# Monitor logs
sudo journalctl -u genlayer-node -f --no-hostname
@@ -163,26 +165,33 @@ echo "=== Alloy bind mount check ==="
echo "Host log file:"
ls -la /opt/genlayer-node/data/node/logs/node.log 2>/dev/null | awk '{print $6, $7, $8, $9}'
echo "Alloy container view:"
-docker exec genlayer-node-alloy ls -la /var/log/genlayer/node.log 2>/dev/null | awk '{print $6, $7, $8, $9}'
+sudo docker exec genlayer-node-alloy ls -la /var/log/genlayer/node.log 2>/dev/null | awk '{print $6, $7, $8, $9}'
# Both timestamps should match! If they differ, Alloy has a stale bind mount.
# Check Alloy is actively tailing logs
-docker logs genlayer-node-alloy 2>&1 | tail -3
+sudo docker logs genlayer-node-alloy 2>&1 | tail -3
```
> **WARNING**: If the host and container timestamps don't match, run:
> `docker restart genlayer-node-alloy`
> See `sharp-edges.yaml` -> `alloy-stale-bind-mount` for details.
-## For Major Version Updates (e.g., v0.4.x -> v0.5.x)
+## For Major Version Updates (e.g., v0.6.x -> v0.7.x)
Major version updates may require:
-1. New shared storage directory (e.g., `/opt/genlayer-node/v0.5/`)
-2. Database migration scripts
-3. Configuration format changes
+1. New shared storage directory (e.g., `/opt/genlayer-node/v0.7/`) — follow the release notes on
+ whether the data directory carries over (move it, never copy `ops.db` to another node)
+2. Database migration (the node may migrate on first start; v0.6 renames a v0.5 `genlayer.db`
+ to `genvm.db` and refuses to start if both exist)
+3. Configuration format changes (diff the packaged template, step 3)
The principle remains: **Prepare everything first, then quick switch.**
+**v0.5.x (Bradbury Phase 1 / Asimov) -> v0.6.x (Bradbury Phase 2) is NOT an in-place update.**
+Phase 2 is a new chain (Arbitrum Orbit, chain id 4221) with new contracts and a new validator
+registration. Do a fresh install (`install-procedure.md`) in a new directory, with a fresh data
+directory; stop and disable the old node first.
+
## Downtime Comparison
| Method | Downtime | Validation Loss |
@@ -193,10 +202,11 @@ The principle remains: **Prepare everything first, then quick switch.**
## Key Principles
1. **Never stop the old node until the new node is 100% ready to start**
-2. **GenVM setup is the slowest part** - do it while old node runs
-3. **Shared database** - instant access to current sync state
+2. **GenVM setup is the slowest part** - do it while old node runs (install.sh runs it in the final directory)
+3. **Shared data directory** - instant access to current sync state and node state (`genvm.db`, `ops.db`, keystore)
4. **Symlink switching** - takes <1 second
5. **Test the new version setup** - verify files exist before switching
+6. **One node at a time** - never run old and new together (same databases, same operator)
## Rollback Procedure (if new version fails)
@@ -208,25 +218,23 @@ sudo systemctl stop genlayer-node
# 2. Switch symlinks back to old version
cd /opt/genlayer-node
-ln -sfn /opt/genlayer-node/v0.4.3/bin bin
-ln -sfn /opt/genlayer-node/v0.4.3/third_party third_party
-ln -sfn /opt/genlayer-node/v0.4.3/data data
-ln -sfn /opt/genlayer-node/v0.4.3/configs configs
-ln -sfn /opt/genlayer-node/v0.4.3/docker-compose.yaml docker-compose.yaml
-ln -sfn /opt/genlayer-node/v0.4.3/.env .env
-ln -sfn /opt/genlayer-node/v0.4.3/alloy-config.river alloy-config.river
-ln -sfn /opt/genlayer-node/v0.4.3/genvm-module-web-docker.yaml genvm-module-web-docker.yaml
+for p in bin opt configs data docker-compose.yaml alloy-config.river alloy-healthcheck.sh genvm-module-web-docker.yaml .env; do
+ sudo -u genlayer ln -sfn ${OLD_VERSION}/$p $p
+done
# 3. Start old version
sudo systemctl start genlayer-node
# 4. Refresh Alloy telemetry (same reason as upgrade)
sleep 5
-docker restart genlayer-node-alloy 2>/dev/null || true
+sudo docker restart genlayer-node-alloy 2>/dev/null || true
# Total rollback time: 10-15 seconds
```
+The data directory is shared: if the new version migrated the databases on its first start, the
+old version may not open them. Check the release notes before rolling back across a migration.
+
## Pre-Update Checklist
Before starting any update:
diff --git a/plugins/genlayernode/skills/genlayernode/validations.yaml b/plugins/genlayernode/skills/genlayernode/validations.yaml
index eceb6c5..c68f9d2 100644
--- a/plugins/genlayernode/skills/genlayernode/validations.yaml
+++ b/plugins/genlayernode/skills/genlayernode/validations.yaml
@@ -27,9 +27,9 @@ validations:
- id: nodejs-installed
type: command
- command: "node --version 2>/dev/null | grep -E 'v(1[89]|[2-9][0-9])' >/dev/null"
+ command: "node --version 2>/dev/null | grep -E '^v(2[0-9]|[3-9][0-9])' >/dev/null"
expect: "exit 0"
- message: "Node.js v18+ is required. Install with: curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - && sudo apt install -y nodejs"
+ message: "Node.js v20+ is required on the machine that runs the GenLayer CLI (not on the node server). Install with: curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - && sudo apt install -y nodejs"
- id: npm-installed
type: command
@@ -41,19 +41,19 @@ validations:
type: command
command: "command -v docker >/dev/null 2>&1"
expect: "exit 0"
- message: "Docker is required. Install with: curl -fsSL https://get.docker.com | sh"
+ message: "Docker is required. Install with: curl -fsSL https://get.docker.com | env -u VERSION sh"
- id: docker-compose-installed
type: command
- command: "docker compose version >/dev/null 2>&1"
+ command: "docker compose version --short 2>/dev/null | sed 's/^v//' | awk -F. 'NR==1{ok=($1>2||($1==2&&$2>=24))} END{exit !ok}'"
expect: "exit 0"
- message: "Docker Compose is required. Usually included with Docker."
+ message: "Docker Compose v2.24+ is required (the release docker-compose.yaml uses env_file required: false). Usually included with Docker; update the docker-compose-plugin package."
- id: python3-installed
type: command
- command: "command -v python3 >/dev/null 2>&1"
+ command: "python3 -c 'import sys; sys.exit(sys.version_info < (3,10))' >/dev/null 2>&1"
expect: "exit 0"
- message: "Python 3 is required. Install with: sudo apt install python3"
+ message: "Python 3.10+ is required. Install with: sudo apt install python3 (Ubuntu 22.04+/Debian 12+)"
- id: python3-pip-installed
type: command
@@ -63,7 +63,7 @@ validations:
- id: python3-venv-installed
type: command
- command: "dpkg -l python3-venv >/dev/null 2>&1 || python3 -m venv --help >/dev/null 2>&1"
+ command: "python3 -c 'import venv, ensurepip' >/dev/null 2>&1"
expect: "exit 0"
message: "python3-venv is required. Install with: sudo apt install python3-venv"
@@ -73,6 +73,12 @@ validations:
expect: "exit 0"
message: "wget is required. Install with: sudo apt install wget"
+ - id: cosign-installed
+ type: command
+ command: "command -v cosign >/dev/null 2>&1"
+ expect: "exit 0"
+ message: "cosign is required to verify the release signature. Install with: curl -fsSLo /tmp/cosign.deb https://github.com/sigstore/cosign/releases/download/v2.4.1/cosign_2.4.1_amd64.deb && sudo dpkg -i /tmp/cosign.deb"
+
- id: jq-installed
type: command
command: "command -v jq >/dev/null 2>&1"
@@ -87,7 +93,7 @@ validations:
type: command
command: "command -v genlayer >/dev/null 2>&1"
expect: "exit 0"
- message: "GenLayer CLI not installed. Install with: npm install -g genlayer"
+ message: "GenLayer CLI not installed. Install the exact version for your network: npm install -g genlayer@"
# ============================================================================
# DOCKER STATE
@@ -123,9 +129,15 @@ validations:
- id: genvm-binaries-exist
type: command
- command: "[ -f third_party/genvm/bin/genvm ] && [ -f third_party/genvm/bin/genvm-modules ]"
+ command: R=$(sed -n 's/^ *root_dir:[[:space:]]*//p' configs/node/config.yaml) && [ -f "$R/bin/genvm-modules" ] && ls "$R"/executor/*/bin/genvm >/dev/null 2>&1
expect: "exit 0"
- message: "GenVM binaries not found. Run: python3 ./third_party/genvm/bin/setup.py"
+ message: "GenVM binaries not found under genvm.root_dir (v0.6+: opt/genvm/). Run: python3 \"$R/bin/setup.py\" in the install directory"
+
+ - id: protocol-manifest-present
+ type: command
+ command: "[ -f configs/node/protocol-manifest.json ]"
+ expect: "exit 0"
+ message: "Protocol manifest not found (required since v0.6). Download your network's manifest from its docs section to configs/node/protocol-manifest.json (Testnet Bradbury: https://docs.genlayer.com/validators/bradbury.protocol-manifest.json)"
- id: llm-key-configured
type: command
@@ -141,7 +153,7 @@ validations:
- id: node-health-ok
type: command
- command: "curl -sf http://localhost:9153/health | grep -q 'up'"
+ command: curl -sf http://localhost:9153/health | jq -e '.status == "healthy"' >/dev/null
expect: "exit 0"
message: "Node health check failed. Check if node is running."
@@ -189,27 +201,31 @@ validations:
- id: rpc-url-configured
type: command
- command: "grep -q 'GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCURL' /opt/genlayer-node/.env 2>/dev/null || grep -q 'GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCURL' .env 2>/dev/null"
+ command: grep -Eq '^ genlayerchainrpcurl:[[:space:]]*"https://' /opt/genlayer-node/configs/node/config.yaml
expect: "exit 0"
- message: "RPC URL not configured in .env file"
+ message: "RPC URL not configured in config.yaml (rollup.genlayerchainrpcurl)"
+
+ - id: rpc-key-configured
+ type: command
+ command: "sudo awk -F= '/^GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCKEY=/{f=length($2)>0} END{exit !f}' /opt/genlayer-node/.env"
+ expect: "exit 0"
+ message: "GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCKEY (RPC provider API key) is not set in .env: when the provider needs one (Testnet Bradbury: Conduit), the WebSocket refuses the connection and the node does not start"
- id: validator-address-configured
type: command
- command: "grep -q 'GENLAYERNODE_NODE_VALIDATORWALLETADDRESS' /opt/genlayer-node/.env 2>/dev/null || grep -q 'GENLAYERNODE_NODE_VALIDATORWALLETADDRESS' .env 2>/dev/null"
+ command: grep -Eq '^ mode:[[:space:]]*"full"' /opt/genlayer-node/configs/node/config.yaml || grep -Eq '^ validatorWalletAddress:[[:space:]]*"0x[0-9a-fA-F]{40}"' /opt/genlayer-node/configs/node/config.yaml
expect: "exit 0"
- message: "Validator address not configured in .env file"
+ message: "Validator address not configured in config.yaml (node.validatorWalletAddress; not needed for a full node)"
- id: rpc-url-reachable
type: command
- command: |
- sh -c '. /opt/genlayer-node/.env 2>/dev/null; [ -n "$GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCURL" ] || . ./.env 2>/dev/null; [ -n "$GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCURL" ] || exit 1; curl -sf -m 30 -X POST "$GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCURL" -H "Content-Type: application/json" -d '\''{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}'\'' | grep -q "\"result\":\"0x"'
+ command: U=$(sed -n 's/^ genlayerchainrpcurl:[[:space:]]*"\([^"]*\)".*/\1/p' /opt/genlayer-node/configs/node/config.yaml); [ -n "$U" ] || exit 1; curl -sf -m 30 -X POST "$U" -H Content-Type:application/json -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' | grep -q '"result":"0x'
expect: "exit 0"
message: "Cannot reach the configured rollup RPC endpoint. Check network connectivity."
- id: configured-rpc-chainid-reachable
type: command
- command: |
- sh -c '. /opt/genlayer-node/.env 2>/dev/null; [ -n "$GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCURL" ] || . ./.env 2>/dev/null; [ -n "$GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCURL" ] || exit 1; curl -sf -m 10 -X POST "$GENLAYERNODE_ROLLUP_GENLAYERCHAINRPCURL" -H "Content-Type: application/json" -d '\''{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}'\'' | grep -q "\"result\":\"0x"'
+ command: U=$(sed -n 's/^ genlayerchainrpcurl:[[:space:]]*"\([^"]*\)".*/\1/p' /opt/genlayer-node/configs/node/config.yaml); [ -n "$U" ] || exit 1; curl -sf -m 10 -X POST "$U" -H Content-Type:application/json -d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}' | grep -q '"result":"0x'
expect: "exit 0"
message: "Configured rollup RPC did not answer eth_chainId. This may be a shared RPC outage; verify before changing node config."
@@ -258,16 +274,17 @@ validations:
message: |
Systemd service missing ExecStartPost for Alloy restart.
Add this line after ExecStart in /etc/systemd/system/genlayer-node.service:
- ExecStartPost=-/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy 2>/dev/null || true'
+ ExecStartPost=-+/bin/sh -c 'sleep 5 && /usr/bin/docker restart genlayer-node-alloy >/dev/null 2>&1 || true'
Then run: sudo systemctl daemon-reload
# Validations that must pass before installation
on_stop:
- architecture-amd64
- - nodejs-installed
- docker-installed
- docker-running
- python3-installed
+ - python3-venv-installed
+ - cosign-installed
# Validations that are warnings
on_warn:
@@ -276,7 +293,6 @@ on_warn:
- jq-installed
- docker-no-sudo
- python3-pip-installed
- - python3-venv-installed
- systemd-has-alloy-restart
# Post-update validations (run after upgrade)