Skip to content

feat: add load-test payload worker for base-load-test integration #171

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
meyer9 merged 16 commits into from
Apr 6, 2026
Merged

feat: add load-test payload worker for base-load-test integration #171

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
55090f5
feat: add load-test payload worker for base-load-test binary integration
meyer9 Mar 27, 2026
d17d2e8
fix: use UnmarshalBinary for typed transaction decoding in proxy
meyer9 Mar 27, 2026
6a6799c
fix: handle errcheck lint errors in load-test worker
meyer9 Mar 27, 2026
c584a34
feat: integrate load-test config into benchmark payload definition
meyer9 Mar 27, 2026
0f641f4
Revert "fix: reduce gzip noise in proxy DebugResponse"
meyer9 Mar 27, 2026
8d306fd
fix: suppress errcheck lint for tmpFile.Close in error path
meyer9 Mar 27, 2026
7ddffa4
feat: add load-test CI workflow and example config
meyer9 Mar 27, 2026
4811b31
fix: point load-test workflow at feature branch for base-load-test bi…
meyer9 Mar 27, 2026
fbbf5a7
fix: point all workflows at base repo SHA with base-load-test binary
meyer9 Mar 27, 2026
7c09461
fix: disable Rust toolchain cache in _build-binaries
meyer9 Mar 27, 2026
e507e7c
fix: use branch name for base repo checkout and fetch remote refs
meyer9 Mar 27, 2026
5e021c7
fix: use versions.env for base repo version
meyer9 Mar 27, 2026
c8c37de
fix: specify package for base-load-test binary target
meyer9 Mar 27, 2026
b98b1a6
fix: split cargo build into two commands for separate packages
meyer9 Mar 27, 2026
f9c9f8e
fix: use proxy port from config
meyer9 Mar 31, 2026
93d3a25
fix
meyer9 Apr 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 13 additions & 3 deletions .github/workflows/_build-binaries.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ on:
description: "Base Reth Node version to build"
required: false
type: string
default: "main"
default: "feature/load-test-benchmark"

# Set minimal permissions for all jobs by default
permissions:
Expand Down Expand Up @@ -155,6 +155,8 @@ jobs:

- name: Set up Rust
uses: actions-rust-lang/setup-rust-toolchain@9399c7bb15d4c7d47b27263d024f0a4978346ba4 # v1.11.0
with:
cache: false

- name: Cache base-reth-node binaries
uses: actions/cache@2f8e54208210a422b2efd51efaa6bd6d7ca8920f # v3.4.3
Expand All @@ -163,9 +165,10 @@ jobs:
path: |
~/bin/base-reth-node
~/bin/builder
key: ${{ runner.os }}-base-reth-node-builder-${{ inputs.base_reth_node_version }}
~/bin/base-load-test
key: ${{ runner.os }}-base-reth-node-builder-load-test-${{ inputs.base_reth_node_version }}

- name: Build base-reth-node and base-builder
- name: Build base-reth-node, base-builder, and base-load-test
if: steps.cache-base-reth-node.outputs.cache-hit != 'true'
run: |
unset CI
Expand All @@ -188,6 +191,13 @@ jobs:
path: ~/bin/builder
retention-days: 1

- name: Upload base-load-test artifact
uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # v4.5.0
with:
name: base-load-test
path: ~/bin/base-load-test
retention-days: 1

build-op-program:
runs-on: ubuntu-latest
permissions:
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/examples.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ jobs:
with:
optimism_version: 3019251e80aa248e91743addd3e833190acb26f1
geth_version: 6cbfcd5161083bcd4052edc3022d9f99c6fe40e0
base_reth_node_version: main

example-benchmarks:
runs-on: ubuntu-latest
Expand Down
96 changes: 96 additions & 0 deletions .github/workflows/load-test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
name: Load Test

on:
pull_request:
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

jobs:
build-binaries:
name: Build binaries
uses: ./.github/workflows/_build-binaries.yaml

load-test:
Comment on lines +13 to +16

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {}

Copilot Autofix

AI 16 days ago

In general, the fix is to add an explicit permissions: block to the workflow (at the top level or per job) that grants only the minimal scopes required. Because this workflow only reads the repository and uses artifacts, it can safely use contents: read and no other write scopes. Defining permissions at the workflow root applies to all jobs that do not override it.

The best fix here is to add a root-level permissions: block just after the on: block, setting contents: read. This documents that the workflow only needs read access to the repo contents and ensures the GITHUB_TOKEN will not unexpectedly have broader privileges if organization or repository defaults change. No functional behavior of the workflow changes, since none of the steps require write access to repository contents or other resources.

Concretely:

  • Edit .github/workflows/load-test.yaml.
  • After line 5 (workflow_dispatch:) and before the concurrency: block, insert:
permissions:
  contents: read

No imports or additional methods are needed because this is purely a YAML configuration change for the workflow.

Suggested changeset 1
.github/workflows/load-test.yaml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/load-test.yaml b/.github/workflows/load-test.yaml
--- a/.github/workflows/load-test.yaml
+++ b/.github/workflows/load-test.yaml
@@ -4,6 +4,9 @@
   pull_request:
   workflow_dispatch:
 
+permissions:
+  contents: read
+
 concurrency:
   group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
   cancel-in-progress: true
EOF
@@ -4,6 +4,9 @@
pull_request:
workflow_dispatch:

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
Copilot is powered by AI and may make mistakes. Always verify output.
name: Run load test benchmark
runs-on: ubuntu-latest
needs: [build-binaries]
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1
with:
egress-policy: audit

- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

- name: Set up Go
uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6.1.0

- name: Download base-reth-node
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: base-reth-node
path: ${{ runner.temp }}/bin/

- name: Download builder
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: builder
path: ${{ runner.temp }}/bin/

- name: Download base-load-test
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: base-load-test
path: ${{ runner.temp }}/bin/

- name: Make binaries executable
run: chmod +x ${{ runner.temp }}/bin/*

- name: Run load test benchmark
run: |
mkdir -p ${{ runner.temp }}/data-dir
mkdir -p ${{ runner.temp }}/output

go run benchmark/cmd/main.go \
--log.level info \
run \
--config configs/examples/load-test.yml \
--root-dir ${{ runner.temp }}/data-dir \
--output-dir ${{ runner.temp }}/output \
--builder-bin ${{ runner.temp }}/bin/builder \
--base-reth-node-bin ${{ runner.temp }}/bin/base-reth-node \
--load-test-bin ${{ runner.temp }}/bin/base-load-test

- name: Setup Node.js
if: always()
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version: "20"

- name: Build Report
if: always()
run: |
cp -r ${{ runner.temp }}/output/ ./output/ || true
pushd report
npm install
npm run build
popd

- name: Upload Output
if: always()
uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # v4.5.0
with:
name: load-test-output
path: ${{ runner.temp }}/output/
retention-days: 7

- name: Upload Report
if: always()
uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # v4.5.0
with:
name: load-test-report
path: report/dist/
retention-days: 7
Comment on lines +17 to +96

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 16 days ago

In general, the fix is to add an explicit permissions block so that the GITHUB_TOKEN used by this workflow is restricted to the least privilege it needs. For a typical test/build workflow that only checks out code and works with artifacts, contents: read is usually sufficient. This matches CodeQL’s suggested minimal starting point and is unlikely to break existing behavior.

The single best fix here, without changing functionality, is to add a root-level permissions block just after the on: section in .github/workflows/load-test.yaml. This will apply to both jobs (build-binaries and load-test) unless they define their own permissions. Based on the visible steps, the workflow only needs read access to repository contents to allow actions/checkout to function, and no write scopes are required. Thus we can safely set:

permissions:
  contents: read

No additional imports, methods, or definitions are needed; this is purely a YAML configuration change within the workflow file.

Suggested changeset 1
.github/workflows/load-test.yaml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/load-test.yaml b/.github/workflows/load-test.yaml
--- a/.github/workflows/load-test.yaml
+++ b/.github/workflows/load-test.yaml
@@ -4,6 +4,9 @@
   pull_request:
   workflow_dispatch:
 
+permissions:
+  contents: read
+
 concurrency:
   group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
   cancel-in-progress: true
EOF
@@ -4,6 +4,9 @@
pull_request:
workflow_dispatch:

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
Copilot is powered by AI and may make mistakes. Always verify output.
32 changes: 21 additions & 11 deletions benchmark/flags/flags.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,22 +15,24 @@ func prefixEnvVars(name string) []string {
}

const (
ConfigFlagName = "config"
RootDirFlagName = "root-dir"
OutputDirFlagName = "output-dir"
TxFuzzBinFlagName = "tx-fuzz-bin"
ProxyPortFlagName = "proxy-port"
BenchmarkRunIDFlagName = "benchmark-run-id"
MachineTypeFlagName = "machine-type"
MachineProviderFlagName = "machine-provider"
MachineRegionFlagName = "machine-region"
FileSystemFlagName = "file-system"
ConfigFlagName = "config"
RootDirFlagName = "root-dir"
OutputDirFlagName = "output-dir"
TxFuzzBinFlagName = "tx-fuzz-bin"
LoadTestBinFlagName = "load-test-bin"
ProxyPortFlagName = "proxy-port"
BenchmarkRunIDFlagName = "benchmark-run-id"
MachineTypeFlagName = "machine-type"
MachineProviderFlagName = "machine-provider"
MachineRegionFlagName = "machine-region"
FileSystemFlagName = "file-system"
ParallelTxBatchesFlagName = "parallel-tx-batches"
)

// TxFuzz defaults
const (
DefaultTxFuzzBin = "../tx-fuzz/cmd/livefuzzer/livefuzzer"
DefaultTxFuzzBin = "../tx-fuzz/cmd/livefuzzer/livefuzzer"
DefaultLoadTestBin = "./base-load-test"
)

var (
Expand Down Expand Up @@ -62,6 +64,13 @@ var (
EnvVars: opservice.PrefixEnvVar(EnvVarPrefix, "TX_FUZZ_BIN"),
}

LoadTestBinFlag = &cli.StringFlag{
Name: LoadTestBinFlagName,
Usage: "Load test binary path",
Value: DefaultLoadTestBin,
EnvVars: opservice.PrefixEnvVar(EnvVarPrefix, "LOAD_TEST_BIN"),
}

ProxyPortFlag = &cli.IntFlag{
Name: "proxy-port",
Usage: "Proxy port",
Expand Down Expand Up @@ -116,6 +125,7 @@ var RunFlags = []cli.Flag{
RootDirFlag,
OutputDirFlag,
TxFuzzBinFlag,
LoadTestBinFlag,
ProxyPortFlag,
BenchmarkRunIDFlag,
MachineTypeFlag,
Expand Down
16 changes: 12 additions & 4 deletions clients/build-base-reth-node.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,13 @@ fi

# Checkout specified version/commit
echo "Checking out version: $BASE_RETH_NODE_VERSION"
git checkout -f "$BASE_RETH_NODE_VERSION"
git fetch origin "$BASE_RETH_NODE_VERSION" || true
git checkout -f "$BASE_RETH_NODE_VERSION" || git checkout -f "origin/$BASE_RETH_NODE_VERSION"

# Build the binaries using cargo
echo "Building base-reth-node and base-builder with cargo..."
# Build with maxperf profile
echo "Building base-reth-node, base-builder, and base-load-test with cargo..."
cargo build --bin base-reth-node --bin base-builder --profile maxperf
cargo build -p base-load-tests --bin base-load-test --profile maxperf

# Copy binaries to output directory
echo "Copying binaries to output directory..."
Expand Down Expand Up @@ -74,4 +75,11 @@ else
exit 1
fi

echo "base-reth-node and base-builder binaries built successfully and placed in $FINAL_OUTPUT_DIR/"
if [ -f "target/maxperf/base-load-test" ]; then
cp target/maxperf/base-load-test "$FINAL_OUTPUT_DIR/"
else
echo "No base-load-test binary found"
exit 1
fi

echo "Binaries built successfully and placed in $FINAL_OUTPUT_DIR/"
2 changes: 1 addition & 1 deletion clients/versions.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ GETH_VERSION="v1.101604.0"

# Base Reth Node Configuration
BASE_RETH_NODE_REPO="https://github.com/base/base"
BASE_RETH_NODE_VERSION="main"
BASE_RETH_NODE_VERSION="feature/load-test-benchmark"

# Build Configuration
# BUILD_DIR="./build"
Expand Down
29 changes: 29 additions & 0 deletions configs/examples/load-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Load test throughput test
description: Test builder throughput using base-load-test binary as transaction generator
payloads:
- name: Load Test
type: load-test
id: load-test
sender_count: 10
transactions:
- weight: 70
type: transfer
- weight: 20
type: calldata
max_size: 256
- weight: 10
type: precompile
target: sha256

benchmarks:
- variables:
- type: payload
value: load-test
- type: node_type
value: builder
- type: validator_node_type
value: base-reth-node
- type: num_blocks
value: 10
- type: gas_limit
value: 1000000000
17 changes: 9 additions & 8 deletions runner/clients/common/proxy/proxy.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ import (
"github.com/ethereum/go-ethereum/common"
ethTypes "github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/rlp"
)

type ProxyServer struct {
Expand Down Expand Up @@ -196,11 +195,13 @@ func (p *ProxyServer) OverrideRequest(method string, rawParams json.RawMessage)
return false, nil, fmt.Errorf("failed to decode hex: %w", err)
}

err = rlp.DecodeBytes(rawTxBytes, &tx)
// Use UnmarshalBinary to support both legacy and typed (EIP-2718) transactions.
// The previous rlp.DecodeBytes only handled legacy transactions.
err = tx.UnmarshalBinary(rawTxBytes)

if err != nil {
p.log.Error("failed to decode RLP", "err", err)
return false, nil, fmt.Errorf("failed to decode RLP: %w", err)
p.log.Error("failed to decode transaction", "err", err)
return false, nil, fmt.Errorf("failed to decode transaction: %w", err)
}

p.pendingTxs = append(p.pendingTxs, &tx)
Expand All @@ -217,21 +218,21 @@ func (p *ProxyServer) DebugResponse(method string, params json.RawMessage, respB
p.log.Debug("method", "method", method)
p.log.Debug("params", "params", params)

// Try gzip decompression; fall back to raw body if the response is plain JSON.
gzipReader, err := gzip.NewReader(bytes.NewReader(respBody))
if err != nil {
p.log.Debug("Response body", "body", string(respBody))
p.log.Error("Error creating gzip reader", "err", err)
return
}
defer func() {
if err := gzipReader.Close(); err != nil {
p.log.Debug("Error closing gzip reader", "err", err)
p.log.Error("Error closing gzip reader", "err", err)
}
}()

uncompressedBody, err := io.ReadAll(gzipReader)

if err != nil {
p.log.Debug("Error reading uncompressed response body", "err", err)
p.log.Error("Error reading uncompressed response body", "err", err)
return
}
p.log.Debug("Uncompressed body", "body", string(uncompressedBody))
Expand Down
Loading
Loading