Skip to content

fix: align benchmark runtimes with project target frameworks#157

Merged
LeeCampbell merged 4 commits intoHdrHistogram:mainfrom
LeeCampbell:fix/benchmark-target-frameworks
Mar 20, 2026
Merged

fix: align benchmark runtimes with project target frameworks#157
LeeCampbell merged 4 commits intoHdrHistogram:mainfrom
LeeCampbell:fix/benchmark-target-frameworks

Conversation

@LeeCampbell
Copy link
Collaborator

Summary

  • Removed 7 obsolete benchmark runtime jobs (Net481 LegacyJit/RyuJit, Core 2.1, 3.1, 5.0, 6.0, 7.0) that no longer match the project's target frameworks
  • Added missing net9.0 and net10.0 jobs to match the csproj's TargetFrameworks (net8.0;net9.0;net10.0)

Test plan

  • dotnet build HdrHistogram.Benchmarking/ succeeds with 0 errors
  • dotnet format HdrHistogram.Benchmarking/ --verify-no-changes passes
  • Run benchmarks locally to confirm all three runtimes execute correctly

🤖 Generated with Claude Code

LeeCampbell and others added 4 commits March 20, 2026 12:49
Replace obsolete benchmark job configuration (Net481, Core 2.1–7.0)
with the actual target frameworks: net8.0, net9.0, and net10.0.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
BenchmarkDotNet 0.15.8 does not recognise the net10.0 moniker via
CreateForNewVersion, throwing NotImplementedException. Remove the
manual job/column config and use DefaultConfig instead. Specific
runtimes can be targeted via CLI args (--runtimes net8.0 net9.0).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Restore the original percentile/statistics columns and multi-runtime
job configuration. Keep net8.0 and net9.0 runtimes (net10.0 is not
yet supported by BenchmarkDotNet 0.15.8). Update build.cmd to pass
--runtimes net8.0 net9.0 so both configured runtimes are exercised.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
BenchmarkDotNet 0.15.8 supports net10.0 via CoreRuntime.Core10_0,
not CreateForNewVersion. Revert build.cmd to original form without
--runtimes flag as the jobs are configured in code.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@LeeCampbell
Copy link
Collaborator Author

Benchmark Results — Updated Runtime Config

Environment: BenchmarkDotNet v0.15.8, Windows 11, Intel Core i5-14400 2.50GHz, 16 logical / 10 physical cores
.NET SDK 10.0.104 — runtimes: .NET 8.0.25, .NET 9.0.14, .NET 10.0.4

LeadingZeroCount 64-Bit

Full results (click to expand)
Method Runtime Mean Error StdDev Op/s Ratio
CurrentImplementation .NET 8.0 0.4212 ns 0.0085 ns 0.0107 ns 2,374,134,286.7 1.00
CurrentImplementation .NET 9.0 0.4347 ns 0.0074 ns 0.0088 ns 2,300,697,013.7 1.00
CurrentImplementation .NET 10.0 0.4199 ns 0.0037 ns 0.0035 ns 2,381,356,428.1 1.00

LeadingZeroCount 32-Bit

Full results (click to expand)
Method Runtime Mean Error StdDev Op/s Ratio
CurrentImplementation .NET 8.0 0.4305 ns 0.0036 ns 0.0032 ns 2,322,925,901.9 1.00
CurrentImplementation .NET 9.0 0.4381 ns 0.0083 ns 0.0078 ns 2,282,460,547.5 1.00
CurrentImplementation .NET 10.0 0.4343 ns 0.0074 ns 0.0085 ns 2,302,434,700.4 1.00

Recording 32-Bit

Method Runtime Mean Op/s Ratio
LongHistogramRecording .NET 8.0 1.677 ns 596,146,928.2 1.00
LongHistogramRecording .NET 9.0 1.528 ns 654,455,498.1 1.00
LongHistogramRecording .NET 10.0 1.767 ns 565,886,595.3 1.00
IntHistogramRecording .NET 8.0 1.402 ns 713,181,057.3 0.84
IntHistogramRecording .NET 9.0 1.295 ns 772,060,501.0 0.85
IntHistogramRecording .NET 10.0 1.329 ns 752,385,018.9 0.75
ShortHistogramRecording .NET 8.0 1.493 ns 669,787,728.5 0.89
ShortHistogramRecording .NET 9.0 1.223 ns 817,436,327.0 0.80
ShortHistogramRecording .NET 10.0 1.203 ns 831,437,495.2 0.68

Note: The ByteBuffer benchmark report in the artifacts is from a prior run with the old (stale) runtime config — all non-installed runtimes show NA. It will need to be re-run once the ByteBuffer benchmark branch is rebased onto this fix.

All three existing benchmark suites ran successfully across .NET 8.0, 9.0, and 10.0 with the updated runtime configuration.

@LeeCampbell LeeCampbell merged commit 85c8788 into HdrHistogram:main Mar 20, 2026
2 checks passed
@LeeCampbell LeeCampbell deleted the fix/benchmark-target-frameworks branch March 20, 2026 06:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant