performance optimization

[He-Pin]

New numbers are updated:
https://github.com/CertainLach/jrsonnet/blob/master/docs/benchmarks.adoc

We should optimize the performance

[He-Pin]

Performance Optimization Progress Report

Benchmark Reference: jrsonnet 0.5.0-pre97 vs sjsonnet 0.6.2 (from jrsonnet benchmarks.adoc)

CPU Reference: AMD Ryzen 9 9950X3D (jrsonnet's benchmarks) | Apple Silicon M-series (our local testing)

Current Gap Analysis (Scala-Native / Rust ratio, sorted worst-first)

#	Benchmark	Rust (ms)	Scala-N (ms)	Ratio	Status
1	Foldl string concat	3.7	328.6	88.8x	Fixed — PR #665 (StringBuilder O(n))
2	std.foldl	2.2	80.0	36.4x	Fixed — PR #665 (while-loop foldl)
3	Large string template	2.1	14.5	6.9x	Optimizing — PR #689 (bulk text block scanner)
4	Kube-prometheus	43.6	204.6	4.7x	In progress (composite benchmark)
5	Big object	839.2	3756.8	4.5x	Optimizing — PR #687/#688 (inline object storage)
6	std.base64	1.2	4.8	4.0x	Planned
7	Comparison for array	10.0	39.1	3.9x	Optimizing — PR #674 (ref equality) + PR #686 (inline binop)
8	Large string join	2.6	9.7	3.7x	Planned
9	std.base64Decode	1.2	4.2	3.5x	Planned
10	Realistic 2	120.9	417.8	3.5x	In progress (composite benchmark)
11	std.manifestTomlEx	750.5	2558.8	3.4x	Backlog
12	std.manifestJsonEx	748.5	2443.6	3.3x	Backlog
13	std.substr	917.8	2991.7	3.3x	Backlog
14	Array sorts	1.3	4.2	3.2x	Fixed — PR #664 (in-place Arrays.sort)
15	Inheritance function recursion	743.5	2394.1	3.2x	Backlog
16	String strips	751.5	2377.2	3.2x	Planned (ASCII bitset)
17	std.parseInt	771.7	2383.6	3.1x	Backlog
18	Tail call	865.2	2406.8	2.8x	Backlog
19	Realistic 1	5.9	12.7	2.2x	Improved — now 1.19x faster on combined branch
20	Graalvm CI	34.4	69.6	2.0x	In progress (composite benchmark)
21	std.base64DecodeBytes	19.2	37.2	1.9x	Planned
22	std.reverse	24.5	44.1	1.8x	Planned
23	Comparison for primitives	218.1	174.8	0.80x	Already faster
24	Simple recursive call	35.8	18.0	0.50x	Already faster
25	std.base64 (byte array)	15.6	7.1	0.46x	Already faster
26	Lazy array	54.8	19.3	0.35x	Already faster
27	Inheritance recursion	160.1	46.7	0.29x	Already faster

Combined Optimization Results (24 PRs merged, Apple Silicon M-series)

Tested with source-built jrsonnet 0.5.0-pre98 on the same machine:

Benchmark	Master (ms)	Optimized (ms)	jrsonnet (ms)	Speedup	vs jrsonnet
realistic1	15.6	11.8	14.1	-24%	1.19x faster
std.foldl	144.3	5.4	5.7	-96%	1.04x faster
comparison2 (primitives)	168.4	65.4	196.5	-61%	3.0x faster
gen_big_object	13.0	11.1	11.3	-15%	1.02x faster
bench.02	74.2	66.6	113.7	-10%	1.71x faster
bench.03	32.4	32.5	51.5	0%	1.59x faster
base64_byte_array	10.3	9.9	17.8	-4%	1.81x faster
realistic2	292.1	241.7	97.1	-17%	2.49x slower
comparison (array)	32.8	26.6	12.7	-19%	2.09x slower
base64	10.4	8.3	5.2	-20%	1.59x slower
bench.01 (tail call)	8.1	8.0	4.8	-1%	1.67x slower
reverse	49.1	30.6	22.7	-38%	1.35x slower
base64DecodeBytes	40.8	25.9	20.2	-36%	1.28x slower

Open PRs (24 performance optimizations)

All PRs target master as draft PRs for review:

Area	PRs	Impact
Core evaluation	#663, #667, #668, #674, #675, #682, #686	Comprehension fuse, singleton bools, structural equality, valTag dispatch
Object creation	#671, #687, #688	Sorted key cache, single/multi-field inline storage
Stdlib	#664, #665, #669, #670, #683	In-place sort, foldl StringBuilder, range allocation, map/flatMap, num cache
Rendering	#676, #678, #680, #681, #684, #685	Indent cache, escape fast-path, direct stdout, bulk append, num stringify, direct long
Format	#679	Runtime format optimization
Parser	#689	Bulk text block scanner
Array	#673	arraycopy concat

Key Achievements

• std.foldl: 144ms to 5.4ms (26.6x speedup) — now faster than jrsonnet
• realistic1: 15.6ms to 11.8ms — now 1.19x faster than jrsonnet
• comparison2: 168ms to 65ms — now 3.0x faster than jrsonnet
• 5 benchmarks already faster than jrsonnet (Scala-native vs Rust)

Next Optimization Targets

1. realistic2 (2.49x gap) — largest comprehensive benchmark, needs object creation + field lookup optimizations
2. comparison (array) (2.09x) — array element comparison fast path
3. large_string_template (2.46x) — string formatting optimization
4. lstripChars (1.72x) — ASCII bitset optimization
5. base64 (1.59x) — encoding algorithm optimization
6. reverse (1.35x) — array reverse caching