Benchmark Report
Eight gated micro-benchmarks measured on a fresh process per run, best of 5. Every case is also a differential test: the bytecode VM and the tree interpreter must print identical output, and each run must reproduce its own output exactly.
01Performance gate
Floors are set at roughly a third of the worst rate observed on reference hardware. They exist to catch catastrophic regressions (a 3×+ slowdown) on shared CI runners, not small drift — a passing gate is a floor, not a target.
| Benchmark | Iters | Throughput | Floor | Margin | Headroom | Status |
|---|---|---|---|---|---|---|
| Arithmetic & Loop2M-iteration integer accumulation loop | 2,000,000 | 178.79M/s | 6.00M | 29.8× | PASS | |
| String Building30K string concatenations and slices | 30,000 | 2.86M/s | 140k | 20.4× | PASS | |
| List Churn200K pushes then pops, summing values | 200,000 | 12.01M/s | 800k | 15.0× | PASS | |
| Map / Dict40K hash inserts and lookups | 40,000 | 2.15M/s | 160k | 13.4× | PASS | |
| Method Dispatch200K method calls on objects | 200,000 | 9.48M/s | 500k | 19.0× | PASS | |
| Closures40K closure allocations and calls | 40,000 | 3.33M/s | 170k | 19.6× | PASS | |
| JSON Encode / Decode2K encode/decode round-trips | 2,000 | 145k/s | 8k | 18.1× | PASS | |
| Spawn & Join2K spawn/join task round-trips | 2,000 | 187k/s | 6k | 31.2× | PASS |
Throughput is iters/sec, where one iteration is one full body of the benchmark file. Bars are log-scaled: the track spans 4× to 20× the floor.
02Bytecode VM vs tree interpreter
The same source runs on both engines. The VM compiles to bytecode with constant folding and peephole passes; the tree interpreter walks the AST directly. Numbers are milliseconds, best of 5 (VM) and 3 (tree).
| Benchmark | VM (ms) | Tree (ms) | VM is | Tree / VM |
|---|---|---|---|---|
| Arithmetic & Loop | 11.2 | 55.3 | 4.94× faster | |
| String Building | 10.5 | 24.6 | 2.34× faster | |
| List Churn | 16.7 | 36.0 | 2.16× faster | |
| Map / Dict | 18.6 | 27.6 | 1.48× faster | |
| Method Dispatch | 21.1 | 108.2 | 5.13× faster | |
| Closures | 12.0 | 51.7 | 4.30× faster | |
| JSON Encode / Decode | 13.8 | 14.4 | 1.05× faster | |
| Spawn & Join | 10.7 | 12.0 | 1.12× faster | |
| Stress Suitecorrectness only, not gated | 28.7 | 32.5 | 1.13× faster |
03Against PHP and Python
The same workloads implemented in PHP 8.3.33 and Python 3.15.0, measured the same way: one fresh process per run, best of 21, wall clock including interpreter startup.
| Benchmark | Spid3r (ms) | PHP (ms) | Python (ms) |
|---|---|---|---|
| Arithmetic & Loop | 11.2 | 42.7 | 54.0 |
| String Building | 10.5 | 25.7 | 15.6 |
| List Churn | 16.7 | 26.3 | 19.8 |
| Map / Dict | 18.6 | 24.7 | 18.0 |
| Method Dispatch | 21.1 | 29.9 | 19.6 |
| Closures | 12.0 | 23.4 | 16.3 |
| JSON Encode / Decode | 13.8 | 24.1 | 24.6 |
| Spawn & Join | 10.7 | 22.5 | 19.4 |
Startup-adjusted compute time
Total wall clock minus each runtime's measured startup, in milliseconds. Lower is better. This is an estimate: startup measured by a no-op invocation understates the fixed cost of loading a program, so treat differences under ~1 ms as noise.
| Benchmark | Spid3r | PHP | Python | Fastest |
|---|---|---|---|---|
| Arithmetic & Loop | 3.59 | 21.1 | 40.3 | Spid3r |
| String Building | 2.90 | 4.13 | 1.95 | Python |
| List Churn | 9.06 | 4.75 | 6.09 | PHP |
| Map / Dict | 11.0 | 3.10 | 4.30 | PHP |
| Method Dispatch | 13.5 | 8.37 | 5.96 | Python |
| Closures | 4.41 | 1.86 | 2.66 | PHP |
| JSON Encode / Decode | 6.19 | 2.50 | 10.9 | PHP |
| Spawn & Join | 3.09 | 0.98 | 5.74 | PHP |
Startup-adjusted compute time, log scale — shorter is faster.
04Phase 11 P-1: Native AOT & Node.js / Rust reference
Cross-language compute comparison on identical checksum-pinned workloads (benchmarks/compare/{prime,json,map})
under the Phase 0 protocol (9 warmups, 21 measured repeats, 2 passes, MAD÷median ≤ 5%, two-pass agreement ≤ 3%,
blocking 5% sensitivity self-test PASS). Raw artifact:
benchmarks/reference.p1.json (ADVISORY_ONLY).
| Workload | Spid3r VM | Spid3r native | Node.js | Rust (-O) | Native vs VM | Native vs Node |
|---|---|---|---|---|---|---|
prime |
31.15 ms1.41% MAD / 1.83% ag | 8.74 ms3.63% MAD / 1.23% ag | 1.70 ms1.17% MAD / 0.06% ag | 1.36 ms0.59% MAD / 0.29% ag | 3.56× | 5.13× slower |
json |
167.92 ms1.42% MAD / 0.39% ag | 199.05 ms0.70% MAD / 0.19% ag | 17.60 ms0.34% MAD / 0.78% ag | — | 0.84× | 11.31× slower |
map |
21.21 ms1.48% MAD / 0.78% ag | 45.17 ms0.42% MAD / 1.20% ag | 1.56 ms1.22% MAD / 0.13% ag | — | 0.47× | 28.92× slower |
wall_ms − startup_baseline_ms exclusively.
Where process spawn is >50% of wall time for fast reference runs (Node 60.6–91.1%, Rust 83.7%),
the table reports high-resolution internal timers (process.hrtime.bigint() / Instant::now())
as the stable compute metric while retaining external subtraction in the JSON log.
On prime, Spid3r native (8.74 ms) is 5.13× slower than Node and 6.42× slower than Rust — Phase 11 P-2..P-9 targets this gap.
Per-engine dispersion and two-pass agreement ledger (Phase 11 P-1)
| Workload | Engine | Metric basis | Median | Ext startup share | MAD/med (≤5%) | 2-pass (≤3%) | Gate |
|---|---|---|---|---|---|---|---|
| prime | Spid3r VM | external | 31.152 ms | 28.3% | 1.41% | 1.83% | PASS |
| prime | Spid3r native (AOT) | external | 8.743 ms | 51.5% | 3.63% | 1.23% | PASS |
| prime | Node.js (v26.11.1) | internal hr | 1.704 ms | 89.0% | 1.17% | 0.06% | PASS |
| prime | Rust -O (rustc 1.99.1) | internal hr | 1.362 ms | 83.7% | 0.59% | 0.29% | PASS |
| json | Spid3r VM | external | 167.918 ms | 7.0% | 1.42% | 0.39% | PASS |
| json | Spid3r native (AOT) | external | 199.051 ms | 5.5% | 0.70% | 0.19% | PASS |
| json | Node.js (v26.11.1) | internal hr | 17.600 ms | 60.6% | 0.34% | 0.78% | PASS |
| map | Spid3r VM | external | 21.206 ms | 35.2% | 1.48% | 0.78% | PASS |
| map | Spid3r native (AOT) | external | 45.171 ms | 17.6% | 0.42% | 1.20% | PASS |
| map | Node.js (v26.11.1) | internal hr | 1.562 ms | 91.1% | 1.22% | 0.13% | PASS |
05Phase 11 P-2: Selfhost toolchain throughput baselines
Frozen Stage-0 (Rust) vs selfhost (Stage-1 AOT --release) compiler throughput across checksummed
workloads in benchmarks/selfhost/ under the Phase 0 protocol (9 warmups, 21 measured repeats, 2 passes,
MAD÷median ≤ 5%, two-pass agreement ≤ 3%, core 6; blocking 5% sensitivity self-test
PASS, numeric band
PASS). Raw artifact:
benchmarks/selfhost/baseline.json (ADVISORY_ONLY).
Phase 1–7 gates read their Tier-A floors directly from this table.
| Stage | Stage 0 (Rust) | Selfhost (AOT) | Selfhost vs Stage 0 | Tier-A floor (−20%) |
|---|---|---|---|---|
| Lexer1.02 MB positive corpus | 16.551 MB/s61.4 ms | 0.240 MB/s4241.8 ms | 0.015× Stage 0 | ≥ 0.192 MB/s |
| Parser121 corpus files | 87.4 prog/s1385.0 ms | 72.0 prog/s1681.5 ms | 0.824× Stage 0 | ≥ 57.6 prog/s |
| Semantic checker121 corpus files | 91.8 prog/s1317.8 ms | 54.0 prog/s2242.2 ms | 0.588× Stage 0 | ≥ 43.2 prog/s |
| Self-compile17 modules / 585 KB | 0.457 MB/s1280.4 ms | 0.048 MB/s12123.3 ms | 0.106× Stage 0 | ≥ 0.039 MB/s |
| Corpus build6 standalone examples | 975.7 ms (6.15/s)975.7 ms | 2205.9 ms (2.72/s)2205.9 ms | 2.26× slower | ≤ 2757.4 ms |
Per-stage dispersion and two-pass agreement ledger (Phase 11 P-2)
| Workload | Compiler | Median compute | MAD/med (≤5%) | 2-pass (≤3%) | Gate |
|---|---|---|---|---|---|
| lex | Stage 0 (Rust) | 61.44 ms | 0.80% | 0.75% | PASS |
| lex | Selfhost (AOT) | 4241.82 ms | 0.59% | 0.13% | PASS |
| parse | Stage 0 (Rust) | 1385.00 ms | 0.78% | 0.60% | PASS |
| parse | Selfhost (AOT) | 1681.52 ms | 2.44% | 1.70% | PASS |
| check | Stage 0 (Rust) | 1317.81 ms | 0.92% | 0.35% | PASS |
| check | Selfhost (AOT) | 2242.21 ms | 2.52% | 2.12% | PASS |
| self_compile | Stage 0 (Rust) | 1280.35 ms | 0.99% | 0.61% | PASS |
| self_compile | Selfhost (AOT) | 12123.26 ms | 0.61% | 0.50% | PASS |
| corpus_build | Stage 0 (Rust) | 975.66 ms | 1.70% | 0.05% | PASS |
| corpus_build | Selfhost (AOT) | 2205.89 ms | 1.24% | 1.25% | PASS |
06Phase 11 P-8: Honest three-way target table (Spid3r-AOT vs Node vs Rust)
Every published figure traces directly to recorded Phase 11 runs (P-1,
P-2, P-3, P-5, and P-6; Core-6 affinity, 9w/21r×2,
blocking 5% sensitivity selftest PASS, ADVISORY_ONLY shared Windows host).
No target is presented as a promise without its Tier label and recorded measurement:
Tier A blocking floor ·
Tier B measured delta vs frozen baseline ·
Tier C stretch / structurally gapped (never blocking; retired via docs/PERF_TARGET_RETIREMENT.md).
| Workload / Gate | Tier | Spid3r (AOT) | Node.js (V8) | Rust (rustc -O) |
Ledger ref | Disposition & structural gap |
|---|---|---|---|---|---|---|
HLC prime50k primes checksum loop |
Tier B Tier C | 5.76 ms compute12.85 ms wall; −39.0% vs 9.44 ms (P-6) | 1.70 ms intV8 JIT (3.38× slower) | 1.36 ms intrustc -O (4.23× slower) | 11.6 (P-6) + 11.1 (P-1) | Tier B METSub-4× vs JIT/rustc retired to Tier C (overflow checks + cancel poll) |
HLC jsonencode/decode workload |
Tier B Tier C | 148.82 ms compute156.62 ms wall; −21.3% vs 189.06 ms (P-6) | 17.60 ms intV8 C++/CSA (8.46× slower) | n/a (absent)not in vendored HLC suite | 11.6 (P-6) + 11.1 (P-1) | RETIRED (R1)Collection-bound: Arc<Mutex<MapData>> + FFI boxing vs V8 hidden classes |
HLC map100k lookup/insert loop |
Tier B Tier C | 28.00 ms compute35.70 ms wall; −37.3% vs 44.67 ms (P-6) | 1.56 ms intV8 IC/SMI (17.93× slower) | n/a (absent)not in vendored HLC suite | 11.6 (P-6) + 11.1 (P-1) | RETIRED (R1/R3)Dynamic index locks Arc<Mutex> per lookup vs inline V8 cache |
Selfhost lex1.02 MB corpus |
Tier A Tier B ret. | 1387.53 ms (0.733 MB/s)2.97× vs P-2; floor ≥ 0.192 MB/s | — | 61.18 ms (16.62 MB/s)Stage-0 Rust (22.7× faster) | 11.6 (P-6) + 1.4 | Tier A PASS≥1/2 Stage-0 retired (1.4 ledger): raw &[u8] vs dynamic bytes FFI |
Selfhost parse & check121 corpus files |
Tier A Tier B | 928.75 / 1081.87 ms130.3/s & 111.8/s (floors ≥ 57.6 / 43.2) | — | 1137.04 / 1213.28 msSelfhost is 1.22× / 1.12× faster | 11.6 (P-6) + 11.2 (P-2) | METSelfhost front-end outpaces Stage-0 Rust CLI batches |
Selfhost self_compile17 modules / 592 KB |
Tier A Tier B | 5932.15 ms (0.0998 MB/s)1.46× vs 8674 ms; floor ≥ 0.0386 MB/s | — | 1134.79 ms (0.522 MB/s)Stage-0 Rust (5.23× faster) | 11.6 (P-6) + 11.2 (P-2) | Tier A/B PASSBoxed AST traversal + LLVM-C call marshalling vs Rust native passes |
Selfhost corpus_build6 standalone examples |
Tier A | 673.84 ms (8.90/s)floor ≤ 2757.4 ms | — | 678.19 ms (8.85/s)Stage-0 Rust (1.01× parity) | 11.6 (P-6) + 11.2 (P-2) | METStage 1 matches Stage 0 end-to-end build latency |
| Codegen quality & CLI lightnessS1/S0 parity + startup + envelope | Tier A | 1.0007× corpus med (max 1.0435×)startup 8.099 ms (≤8.5); VM 10.27 MiB | — | Stage-0 default<O3>HLC S1/S0 0.995×–1.007× |
11.3 (P-3) + 11.5 (P-5) | METBit-identical Stage 2↔3 object; release budgets enforced in CI/local |
i64 arithmetic, periodic cancellation polling (CANCEL_POLL_INTERVAL=2048), and Arc<Mutex<MapData>> runtime containers across FFI trampolines.
(2) Rust compiles directly to unboxed stack/SSA values with zero runtime type-tag overhead. Targets shown as unreachable under Spid3r’s safety/no-new-dependency/engine-contract invariants are retired openly in docs/PERF_TARGET_RETIREMENT.md (§4 & §8, pending owner confirmation) rather than relaxed silently.
07Phase 11 P-9: Continuous performance log (W-program trajectory)
End-to-end record from the first selfhost lexer measurement (Phase 1) through compiler hardening and
hot-path engineering (P-1–P-8), backed by frozen JSON baselines and checked by
tools/bench_report.py --verify-sync.
| Milestone | Date | Focus | Measured Outcome (baseline → after) | Ledger ref |
|---|---|---|---|---|
| Phase 1.2 / 1.4 | 2026-11-14 | Selfhost lexer quadratic fix & baseline | Fragment joins + bytes O(1) indexing cut 1 MB lex from 300s+ OOM to 5.4s (0.24 MB/s); Tier-B ≥1/2 Stage-0 (0.005×) retired openly to Tier-A floor ≥ 0.19 MB/s. |
1.2 / 1.4 ledger |
| Phase 11 P-1 | 2026-11-15 | Three-way HLC reference locked | prime: VM 31.15, AOT 8.74, Node 1.70, Rust 1.36 ms; json: VM 167.92, AOT 199.05, Node 17.60 ms; map: VM 21.21, AOT 45.17, Node 1.56 ms (benchmarks/reference.p1.json). |
11.1 (P-1) |
| Phase 11 P-2 | 2026-11-16 | Selfhost toolchain baseline frozen | Stage-1 AOT: lex 4241.82 ms (0.240 MB/s), parse 1681.52 ms (72.0/s), check 2242.21 ms (54.0/s), self_compile 12123.26 ms (0.0482 MB/s), corpus_build 2205.89 ms (benchmarks/selfhost/baseline.json). |
11.2 (P-2) |
| Phase 11 P-3 | 2026-11-16 | Stage-1 release <O3> & runtime alignment |
Fixed debug staticlib lookup + wired LLVM PassBuilder default<O3>: HLC S1/S0 parity 0.995×–1.007× (prime 1.237× vs unopt IR); 111-corpus median 1.0007× (max 1.0435× ≤ 1.25×); self_compile ~16.3 s vs 35.5 s. |
11.3 (P-3) |
| Phase 11 P-4 | 2026-11-16 | ThinLTO + split PGO hardening | Replaced process::exit with ExitCode so CRT flushes .profraw: self_compile S0 1313.41 → 1144.20 ms (−12.88%), S1 9637.79 → 8901.70 ms (−7.64%, −26.57% vs P-2); binaries −8.6% to −9.6%. |
11.4 (P-4) |
| Phase 11 P-5 | 2026-11-16 | Lazy init & lightness budgets | Lazy stdlib/tzdata resolution + single-read CLI: spid3r --version 8.099 ms ≤ 8.5 ms Tier-A floor; VM 10.27 MiB WS / 2.75 MiB commit; release size ceilings in benchmarks/budgets.p5.json. |
11.5 (P-5) |
| Phase 11 P-6 | 2026-11-16 | W4.1–W4.3 hot-path backlog | W4.1 bump-slab + small-int cache + zero-alloc dispatch, W4.2 lexer BS.slice().to_string() + <18-digit int fast path, W4.3 top-level scalar const propagation: HLC AOT prime 5.76 ms (−39.0%), json 148.82 ms (−21.3%), map 28.00 ms (−37.3%); selfhost lex 1387.53 ms (2.97×), parse 928.75 ms (1.22× Stage 0), check 1081.87 ms (1.12× Stage 0), self_compile 5932.15 ms (1.46×). |
11.6 (P-6) |
| Phase 11 P-7 | 2026-11-17 | CI advisory perf tracking | Extended Tier-A floors to all 5 selfhost workloads and wired the 2-consecutive-run >1.05× && >3× pooled MAD rule + actions/cache@v4 deltas into phase0 and aot CI jobs; teeth verified red (exit 1) on regressed lex runs. |
11.7 (P-7) |
| Phase 11 P-8 / P-9 | 2026-11-17 | Honest target table & doc sync | Published Section 06 three-way Tier A/B/C table, structural-gap notes, docs/PERF_TARGET_RETIREMENT.md §8, and this continuous log synced between benchmark.html and frozen JSON baselines. |
11.8 – 11.9 |
08Method and caveats
- One process per run. Each sample is a fresh OS process; the harness takes the best of N. Nothing is warmed up across runs, so JIT-less runtimes are not penalised by a cold-start handicap they would not have in a long-lived server.
- Correctness is part of the measurement.
--comparerequires the VM and tree engines to print byte-identical output, and every repeat run must reproduce its own output, or the benchmark is reported as failed rather than timed. - Micro-benchmarks, not applications. These isolate a loop, a hash table or a call path. They say nothing about startup-to-first-byte for a real program, memory ceiling, or I/O.
- Absolute numbers are host-specific. The CPU on this machine runs at its base clock under the Balanced power plan; recorded numbers from a turbo-boosted run are ~25–30% higher across every workload and every runtime, including PHP. Compare ratios, not raw milliseconds, across machines.
- Not measured here: the AOT/native backend (needs LLVM, runs in its own CI job) and the WebAssembly build (see the playground).
Reproduce
cargo build --release ./target/release/spid3r bench --gate --compare --runs 5 # this page, from the same measurements python tools/bench_report.py
Raw samples: benchmarks/report-data.json. The historical comparison set
(2026-10-20) lives in benchmarks/results.json.