Skip to content
Browse the docs

Java parity: the simulator against the frozen engine

View on GitHubReport a problem with this page

Every case, run and kept knee matches the frozen engine exactly. All 2,980 runs of the 149 template cases replay it figure for figure and digest for digest, and all 65 kept calibrations' knees are its knees.

Generated by ./gradlew :archsim-benchmarks:run --args="java-parity". The formal criterion, stage 4's bar (the port plan): for every template case, with 20 seeds each, the Java simulator's median p50, p95, throughput and each part's utilization fall inside the frozen engine 0.16.0's seed range and within 2% of its median; every kept calibration's re-simulated knee is within 2% of the frozen engine's.

  • 149 of 149 template cases meet the bar: 149 of 149 inside Python's seed range, 149 of 149 within 2% of its median; 0 with a seed failing otherwise than Python's (runs failed: Python 0, Java 0).
  • 2,980 of 2,980 runs replay the frozen engine's exactly, every figure kept; 2,980 give its result digest, and 149 of 149 cases replay in every seed.
  • No metric is off: every Java median is Python's.

Cases that miss the bar#

None: every metric of every case is inside Python's range and within 2% of its median, and no seed failed otherwise than Python's.

Kept calibrations' knees#

65 of 65 kept calibrations' re-simulated knees are within 2% of the frozen engine's, 65 the same; 65 of 65 checked on the frozen engine, 65 of 65 on Java, 0 with other derived inputs than Python's.

Experiment Calibration Python's knee (req/s) Java's knee (req/s) Same inputs Within 2%
cache-aside 2026-10-26T11-53-13Z 3,455.6 3,455.6 yes yes
cache-aside 2026-10-30T10-50-27Z 2,351 2,351 yes yes
cache-aside 2026-10-30T22-22-04Z 5,306.9 5,306.9 yes yes
cache-aside 2026-10-31T00-38-29Z 4,229.8 4,229.8 yes yes
cache-aside 2026-10-31T02-44-32Z 4,474.5 4,474.5 yes yes
cache-aside 2026-11-01T23-49-28Z 4,932.5 4,932.5 yes yes
cache-aside 2026-11-02T01-58-24Z 5,327.4 5,327.4 yes yes
cache-aside 2026-11-02T04-11-08Z 5,219.8 5,219.8 yes yes
cache-aside 2026-11-04T08-18-36Z 6,007.3 6,007.3 yes yes
cache-aside 2026-11-04T10-33-51Z 5,452.1 5,452.1 yes yes
cache-aside 2026-11-04T12-50-54Z 5,652.4 5,652.4 yes yes
cache-ttl 2026-11-04T00-02-42Z 5,303.2 5,303.2 yes yes
cache-ttl 2026-11-04T03-05-38Z 5,401.4 5,401.4 yes yes
cache-ttl 2026-11-04T06-11-13Z 5,104.1 5,104.1 yes yes
chain 2026-11-03T23-27-46Z 4,353.1 4,353.1 yes yes
chain 2026-11-04T02-28-58Z 4,530.6 4,530.6 yes yes
chain 2026-11-04T05-34-15Z 4,455.3 4,455.3 yes yes
go-mixed 2026-11-03T22-16-39Z 9,517.4 9,517.4 yes yes
go-mixed 2026-11-04T01-13-30Z 9,802.4 9,802.4 yes yes
go-mixed 2026-11-04T04-19-22Z 9,915.6 9,915.6 yes yes
java-mixed 2026-11-03T21-34-40Z 7,915.3 7,915.3 yes yes
java-mixed 2026-11-04T00-33-23Z 7,857.3 7,857.3 yes yes
java-mixed 2026-11-04T03-38-18Z 7,906.8 7,906.8 yes yes
kafka 2026-10-26T12-22-50Z 995.7 995.7 yes yes
kafka 2026-10-30T11-20-01Z 1,601.1 1,601.1 yes yes
kafka 2026-10-30T23-00-02Z 2,828.3 2,828.3 yes yes
kafka 2026-10-31T01-19-14Z 2,139.4 2,139.4 yes yes
kafka 2026-10-31T03-26-27Z 2,100.8 2,100.8 yes yes
kafka 2026-11-02T00-26-56Z 2,551.2 2,551.2 yes yes
kafka 2026-11-02T02-36-13Z 2,578.4 2,578.4 yes yes
kafka 2026-11-02T04-48-49Z 2,527.9 2,527.9 yes yes
kafka 2026-11-04T08-59-13Z 2,676.9 2,676.9 yes yes
kafka 2026-11-04T11-12-22Z 2,727.5 2,727.5 yes yes
kafka 2026-11-04T13-27-44Z 2,090.2 2,090.2 yes yes
pg-hot-row 2026-10-26T08-42-10Z 418.6 418.6 yes yes
pg-hot-row 2026-10-26T10-11-09Z 503.8 503.8 yes yes
pg-hot-row 2026-10-26T11-33-38Z 444.7 444.7 yes yes
pg-hot-row 2026-10-30T10-32-56Z 519 519 yes yes
pg-hot-row 2026-10-30T21-58-50Z 730.3 730.3 yes yes
pg-hot-row 2026-10-31T00-14-43Z 1,055.3 1,055.3 yes yes
pg-hot-row 2026-10-31T02-21-39Z 862.2 862.2 yes yes
pg-hot-row 2026-11-01T23-24-56Z 862.3 862.3 yes yes
pg-hot-row 2026-11-02T01-33-51Z 851.3 851.3 yes yes
pg-hot-row 2026-11-02T03-46-41Z 851 851 yes yes
pg-hot-row 2026-11-04T07-49-19Z 825.9 825.9 yes yes
pg-hot-row 2026-11-04T10-07-36Z 862.8 862.8 yes yes
pg-hot-row 2026-11-04T12-24-30Z 856.6 856.6 yes yes
pg-mixed 2026-10-26T08-23-18Z 2,314.2 2,314.2 yes yes
pg-mixed 2026-10-26T09-46-26Z 708.8 708.8 yes yes
pg-mixed 2026-10-26T11-13-41Z 2,064.6 2,064.6 yes yes
pg-mixed 2026-10-28T23-21-36Z 2,238.6 2,238.6 yes yes
pg-mixed 2026-10-30T21-30-44Z 4,228.7 4,228.7 yes yes
pg-mixed 2026-10-30T23-44-23Z 5,106.5 5,106.5 yes yes
pg-mixed 2026-10-31T01-52-33Z 5,002.6 5,002.6 yes yes
pg-mixed 2026-10-31T04-02-33Z 4,728.4 4,728.4 yes yes
pg-mixed 2026-10-31T04-34-12Z 4,955.1 4,955.1 yes yes
pg-mixed 2026-11-01T22-53-59Z 4,779.9 4,779.9 yes yes
pg-mixed 2026-11-02T01-02-03Z 4,855.6 4,855.6 yes yes
pg-mixed 2026-11-02T03-15-28Z 5,054.7 5,054.7 yes yes
pg-mixed 2026-11-04T07-15-57Z 5,405.6 5,405.6 yes yes
pg-mixed 2026-11-04T09-35-52Z 5,059.4 5,059.4 yes yes
pg-mixed 2026-11-04T11-52-16Z 5,203.3 5,203.3 yes yes
python-mixed 2026-11-03T22-58-02Z 1,850.5 1,850.5 yes yes
python-mixed 2026-11-04T01-55-30Z 1,788.1 1,788.1 yes yes
python-mixed 2026-11-04T05-00-48Z 1,762.4 1,762.4 yes yes

Speed#

Each template as shipped (no scenarios): the median of 5 runs on each engine, measured one engine after the other on the same machine with no other heavy work running; docs/performance.md says how much the runs spread. The Java runs follow a warm-up run in the same JVM and include StrictMath's cost: the simulator takes its logarithms, exponentials and powers from StrictMath, so it draws Python's random numbers on every platform (ADR-0047).

Template Requests Python 0.16.0 Java Speed-up Same result
auth-token-introspection 112,701 27.67 s 2.01 s x13.8 yes
black-friday-kubernetes 107,990 36.72 s 1.78 s x20.6 yes
cache-aside-starter 23,980 4.00 s 0.33 s x12.0 yes
cdn-video-streaming 149,940 16.62 s 1.15 s x14.5 yes
cqrs-kafka-read-model 102,483 33.98 s 2.26 s x15.0 yes
flash-sale-order-flow 107,976 25.90 s 1.87 s x13.9 yes
iot-cassandra-quorum 99,869 17.56 s 1.30 s x13.5 yes
kafka-order-pipeline 54,106 8.29 s 0.53 s x15.7 yes
kubernetes-pods-pending 108,476 18.93 s 1.33 s x14.2 yes
kubernetes-shared-pool 110,391 28.86 s 2.08 s x13.9 yes
llm-rag-assistant 5,975 1.78 s 0.16 s x11.4 yes
microservices-call-chain 60,115 17.92 s 1.29 s x13.9 yes
mongodb-hot-shard 111,811 34.20 s 2.35 s x14.6 yes
mysql-replica-lag 89,666 17.29 s 1.26 s x13.7 yes
nats-live-telemetry 99,869 30.15 s 2.00 s x15.1 yes
notification-fanout-quotas 4,875 2.32 s 0.18 s x12.5 yes
public-api-slow-dependency 134,825 30.36 s 2.09 s x14.5 yes
queueing-lab 24,154 1.34 s 0.08 s x17.0 yes
retry-storm-ttl-cache 103,083 19.26 s 1.31 s x14.7 yes
right-size-catalog-api 107,845 20.10 s 1.53 s x13.1 yes
search-opensearch-catalog 111,811 32.89 s 2.71 s x12.1 yes
serverless-burst 95,689 19.58 s 1.15 s x17.1 yes
serverless-image-pipeline 43,761 13.18 s 1.12 s x11.8 yes
social-feed-hot-key 119,888 26.88 s 2.06 s x13.0 yes
sqlserver-saas-blocking 134,462 24.83 s 2.63 s x9.4 yes

Each case#

Template Case Runs Exact replays Meets the bar Furthest metric Python's median Java's median Off by
auth-token-introspection base 20 20 yes none off
auth-token-introspection baseline 20 20 yes none off
auth-token-introspection idp-down 20 20 yes none off
auth-token-introspection local-jwt-idp-down 20 20 yes none off
auth-token-introspection no-cache 20 20 yes none off
auth-token-introspection short-ttl 20 20 yes none off
black-friday-kubernetes base 20 20 yes none off
black-friday-kubernetes baseline 20 20 yes none off
black-friday-kubernetes cache-down 20 20 yes none off
black-friday-kubernetes fixed-pods 20 20 yes none off
black-friday-kubernetes payments-slow 20 20 yes none off
black-friday-kubernetes tight-limit 20 20 yes none off
cache-aside-starter base 20 20 yes none off
cache-aside-starter baseline 20 20 yes none off
cache-aside-starter cache-hit-20 20 20 yes none off
cache-aside-starter cache-hit-95 20 20 yes none off
cache-aside-starter one-api-replica 20 20 yes none off
cache-aside-starter three-api-replicas 20 20 yes none off
cache-aside-starter traffic-burst 20 20 yes none off
cdn-video-streaming base 20 20 yes none off
cdn-video-streaming baseline 20 20 yes none off
cdn-video-streaming cold-start 20 20 yes none off
cdn-video-streaming premiere 20 20 yes none off
cdn-video-streaming shield-down 20 20 yes none off
cdn-video-streaming short-ttl 20 20 yes none off
cqrs-kafka-read-model base 20 20 yes none off
cqrs-kafka-read-model baseline 20 20 yes none off
cqrs-kafka-read-model hot-key 20 20 yes none off
cqrs-kafka-read-model hot-key-more-partitions 20 20 yes none off
cqrs-kafka-read-model index-down 20 20 yes none off
cqrs-kafka-read-model index-down-retry 20 20 yes none off
cqrs-kafka-read-model projector-slow 20 20 yes none off
flash-sale-order-flow base 20 20 yes none off
flash-sale-order-flow baseline 20 20 yes none off
flash-sale-order-flow carrier-hangs 20 20 yes none off
flash-sale-order-flow carrier-hangs-buffer 20 20 yes none off
flash-sale-order-flow carrier-hangs-visibility 20 20 yes none off
flash-sale-order-flow deal 20 20 yes none off
flash-sale-order-flow deal-short-tx 20 20 yes none off
iot-cassandra-quorum all 20 20 yes none off
iot-cassandra-quorum base 20 20 yes none off
iot-cassandra-quorum baseline 20 20 yes none off
iot-cassandra-quorum one 20 20 yes none off
iot-cassandra-quorum slow-ring 20 20 yes none off
kafka-order-pipeline base 20 20 yes none off
kafka-order-pipeline baseline 20 20 yes none off
kafka-order-pipeline db-slow 20 20 yes none off
kafka-order-pipeline more-consumers 20 20 yes none off
kafka-order-pipeline more-partitions 20 20 yes none off
kubernetes-pods-pending base 20 20 yes none off
kubernetes-pods-pending baseline 20 20 yes none off
kubernetes-pods-pending fast-nodes 20 20 yes none off
kubernetes-pods-pending headroom 20 20 yes none off
kubernetes-pods-pending no-ca 20 20 yes none off
kubernetes-shared-pool autoscaler 20 20 yes none off
kubernetes-shared-pool base 20 20 yes none off
kubernetes-shared-pool baseline 20 20 yes none off
kubernetes-shared-pool cap-search 20 20 yes none off
kubernetes-shared-pool more-nodes 20 20 yes none off
kubernetes-shared-pool search-target 20 20 yes none off
llm-rag-assistant base 20 20 yes none off
llm-rag-assistant baseline 20 20 yes none off
llm-rag-assistant cache-down 20 20 yes none off
llm-rag-assistant launch 20 20 yes none off
llm-rag-assistant launch-provisioned 20 20 yes none off
llm-rag-assistant llm-slow 20 20 yes none off
microservices-call-chain base 20 20 yes none off
microservices-call-chain baseline 20 20 yes none off
microservices-call-chain breaker 20 20 yes none off
microservices-call-chain inverted 20 20 yes none off
microservices-call-chain no-timeout 20 20 yes none off
microservices-call-chain pricing-hangs 20 20 yes none off
microservices-call-chain retry-every-hop 20 20 yes none off
mongodb-hot-shard base 20 20 yes none off
mongodb-hot-shard baseline 20 20 yes none off
mongodb-hot-shard cache-front 20 20 yes none off
mongodb-hot-shard eight-shards 20 20 yes none off
mongodb-hot-shard uniform 20 20 yes none off
mysql-replica-lag base 20 20 yes none off
mysql-replica-lag baseline 20 20 yes none off
mysql-replica-lag storage-stall 20 20 yes none off
mysql-replica-lag surge 20 20 yes none off
mysql-replica-lag surge-16-workers 20 20 yes none off
mysql-replica-lag surge-primary 20 20 yes none off
nats-live-telemetry bad-deploy 20 20 yes none off
nats-live-telemetry bad-deploy-jetstream 20 20 yes none off
nats-live-telemetry base 20 20 yes none off
nats-live-telemetry baseline 20 20 yes none off
nats-live-telemetry final 20 20 yes none off
nats-live-telemetry final-scaled 20 20 yes none off
notification-fanout-quotas base 20 20 yes none off
notification-fanout-quotas baseline 20 20 yes none off
notification-fanout-quotas more-email-senders 20 20 yes none off
notification-fanout-quotas sms-down 20 20 yes none off
notification-fanout-quotas sms-down-backoff 20 20 yes none off
notification-fanout-quotas sms-everyone 20 20 yes none off
public-api-slow-dependency base 20 20 yes none off
public-api-slow-dependency baseline 20 20 yes none off
public-api-slow-dependency circuit-breaker 20 20 yes none off
public-api-slow-dependency fail-open 20 20 yes none off
public-api-slow-dependency fraud-slow 20 20 yes none off
public-api-slow-dependency retry-budget 20 20 yes none off
queueing-lab base 20 20 yes none off
queueing-lab load-110 20 20 yes none off
queueing-lab load-110-shed 20 20 yes none off
queueing-lab load-50 20 20 yes none off
queueing-lab load-90 20 20 yes none off
retry-storm-ttl-cache base 20 20 yes none off
retry-storm-ttl-cache burst-no-retries 20 20 yes none off
retry-storm-ttl-cache burst-with-retries 20 20 yes none off
retry-storm-ttl-cache no-burst 20 20 yes none off
retry-storm-ttl-cache short-ttl 20 20 yes none off
right-size-catalog-api base 20 20 yes none off
right-size-catalog-api baseline 20 20 yes none off
right-size-catalog-api big-db 20 20 yes none off
right-size-catalog-api db-memory 20 20 yes none off
right-size-catalog-api five-api 20 20 yes none off
right-size-catalog-api replica 20 20 yes none off
right-size-catalog-api two-api 20 20 yes none off
search-opensearch-catalog base 20 20 yes none off
search-opensearch-catalog baseline 20 20 yes none off
search-opensearch-catalog overshard 20 20 yes none off
search-opensearch-catalog peak 20 20 yes none off
search-opensearch-catalog peak-replica 20 20 yes none off
search-opensearch-catalog slowdown 20 20 yes none off
serverless-burst base 20 20 yes none off
serverless-burst baseline 20 20 yes none off
serverless-burst concurrency 20 20 yes none off
serverless-burst on-demand 20 20 yes none off
serverless-burst provisioned 20 20 yes none off
serverless-burst steady 20 20 yes none off
serverless-image-pipeline base 20 20 yes none off
serverless-image-pipeline baseline 20 20 yes none off
serverless-image-pipeline java-worker 20 20 yes none off
serverless-image-pipeline low-concurrency 20 20 yes none off
serverless-image-pipeline s3-slow 20 20 yes none off
serverless-image-pipeline small-worker 20 20 yes none off
social-feed-hot-key base 20 20 yes none off
social-feed-hot-key baseline 20 20 yes none off
social-feed-hot-key celebrity 20 20 yes none off
social-feed-hot-key celebrity-12-shards 20 20 yes none off
social-feed-hot-key celebrity-fail-fast 20 20 yes none off
social-feed-hot-key celebrity-memcached 20 20 yes none off
sqlserver-saas-blocking base 20 20 yes none off
sqlserver-saas-blocking baseline 20 20 yes none off
sqlserver-saas-blocking failover 20 20 yes none off
sqlserver-saas-blocking month-end 20 20 yes none off
sqlserver-saas-blocking month-end-append 20 20 yes none off
sqlserver-saas-blocking month-end-rcsi 20 20 yes none off

How it was measured#

  • The baselines: scripts/export_simulation_baselines.py simulated every template case in full on the frozen engine, with the workload's seed and the 19 after it (ADR-0052).
  • The Java runs: ./gradlew :archsim-engine:parity -Parchsim.parity.out=docs/validation simulated the same runs (4 at a time) and wrote simulation-parity.json; the javaSeconds it records is not a speed figure.
  • The knees: BENCH_ENGINE=java uv run python benchmarks/run.py resimulate wrote benchmarks/results/dev/0.16.0-java/ before the Java engine had a version of its own (it names the folder by its version now), and the same harness on the frozen engine, uv run python benchmarks/run.py resimulate --label 0.16.0-port, benchmarks/results/dev/0.16.0-port/, each over every kept run with a two-pass calibration, the only kind with a knee to re-simulate (dev/0.16.0/ predates the spread anchor every check now derives, dev/0.16.0-w29/ the held-out runs).
  • The speed: uv run python scripts/perf.py --label python-0.16.0 --repeats 5, then ./gradlew :archsim-engine:speed -Parchsim.speed.out=benchmarks/perf/java-port.json -Parchsim.speed.repeats=5, one after the other with no other heavy work running.