Skip to content
Browse the docs

Developing ArchSim: layout and tests

View on GitHubReport a problem with this page

How the repository is organised and how ArchSim is tested. To set up a development environment and run every check before a pull request, follow CONTRIBUTING.md; why the code is built the way it is, decision by decision, is in the architecture decision records.

Repository layout#

archsim-model/      Java design model: the document's records, upgrades, JSON Schema and validation
archsim-engine/     Java engine: the simulator and its deterministic parts (sizing, cost, review, ...)
archsim-app/        Java API (Spring Boot): projects, runs, workspaces, reviews, sharing, sign-in;
                    started as api, worker or migrate
archsim-api-tests/  Java black-box tests of the HTTP API
archsim-cli/        Java command line, `archsim`
archsim-mcp/        Java MCP server for AI coding agents
archsim-benchmarks/ Java accuracy benchmarks' harness; its data stays in benchmarks/
archsim-tools/      Java tooling: the commit message check, CI's changed parts, the price refresh
build-logic/        The Gradle build's shared conventions (with gradle/ and config/checkstyle/)
apps/web/           Next.js editor: canvas, inspector, results, dashboard and comparison
examples/           Example architectures (versioned JSON) with scenarios: the templates
parity/             What the Python engine 0.16.0 recorded: the Java tests' reference data
benchmarks/         The accuracy benchmarks' stacks, k6 load and results
docs/               Documentation, the ADRs and the accuracy report

Each Java module has a README of its own: archsim-model, archsim-engine, archsim-app, archsim-api-tests, archsim-cli, archsim-mcp, archsim-benchmarks and archsim-tools; so do the web app and the parity data. How to run the benchmarks is in validation.md.

Tests and checks#

./gradlew build                                 # the Java code: format, Checkstyle, Error Prone, tests
java archsim-tools/src/main/java/io/archsim/tools/commits/CheckCommits.java --range main..HEAD
cd apps/web && npm test && npm run lint && npm run typecheck && npm run build

make targets wrap several of these (see the Makefile), and CONTRIBUTING.md lists the rest: the web app's end-to-end tests and the black-box API tests.

What the tests check#

  • Parity with the frozen engine. The Java engine's tests replay the runs the Python engine 0.16.0 recorded, figure for figure, and match its golden files (parity/README.md).
  • Queueing theory. The simulator is checked against M/M/1 and M/M/c (Erlang C) queues and Little's law, the instant estimate against the simulation, and the starter example's scenarios against a directional reference benchmark.
  • The HTTP API, from outside. The black-box suite (archsim-api-tests) speaks HTTP to the Java API and its worker on PostgreSQL.
  • The main path in a browser. Playwright drives the web app against a real back end (npm run e2e in apps/web).
  • Accuracy against real systems. The benchmark harness calibrates ArchSim on real stacks, saves its predictions and measures them with k6; the results are in validation.md.

Changing what a simulation produces for the same document and seed is held to a higher bar: see Changing simulation semantics.