Skip to content
Browse the docs

Accepted5 Nov 2026

0036. Run the engine's synchronous calls in a small internal Python service

View on GitHubReport a problem with this page

Context and Problem Statement#

During stage 2 the Java API serves every endpoint, but the engine stays Python until stage 3 (proposal). Runs already cross the language boundary through the PostgreSQL queue. Some answers, though, are computed while the request waits: an analysis's sizes, cost and review (POST /analyze), a run's prepared input and quick check, comparisons, seed summaries, exports, imports and calibrations. Validating a design is mostly not among them: archsim-model already validates with the engine's codes and words. Validating one of its scenarios is, because resolving a scenario sizes the "auto" parts first, which Java does from stage 3. How does the Java API call the Python engine synchronously, and where does that code live, given that the engine package is frozen and the FastAPI code is deleted at the end of stage 2?

Decision Drivers#

  • Answers identical to today's, numbers included, without re-encoding them in Java.
  • Nothing under packages/ changes (the engine is frozen at 0.16.0).
  • It disappears piece by piece: each endpoint goes when stage 3 ports its call.
  • No new kind of infrastructure; compose already runs Python containers.

Considered Options#

  • A new internal HTTP service, services/engine-service (Starlette and Uvicorn, already in the lock file), one endpoint per engine call, stateless
  • Keep the FastAPI API running behind the Java API for those endpoints
  • A Python subprocess per call
  • Engine calls through the PostgreSQL queue, like runs

Decision Outcome#

Chosen option: "a new internal HTTP service", because it is the smallest thing that keeps the answers exactly as they are and can shrink one endpoint at a time.

  • services/engine-service is a uv workspace member, archsim-engine-service, depending on archsim-engine, Starlette, Uvicorn and pydantic only: no database, no settings beyond the engine's own (ARCHSIM_PRICES_DIR), no authentication. It listens inside compose only.
  • One POST endpoint per engine call, JSON in and out. Stage 2 started with eight: {"document", "scenarioId"} for /analyze and for /validate, which the Java API called only with a scenario; /prepare for a run about to be queued (prepare_run within the run's limits and seed, with quick_check); /compare for two reports (compare_reports); /summarize-seeds for a seed group (summarize_seeds); /export and /import for designs as infrastructure files (export_document, import_design); /calibrate for POST /calibrate. A refusal answers 422 {"message", "issues", "kind"}: invalid with the engine's words and issues for a document it cannot read, limit for a run over its limits (until /prepare went), unreadable for text, a mapping or telemetry that cannot be used, in the words the Python API answered; a request it cannot read, 400 {"message"}. Answers are encoded as FastAPI encoded them, so the Java API passes the bytes on. Each call runs in a worker thread, as FastAPI ran sync routes, so a long analysis holds up no other request; the service has no body limit of its own, as only the Java API, which limits bodies, calls it.
  • /calibrate is the one call that reaches out: it queries the Prometheus server a request names. The service holds no settings, so the Java API sends the policy with each request, worked out from ARCHSIM_OUTBOUND, ARCHSIM_PROMETHEUS_ALLOWED_HOSTS, the Prometheus timeout and size limits and the workspace (a sandbox never calls out), with the replay's limits. A policy the service cannot read is refused (400), never guessed. The engine's own client then checks the host and, under an allowlist, that its name resolves to public addresses, as it did inside the Python API.
  • The Java API reaches it at ARCHSIM_ENGINE_URL (default http://127.0.0.1:8001) through one client in its engine package, over HTTP/1.1, and waits ARCHSIM_ENGINE_TIMEOUT seconds (default 120) for an answer. If the service is down, or does not answer in time, the endpoint answers 503 with the error body saying which; GET /health stays about the database, as today. The Python API had no such timeout.
  • The black-box suite starts the service next to the Java API.

Status (5 Nov 2026): stage 3 has removed three endpoints: /validate (dd01bd3), then /analyze and /prepare (126125a). Five remain: /compare, /summarize-seeds, /export, /import and /calibrate.

Status (7 Nov 2026): stage 3 has removed the other five: /summarize-seeds (25ed293), /compare (8540384), /export (b292067), /import (17308de) and /calibrate (1395bc0). Until stage 4's exit, the service keeps one call, the interim /simulate a calibration replays (ADR-0045); it no longer calls out.

Status (10 Nov 2026): stage 4 moved the service's last call, the simulation a calibration replays, into the API (ADR-0053, ADR-0054). Nothing called the service any more, and it was deleted with the Python worker.

Consequences#

  • Good, because each endpoint answers byte for byte what the Python API answered.
  • Good, because the engine and the FastAPI code stay untouched until each is deleted.
  • Good, because the service is stateless: it can be restarted or scaled on its own.
  • Bad, because the service checks no caller: whatever can reach it chooses the outbound policy of the calibrations it asks for. It must listen where only the Java API reaches it (inside compose, never on a published port).
  • Bad, because there is one more container and one more network hop per analysis (a few milliseconds on the compose network, small next to the engine's own time).
  • Bad, because a request validated in Java is read again in Python; both use the same document rules, so they agree.

Confirmation#

services/engine-service/tests compare each endpoint with the engine call it wraps, and its bytes with FastAPI's encoding. The Java client's tests use a stub server; CI runs the black-box suite against the Java API with the real service.

Pros and Cons of the Options#

Keep the FastAPI API behind the Java API#

  • Good, because there is nothing to write.
  • Bad, because the stage's exit deletes the FastAPI code, and it would keep a second copy of authentication, settings and the database in the request path.

A Python subprocess per call#

  • Good, because there is no extra service.
  • Bad, because importing the engine takes about a second, on every analysis.

Through the PostgreSQL queue#

  • Good, because the seam exists already.
  • Bad, because a waiting request would poll the database; the queue is built for runs that take minutes, not answers in milliseconds.

More Information#

  • ADR-0014: the Java application's roles.
  • Stage 3 of the port removes each endpoint as its call moves to archsim-engine; when the last one goes, the service is deleted.
  • ADR-0045 amends this decision: the service keeps one call, the interim /simulate a calibration replays, until stage 4.