Yii3 application-server benchmarks

This repository measures the same Yii3 API application on several PHP application servers under repeatable, constant-throughput HTTP load. All runtime implementations live together on master, use the same application code, database seed, benchmark client, and report generator, and can be run individually or as one batch.

The suite is intended for comparing runtime behavior—not for declaring a universally fastest server. Results depend on the host, Docker version, CPU scheduling, runtime configuration, request rate, and benchmark duration. Compare runs made on the same machine with the same settings and minimal background activity.

What is included

Runtime key	Application server	Execution model
frankenphp-classic	FrankenPHP	A normal PHP application bootstrap for each request
frankenphp-worker	FrankenPHP	A persistent Yii worker
roadrunner	RoadRunner	A persistent Yii worker managed by RoadRunner
php-fpm	PHP-FPM + Nginx	Traditional FastCGI processes behind Nginx
freeunit	FreeUnit	PHP application hosted by FreeUnit; grouped with non-worker runtimes
rapira	Rapira worker	A persistent Yii worker using yii-runner-rapira
rapira-classic	Rapira classic	A fresh Yii application per request; grouped with non-worker runtimes
rapira-dispatcher	Rapira dispatcher	A persistent Yii application using Rapira exchanges; grouped with worker runtimes

Every runtime is an isolated Docker Compose profile defined in docker/benchmarks.compose.yml. Each run receives its own PostgreSQL and Valkey containers and uses the same source tree mounted at /app. The PostgreSQL database is seeded from docker/postgres/initdb.d/10-benchmark.sql.

The runtime configs use a production-oriented benchmark baseline:

• All runtimes start 20 PHP execution workers. FrankenPHP worker mode reserves one additional thread for non-worker requests. PHP-FPM uses a static pool, avoiding worker ramp-up during measurement.
• Every PHP image loads php.ini-production plus docker/runtimes/php-production.ini: OPcache is enabled for web and CLI SAPIs, JIT is disabled, errors go to stderr, and PHP memory is limited to 256 MiB.
• OPcache timestamp validation is disabled. Restart the runtime after changing PHP files, including files in the mounted source tree. The benchmark suite rebuilds and restarts each runtime automatically.
• FPM, FreeUnit, Rapira and FrankenPHP workers recycle after 10,000 requests; RoadRunner uses its memory supervisor. API body limits are 8 MiB, and request/queue timeouts are configured where supported.
• HTTP readiness checks gate benchmark startup. Containers have a 45-second shutdown grace period, bounded Docker logs, and an increased open-file limit. Nginx access logging is disabled to match the other servers, while errors remain logged. FastCGI keepalive is intentionally disabled so idle Nginx connections cannot reserve the smaller FPM worker pool.
• Each endpoint receives a separate unmeasured warm-up before load and resource samples are recorded.

These are production-like application-server settings for a controlled local benchmark. HTTP on port 9991, bind-mounted application code, disposable database storage and benchmark credentials remain intentional; a deployed service still needs its own TLS ingress, secrets and persistent database storage. Worker counts must be sized for the deployment's CPU and memory budget. Historical results use the configs in effect when they were recorded and must be rerun to compare this baseline.

Server releases checked on 2026-09-22 are pinned in the benchmark Dockerfile and Compose file:

Component	Version
PHP / PHP-FPM	8.5.10
FrankenPHP	1.12.7
RoadRunner	2025.1.15
FreeUnit	1.36.1
Rapira (all modes)	0.8.1
Nginx	1.31.6 (mainline)
PostgreSQL	18.6
Valkey	9.1.2

FreeUnit's published latest-php8.5 image still contains 1.35.5. Its build target therefore compiles the checksummed 1.36.1 release source against PHP 8.5.10, with TLS and compression support. Optional JavaScript routing and OpenTelemetry modules are not built; the benchmark does not use them. Version pins should be refreshed from upstream releases when updating the benchmark baseline.

Two endpoints are benchmarked:

• / measures framework and runtime overhead with a minimal response.
• /postgres/orders measures a database-backed request that reads joined orders and customers rows through yiisoft/db-pgsql and persistent PDO connections.

Load is generated by a pinned build of wrkx, a maintained wrk2 derivative with constant-throughput load and coordinated-omission-aware latency recording. Docker CPU and memory usage are sampled alongside the HTTP results.

Requirements

• Linux with Docker Engine and the Docker Compose v2 plugin.
• GNU Make and Bash.
• Git for contributing.
• Enough available CPU, memory, disk space, and time to build all runtime images.
• Port 9991 available on the host.

PHP, Composer, wrkx, PostgreSQL, and Valkey do not need to be installed on the host for benchmark runs. The first run is slower because Docker must download and build the runtime images; later runs reuse cached layers.

Quick start

Clone the repository and run the complete matrix:

git clone git@github.com:Yii3-Benchmarks/app-api.git
cd app-api
make bench-all

This command sequentially:

1. Builds and starts each runtime with isolated PostgreSQL and Valkey services.
2. Waits for the stack and endpoint preflight check to succeed.
3. Benchmarks / and /postgres/orders with wrkx.
4. Captures application, database, and cache resource usage.
5. Stops the stack and removes its volumes before moving to the next runtime.
6. Generates one self-contained HTML report for the complete suite.

Results are written to a timestamped directory:

runtime/benchmarks/<timestamp>-suite/
├── <timestamp>-<runtime>-<target>-<mode>/
│   ├── metadata.env
│   ├── summary.json
│   ├── wrkx-timeseries.json
│   ├── wrkx-*.log
│   └── docker-stats.csv
└── report.html

Running selected benchmarks

Benchmark one runtime and the minimal endpoint:

make bench RUNTIME=roadrunner MODE=steady RATE=8000 DURATION=60s

Benchmark its PostgreSQL endpoint:

make bench-db RUNTIME=php-fpm MODE=steady RATE=4000 DURATION=60s

Benchmark all Rapira modes on both endpoints:

make bench-all RUNTIMES="rapira rapira-classic rapira-dispatcher"

All Rapira modes use the pinned 0.8.1-php8.5 server image and the same worker-rapira.php entry point. The Yii runner detects the configured mode: classic handles one request per application bootstrap, while worker and dispatcher keep the application in memory. rapira continues to select worker mode. Its Yii runner and PHP contract currently require development packages; Composer records their exact revisions in the local lock file.

Run a subset of runtimes through both endpoints:

make bench-all RUNTIMES="frankenphp-worker roadrunner freeunit"

The underlying suite script also accepts a target subset:

RUNTIMES="roadrunner freeunit" TARGETS="home" MODE=steady RATE=5000 DURATION=60s \
    ./tools/run-benchmark-suite.sh

To start a runtime without benchmarking it:

make runtime-up RUNTIME=frankenphp-worker
curl http://localhost:9991/
curl http://localhost:9991/postgres/orders
make runtime-down RUNTIME=frankenphp-worker

runtime-down removes the selected runtime's database and cache volumes. Do not use it if you need to preserve manual changes made inside those benchmark containers.

Benchmark configuration

The default mode is ramp. Configuration is passed as Make variables or environment variables.

Variable	Default	Meaning
RUNTIME	frankenphp-classic	Runtime used by make bench and make bench-db
RUNTIMES	all eight runtimes	Space-separated runtimes used by make bench-all
TARGETS	home postgres-orders	Space-separated endpoint keys for the suite script
MODE	ramp	steady for one rate or ramp for sequential rate stages
RATE	10000	Requests per second in steady mode
DURATION	160s	Steady-mode duration
THREADS	host CPU count	wrkx worker threads
CONNECTIONS	256	Concurrent HTTP connections
WARMUP_DURATION	10s	Unmeasured warm-up per endpoint; 0s disables it
WARMUP_RATE	1000	Requests per second during warm-up
STAGES	thirteen stages from 2.5k to 200k RPS	JSON stage list for ramp mode
OUTPUT_ROOT	timestamped suite directory	Result destination

Example custom ramp:

STAGES='[{"target":1000,"duration":"30s"},{"target":3000,"duration":"30s"}]' \
    make bench RUNTIME=freeunit MODE=ramp

wrkx does not change rate continuously during a run. Ramp mode executes each STAGES entry as a separate constant-rate run and records one aggregate point per stage. wrkx uses an initial calibration period, so stages shorter than 20 seconds are not recommended.

Reports

The generated HTML report compares issued and successful RPS, errors, average and p95 latency, application CPU, and application memory. Charts are grouped into worker/non-worker and DB/non-DB comparisons. DB and non-DB summary tables show Successful RPS and Target RPS at the cap in separate sortable columns, sorted by Successful RPS descending by default, with unreached caps last. Stage-based runs mark the first stage more than 5% below target as the cap; this can reflect server or load-generator saturation. The default ramp extends to 200k RPS to test beyond the old 50k ceiling. It is self-contained and can be opened directly in a browser or attached to an issue.

Regenerate a report from existing results:

make bench-report INPUT=runtime/benchmarks/<suite-directory>

Combine explicitly selected runs:

make bench-report INPUT="runtime/benchmarks/<run-1> runtime/benchmarks/<run-2>"

Raw wrkx output and exact run settings are retained next to the compact data. Include them when reporting unexpected results; an HTML chart alone is usually insufficient to reproduce a finding.

Repository structure

benchmark/                      wrkx image and Lua result adapter
config/, public/, src/          shared Yii3 API application
docker/benchmarks.compose.yml   isolated benchmark services and runtime profiles
docker/runtimes/                runtime images and server configuration
docker/postgres/initdb.d/       reproducible PostgreSQL benchmark data
tools/run-benchmark-suite.sh    multi-runtime orchestration and cleanup
tools/run-wrkx-benchmark.sh     one endpoint/stage benchmark runner
tools/compile-wrkx-results.php  wrkx output normalization
tools/render-benchmark-report.* HTML report generator
worker-frankenphp.php           FrankenPHP persistent worker entry point
worker-roadrunner.php           RoadRunner persistent worker entry point
worker-rapira.php               Rapira entry point for all three modes

The remaining application-template Docker files support development and tests. The benchmark matrix specifically uses docker/benchmarks.compose.yml and docker/runtimes/.

Testing changes

Install or update project dependencies through the development container when needed:

make composer-update

Run the automated checks relevant to your change:

make test
docker compose -f docker/benchmarks.compose.yml --profile roadrunner config --quiet
bash -n tools/run-benchmark-suite.sh tools/run-wrkx-benchmark.sh

Before submitting benchmark-related changes, run at least one short steady benchmark for the affected runtime. Use a duration of 20 seconds or more so wrkx calibration is meaningful:

make bench RUNTIME=roadrunner MODE=steady RATE=100 DURATION=20s THREADS=2 CONNECTIONS=8

Changes that affect shared application behavior should be checked against every runtime with make bench-all when practical.

Contributing

Contributions are welcome for runtime upgrades, new application servers, benchmark correctness, reporting, and reproducibility improvements.

1. Create a branch from master.
2. Keep shared application behavior identical across runtimes. Runtime-specific code belongs in docker/runtimes/ or a clearly named worker entry point.
3. Add or update tests and documentation with the implementation.
4. Run the checks above and record the exact smoke benchmark command you used.
5. Open a pull request describing the motivation, affected runtimes, validation performed, and any compatibility or performance tradeoffs. Do not present performance changes without the host and benchmark configuration.

Adding a runtime

To add another application server:

1. Add a named build target to docker/runtimes/Dockerfile and its configuration under docker/runtimes/.
2. Add a matching profile and service to docker/benchmarks.compose.yml, exposing the application on host port 9991.
3. Add the runtime key, readable label, and resource-sampled service names to tools/run-benchmark-suite.sh.
4. Add required PHP packages to composer.json; keep one shared lock file.
5. Add the runtime to the RUNTIMES default in Makefile and to the table in this README.
6. Verify / and /postgres/orders, run a short steady benchmark, and confirm report generation and automatic teardown.

Avoid committing generated benchmark output unless it is intentionally used as a published reference result. Never change only one runtime's application logic to improve its score—the suite must compare equivalent work.

License

The project is released under the BSD-3-Clause License. See LICENSE.md.