perf: benchmark regional simulation scale
This commit is contained in:
Rijad Zuzo
@@ -24,8 +24,24 @@ Loaded-resource discovery has its own bounded runner:
|
||||
--script res://tools/benchmark_loaded_resource_discovery.gd
|
||||
```
|
||||
|
||||
The regional record/scheduler workload has a separate honest runner:
|
||||
|
||||
```bash
|
||||
/Applications/Godot.app/Contents/MacOS/Godot \
|
||||
--headless --path "$PWD" \
|
||||
--script res://tools/benchmark_regional_scale.gd
|
||||
```
|
||||
|
||||
It constructs five settlements, 20 caravans, 2,000 named people, 50,000
|
||||
aggregate residents, and 2,025 deterministic due jobs. It measures record
|
||||
construction, canonical roundtrip, scheduler construction, and job draining;
|
||||
it does not claim rendered or full-economy throughput.
|
||||
|
||||
Reviewed captures:
|
||||
|
||||
- [Regional scale baseline 01](REGIONAL_SCALE_BASELINE_01.md) records the
|
||||
primitive regional state and deterministic scheduler workload at five
|
||||
settlements, 20 caravans, 2,000 named people, and 50,000 aggregate residents.
|
||||
- [Simulation scaling baseline 01](SIMULATION_SCALING_BASELINE_01.md) records
|
||||
the first full-fidelity population/history measurements and the bounded next
|
||||
optimization.
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
# Regional scale baseline 01
|
||||
|
||||
This is the first reviewed capture of workload schema 1,
|
||||
`regional_records_scheduler_v1`. It is a structural headless baseline for the
|
||||
regional record and deterministic due-job contracts—not a claim about rendered
|
||||
gameplay or the complete economy.
|
||||
|
||||
The machine-readable report is in
|
||||
[`regional_scale_baseline_01.json`](regional_scale_baseline_01.json). It was
|
||||
captured with Godot 4.7 on the `Apple_M1_Max_64_GB` development host, seed
|
||||
424242, three default-budget samples, and parity runs at budgets one and
|
||||
unlimited.
|
||||
|
||||
## Fixed workload
|
||||
|
||||
- five locations, settlements, polities, and ring routes;
|
||||
- 20 caravans with 12 named members each;
|
||||
- 2,000 persistent named people;
|
||||
- five aggregate cohorts totaling 50,000 residents;
|
||||
- 2,025 keyed deterministic due jobs;
|
||||
- maximum active-presentation metadata of 40;
|
||||
- 736,115 bytes of canonical regional state and 540,942 bytes of initial
|
||||
scheduler state.
|
||||
|
||||
All records round-tripped through their primitive-only schemas. The fixture
|
||||
contained 52,000 people and 2,734 cargo units without duplicate global IDs or
|
||||
membership errors. Budget one, budget eight, and unlimited execution drained
|
||||
the backlog to zero in exactly the same order and produced final checksum
|
||||
`57c8a8eb9dd48bbea309cdb5654ce508f6e4965b07228217aa55fc43de699a25`.
|
||||
|
||||
## Results
|
||||
|
||||
| Phase | Median | Samples |
|
||||
| --- | ---: | --- |
|
||||
| Fixture construction | 273.79 ms | 272.37, 273.79, 282.07 ms |
|
||||
| Canonical roundtrip | 255.86 ms | 255.33, 255.86, 266.03 ms |
|
||||
| Scheduler construction | 74.10 ms | 73.66, 74.10, 76.50 ms |
|
||||
| Drain 2,025 jobs (budget 8) | 57.47 ms | 55.52, 57.47, 59.51 ms |
|
||||
| Complete measured run | 785.65 ms | 776.58, 785.65, 808.22 ms |
|
||||
|
||||
Budget eight drained the queue in 254 batches. Budget one used 2,025 batches;
|
||||
unlimited used one. Their execution-order, fixture, and final checksums were
|
||||
identical.
|
||||
|
||||
## Interpretation
|
||||
|
||||
The baseline proves that the target record counts fit in a small canonical
|
||||
payload and that job-budget changes do not alter outcomes. It intentionally
|
||||
constructs only 2,025 event-boundary jobs, so the 57.47 ms drain is not a
|
||||
30-day simulation-time throughput result. It also excludes scene loading,
|
||||
navigation, rendering, dialogue/situation queries, market transactions,
|
||||
conflict, event rollups, chunked disk I/O, and live caravan transfers.
|
||||
|
||||
The next benchmark must run scheduled settlement and caravan domain handlers
|
||||
over 30 simulated days. Separately, a recorded weak-PC active-world capture
|
||||
must measure the Low presentation tier with at most 40 detailed visuals and two
|
||||
new navigation requests per frame. These measurements cannot be inferred from
|
||||
this development-host structural result.
|
||||
|
||||
## Capture command
|
||||
|
||||
```bash
|
||||
/Applications/Godot.app/Contents/MacOS/Godot \
|
||||
--headless --path "$PWD" \
|
||||
--script res://tools/benchmark_regional_scale.gd -- \
|
||||
--host-label=Apple_M1_Max_64_GB \
|
||||
--output=res://docs/benchmarks/regional_scale_baseline_01.json
|
||||
```
|
||||
@@ -0,0 +1,56 @@
|
||||
{
|
||||
"benchmark_scope": "regional_data_and_scheduler_only",
|
||||
"budget_parity": {
|
||||
"budget_one_drain_batches": 2025,
|
||||
"budget_one_final_checksum": "57c8a8eb9dd48bbea309cdb5654ce508f6e4965b07228217aa55fc43de699a25",
|
||||
"identical": true,
|
||||
"unlimited_drain_batches": 1,
|
||||
"unlimited_final_checksum": "57c8a8eb9dd48bbea309cdb5654ce508f6e4965b07228217aa55fc43de699a25"
|
||||
},
|
||||
"captured_utc": "2026-08-12T19:36:00Z",
|
||||
"default_budget": {
|
||||
"backlog_final": 0,
|
||||
"backlog_initial": 2025,
|
||||
"backlog_peak": 2025,
|
||||
"drain_batches": 254,
|
||||
"elapsed_usec_median": 785655,
|
||||
"elapsed_usec_samples": [776580, 785655, 808216],
|
||||
"execution_usec_median": 57471,
|
||||
"execution_usec_samples": [55516, 57471, 59508],
|
||||
"fixture_build_usec_median": 273789,
|
||||
"fixture_build_usec_samples": [272365, 273789, 282068],
|
||||
"roundtrip_usec_median": 255860,
|
||||
"roundtrip_usec_samples": [255326, 255860, 266034],
|
||||
"scheduler_build_usec_median": 74100,
|
||||
"scheduler_build_usec_samples": [73657, 74100, 76495]
|
||||
},
|
||||
"default_execution_budget": 8,
|
||||
"engine_version": "4.7-stable (official)",
|
||||
"fixture": {
|
||||
"caravan_count": 20,
|
||||
"cargo_total": 2734.0,
|
||||
"cohort_resident_count": 50000,
|
||||
"detailed_caravan_member_count": 240,
|
||||
"detailed_members_per_caravan": 12,
|
||||
"execution_order_checksum": "38815c49987cf66ec0d78b82d332789f708742959bab83c9feee631d8bbf8174",
|
||||
"final_checksum": "57c8a8eb9dd48bbea309cdb5654ce508f6e4965b07228217aa55fc43de699a25",
|
||||
"fixture_checksum": "5ee2dcda7067b34e1ffc22861fbcf1e6d1897f87523a88fbf216fc9ea6a19ed5",
|
||||
"initial_job_count": 2025,
|
||||
"location_count": 5,
|
||||
"max_active_visuals": 40,
|
||||
"named_person_count": 2000,
|
||||
"regional_state_bytes": 736115,
|
||||
"scheduler_state_bytes": 540942,
|
||||
"seed": 424242,
|
||||
"settlement_count": 5,
|
||||
"total_headcount": 52000
|
||||
},
|
||||
"full_economy_throughput_measured": false,
|
||||
"host_label": "Apple_M1_Max_64_GB",
|
||||
"platform": "macOS",
|
||||
"processor_count": 10,
|
||||
"rendered_throughput_measured": false,
|
||||
"sample_count": 3,
|
||||
"schema_version": 1,
|
||||
"workload_id": "regional_records_scheduler_v1"
|
||||
}
|
||||
Reference in New Issue
Block a user