admin/gamedev-the-steward
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

feat: establish simulation scaling baseline

Browse Source
This commit is contained in:
Rijad Zuzo
2026-07-16 13:07:01 +02:00
parent 4f88d15e12
commit 4be72639d8
14 changed files with 1004 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/benchmarks/README.md
+23
View File
@@ -0,0 +1,23 @@
# Simulation benchmark ledger
This directory stores reviewed benchmark snapshots. Each snapshot must name the
workload, seed, Godot version, hardware context, sample count, simulated tick
count, and important exclusions so later comparisons remain honest.
Run the current scaling harness from the project root with Godot 4.7:
```bash
/Applications/Godot.app/Contents/MacOS/Godot \
--headless --path "$PWD" \
--script res://tools/benchmark_simulation_scaling.gd
```
The default report is written under `user://`. Pass
`-- --host-label="<hardware>" --output=res://docs/benchmarks/<name>.json` only
when intentionally capturing a reviewed project baseline.
Reviewed captures:
- [Simulation scaling baseline 01](SIMULATION_SCALING_BASELINE_01.md) records
the first full-fidelity population/history measurements and the bounded next
optimization.
docs/benchmarks/SIMULATION_SCALING_BASELINE_01.md
+99
View File
@@ -0,0 +1,99 @@
# Simulation scaling baseline 01
This is the first reviewed Milestone 8 measurement of the current
full-fidelity, data-only simulation. It establishes a reproducible reference
before introducing batching, spatial partitions, or simulation LOD.
The machine-readable samples and checksums are stored beside this note in
[`simulation_scaling_baseline_01.json`](simulation_scaling_baseline_01.json).
## Capture context
- captured: 2026-07-16;
- engine: Godot 4.7 stable;
- host: Apple M1 Max with 64 GB memory on macOS, with 10 processors reported
by Godot;
- benchmark seed: `8088`;
- workload: `full_fidelity_headless_arrival`;
- samples: three fresh managers per case, using the median;
- cadence: 10 warmup ticks, then 200 measured ticks at 1.2 simulated seconds
per tick.
Run the reviewed capture from the project root:
```bash
/Applications/Godot.app/Contents/MacOS/Godot \
--headless --path "$PWD" \
--script res://tools/benchmark_simulation_scaling.gd -- \
--host-label="Apple M1 Max, 64 GB" \
--output=res://docs/benchmarks/simulation_scaling_baseline_01.json
```
## Workload contract
Every NPC remains a named `SimNPC` and receives the normal needs, schedule,
task, opportunity, and action-selection work each tick. The pantry starts
empty, so ordinary food-supply decisions exercise the current population-wide
queries. Travel completes through the same deterministic immediate-arrival
convention used by existing headless continuation scenarios. Seeded history is
made of valid immutable storage-deposit events plus at most three known-event
references per NPC, and every prepared fixture must parse through the normal
`SimulationStateRecord` schema.
The timed section excludes manager construction, fixture preparation,
serialization, world scenes, rendering, navigation, and `NpcVisual`. This is a
simulation-throughput baseline, not a complete frame-time or memory profile.
## Results
| Case | NPCs | Seeded history | Median us/tick | Ticks/s | Simulated realtime | Arrival share | End JSON | Events added |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Population 6 | 6 | 0 | 55.74 | 17,942.05 | 21,530.5x | 5.2% | 0.06 MiB | 209 |
| Population 60 | 60 | 0 | 621.87 | 1,608.07 | 1,929.7x | 6.4% | 0.56 MiB | 2,090 |
| Population 600 | 600 | 0 | 15,187.74 | 65.84 | 79.0x | 14.0% | 5.67 MiB | 20,950 |
| History 600 | 60 | 600 | 677.12 | 1,476.85 | 1,772.2x | 5.6% | 0.71 MiB | 2,090 |
| History 6,000 | 60 | 6,000 | 1,421.19 | 703.64 | 844.4x | 2.7% | 2.04 MiB | 2,090 |
All three samples in each case produced the same deterministic checksum and
state counters. The benchmark runner rejects the report if they diverge.
## Findings
Population scaling is the first measured knee. Raising the fixture from 60 to
600 NPCs increases median tick cost by about 24.4x for 10x the population.
Immediate-arrival completion accounts for only 14.0% of the 600-NPC timed
section, so most of that growth remains inside the ordinary simulation tick.
The benchmark is not a function profiler, but code inspection identifies a
bounded first candidate: while food is scarce, each applicable idle decision
calls `RelationshipSystem.get_trusted_starving_subject()`, which rebuilds a map
by scanning every NPC. Building the same stable-ID population view once per
tick should remove repeated work without changing action semantics or tie
breaks. The benchmark and checksum give that change a concrete comparison.
History also has a visible but less urgent cost. At a fixed 60 NPCs, increasing
seeded objective events from 600 to 6,000 raises tick cost about 2.1x and final
serialized state from 0.71 MiB to 2.04 MiB. The 180 seeded known-event
references age out during the measured window, while the objective event log
remains complete. Event-log indexing or archiving therefore remains a later,
separate decision.
Organic event growth matters independently of CPU time. The 600-NPC case adds
20,950 objective events and grows serialized state by 5,235,086 bytes over only
200 measured ticks. Future long-session work should measure event retention and
save cost explicitly rather than treating tick throughput as the whole scale
problem.
## Reference target and next slice
The first local reference target is at least **50 measured ticks per second for
600 data-only full-fidelity NPCs** on this Apple M1 Max workload, with identical
deterministic state and checksum. Baseline 01 reaches 65.84 ticks per second.
This is a local comparison target, not a cross-machine CI timing assertion or a
claim about 600 rendered and navigating characters.
The next slice should build one reusable per-tick population view for the
existing trusted-starving-subject consumer, preserve exact selection and
continuation results, and rerun this ledger. Spatial partitioning and
active/abstract LOD should wait until that bounded change shows what cost
remains.
docs/benchmarks/simulation_scaling_baseline_01.json
+308
View File
@@ -0,0 +1,308 @@
{
"benchmark_seed": 8088,
"captured_utc": "2026-07-16T11:05:19Z",
"cases": [
{
"arrival_share_percent": 5.23010675518077,
"arrival_usec_median": 583,
"arrival_usec_samples": [
578,
583,
587
],
"arrivals_processed": 209,
"case_id": "population_006",
"elapsed_usec_max": 11309,
"elapsed_usec_median": 11147,
"elapsed_usec_min": 10991,
"elapsed_usec_samples": [
10991,
11147,
11309
],
"end_event_count": 221,
"end_known_reference_count": 0,
"end_state_bytes": 58576,
"end_tick": 210,
"events_recorded": 209,
"final_checksum": "100dd5dc1e0c22ea23f35dd876d6f56fd5c33c58b1586b6fa9aec47ee1536511",
"history_seed_events": 0,
"measured_ticks": 200,
"npc_updates": 1200,
"population": 6,
"realtime_factor_median": 21530.4566251009,
"sample_count": 3,
"schema_version": 1,
"seed": 8088,
"setup_usec_median": 772,
"setup_usec_samples": [
535,
772,
3251
],
"simulation_usec_median": 10544,
"simulation_usec_samples": [
10387,
10544,
10700
],
"start_event_count": 12,
"start_known_reference_count": 0,
"start_state_bytes": 7632,
"start_tick": 10,
"state_growth_bytes": 50944,
"tick_interval": 1.2,
"ticks_per_second_median": 17942.0471875841,
"usec_per_tick_median": 55.735,
"warmup_arrivals": 12,
"warmup_ticks": 10,
"workload_id": "full_fidelity_headless_arrival"
},
{
"arrival_share_percent": 6.39769081713877,
"arrival_usec_median": 7957,
"arrival_usec_samples": [
7747,
7957,
8499
],
"arrivals_processed": 2090,
"case_id": "population_060",
"elapsed_usec_max": 142845,
"elapsed_usec_median": 124373,
"elapsed_usec_min": 123769,
"elapsed_usec_samples": [
123769,
124373,
142845
],
"end_event_count": 2210,
"end_known_reference_count": 0,
"end_state_bytes": 587324,
"end_tick": 210,
"events_recorded": 2090,
"final_checksum": "58c7f96fc5df7f5399fd82d1a9c212b3c53067e807059e88ffa484ba0eb41e16",
"history_seed_events": 0,
"measured_ticks": 200,
"npc_updates": 12000,
"population": 60,
"realtime_factor_median": 1929.67927122446,
"sample_count": 3,
"schema_version": 1,
"seed": 8088,
"setup_usec_median": 3734,
"setup_usec_samples": [
3603,
3734,
5045
],
"simulation_usec_median": 116595,
"simulation_usec_samples": [
115787,
116595,
134317
],
"start_event_count": 120,
"start_known_reference_count": 0,
"start_state_bytes": 71278,
"start_tick": 10,
"state_growth_bytes": 516046,
"tick_interval": 1.2,
"ticks_per_second_median": 1608.06605935372,
"usec_per_tick_median": 621.865,
"warmup_arrivals": 120,
"warmup_ticks": 10,
"workload_id": "full_fidelity_headless_arrival"
},
{
"arrival_share_percent": 14.0226590657991,
"arrival_usec_median": 425945,
"arrival_usec_samples": [
425585,
425945,
427652
],
"arrivals_processed": 20950,
"case_id": "population_600",
"elapsed_usec_max": 3040453,
"elapsed_usec_median": 3037548,
"elapsed_usec_min": 3028975,
"elapsed_usec_samples": [
3028975,
3037548,
3040453
],
"end_event_count": 22150,
"end_known_reference_count": 0,
"end_state_bytes": 5948447,
"end_tick": 210,
"events_recorded": 20950,
"final_checksum": "fa4efac63fde63f92eb28141bebd1922f665e85ffe7b7ac2b9702dae6e4c24b7",
"history_seed_events": 0,
"measured_ticks": 200,
"npc_updates": 120000,
"population": 600,
"realtime_factor_median": 79.0110971085889,
"sample_count": 3,
"schema_version": 1,
"seed": 8088,
"setup_usec_median": 35480,
"setup_usec_samples": [
35329,
35480,
35664
],
"simulation_usec_median": 2609839,
"simulation_usec_samples": [
2603329,
2609839,
2614455
],
"start_event_count": 1200,
"start_known_reference_count": 0,
"start_state_bytes": 713361,
"start_tick": 10,
"state_growth_bytes": 5235086,
"tick_interval": 1.2,
"ticks_per_second_median": 65.8425809238241,
"usec_per_tick_median": 15187.74,
"warmup_arrivals": 1200,
"warmup_ticks": 10,
"workload_id": "full_fidelity_headless_arrival"
},
{
"arrival_share_percent": 5.63567488535921,
"arrival_usec_median": 7632,
"arrival_usec_samples": [
7621,
7632,
7830
],
"arrivals_processed": 2090,
"case_id": "history_000600",
"elapsed_usec_max": 136012,
"elapsed_usec_median": 135423,
"elapsed_usec_min": 134887,
"elapsed_usec_samples": [
134887,
135423,
136012
],
"end_event_count": 2810,
"end_known_reference_count": 0,
"end_state_bytes": 743004,
"end_tick": 810,
"events_recorded": 2090,
"final_checksum": "7ae86044f7683359d6bfbc8753e2656f3af90f23d040039154c6869dca48cf65",
"history_seed_events": 600,
"measured_ticks": 200,
"npc_updates": 12000,
"population": 60,
"realtime_factor_median": 1772.22480671673,
"sample_count": 3,
"schema_version": 1,
"seed": 8088,
"setup_usec_median": 22795,
"setup_usec_samples": [
22782,
22795,
22881
],
"simulation_usec_median": 127766,
"simulation_usec_samples": [
127030,
127766,
128361
],
"start_event_count": 720,
"start_known_reference_count": 180,
"start_state_bytes": 253098,
"start_tick": 610,
"state_growth_bytes": 489906,
"tick_interval": 1.2,
"ticks_per_second_median": 1476.85400559728,
"usec_per_tick_median": 677.115,
"warmup_arrivals": 120,
"warmup_ticks": 10,
"workload_id": "full_fidelity_headless_arrival"
},
{
"arrival_share_percent": 2.72202422626189,
"arrival_usec_median": 7737,
"arrival_usec_samples": [
7650,
7737,
7809
],
"arrivals_processed": 2090,
"case_id": "history_006000",
"elapsed_usec_max": 287265,
"elapsed_usec_median": 284237,
"elapsed_usec_min": 280605,
"elapsed_usec_samples": [
280605,
284237,
287265
],
"end_event_count": 8210,
"end_known_reference_count": 0,
"end_state_bytes": 2141615,
"end_tick": 6210,
"events_recorded": 2090,
"final_checksum": "b4df2bac21ed5ce6dcd92736e210db5f3114d959dcb15ae7796493362911b6d7",
"history_seed_events": 6000,
"measured_ticks": 200,
"npc_updates": 12000,
"population": 60,
"realtime_factor_median": 844.365793334436,
"sample_count": 3,
"schema_version": 1,
"seed": 8088,
"setup_usec_median": 154433,
"setup_usec_samples": [
153698,
154433,
155153
],
"simulation_usec_median": 276401,
"simulation_usec_samples": [
272849,
276401,
279590
],
"start_event_count": 6120,
"start_known_reference_count": 180,
"start_state_bytes": 1649700,
"start_tick": 6010,
"state_growth_bytes": 491915,
"tick_interval": 1.2,
"ticks_per_second_median": 703.63816111203,
"usec_per_tick_median": 1421.185,
"warmup_arrivals": 120,
"warmup_ticks": 10,
"workload_id": "full_fidelity_headless_arrival"
}
],
"engine_version": "4.7-stable (official)",
"exclusions": [
"manager_and_fixture_setup",
"state_serialization",
"world_scene",
"rendering",
"navigation",
"npc_visual"
],
"host_label": "Apple M1 Max, 64 GB",
"measured_ticks": 200,
"platform": "macOS",
"processor_count": 10,
"sample_count": 3,
"schema_version": 1,
"timed_phases": [
"simulation_tick",
"headless_arrival_completion"
],
"warmup_ticks": 10,
"workload": "All NPCs receive full per-tick needs/task updates and ordinary action decisions; travel resolves through the deterministic immediate-arrival headless convention.",
"workload_id": "full_fidelity_headless_arrival"
}