feat: establish simulation scaling baseline
This commit is contained in:
@@ -0,0 +1,99 @@
|
||||
# Simulation scaling baseline 01
|
||||
|
||||
This is the first reviewed Milestone 8 measurement of the current
|
||||
full-fidelity, data-only simulation. It establishes a reproducible reference
|
||||
before introducing batching, spatial partitions, or simulation LOD.
|
||||
|
||||
The machine-readable samples and checksums are stored beside this note in
|
||||
[`simulation_scaling_baseline_01.json`](simulation_scaling_baseline_01.json).
|
||||
|
||||
## Capture context
|
||||
|
||||
- captured: 2026-07-16;
|
||||
- engine: Godot 4.7 stable;
|
||||
- host: Apple M1 Max with 64 GB memory on macOS, with 10 processors reported
|
||||
by Godot;
|
||||
- benchmark seed: `8088`;
|
||||
- workload: `full_fidelity_headless_arrival`;
|
||||
- samples: three fresh managers per case, using the median;
|
||||
- cadence: 10 warmup ticks, then 200 measured ticks at 1.2 simulated seconds
|
||||
per tick.
|
||||
|
||||
Run the reviewed capture from the project root:
|
||||
|
||||
```bash
|
||||
/Applications/Godot.app/Contents/MacOS/Godot \
|
||||
--headless --path "$PWD" \
|
||||
--script res://tools/benchmark_simulation_scaling.gd -- \
|
||||
--host-label="Apple M1 Max, 64 GB" \
|
||||
--output=res://docs/benchmarks/simulation_scaling_baseline_01.json
|
||||
```
|
||||
|
||||
## Workload contract
|
||||
|
||||
Every NPC remains a named `SimNPC` and receives the normal needs, schedule,
|
||||
task, opportunity, and action-selection work each tick. The pantry starts
|
||||
empty, so ordinary food-supply decisions exercise the current population-wide
|
||||
queries. Travel completes through the same deterministic immediate-arrival
|
||||
convention used by existing headless continuation scenarios. Seeded history is
|
||||
made of valid immutable storage-deposit events plus at most three known-event
|
||||
references per NPC, and every prepared fixture must parse through the normal
|
||||
`SimulationStateRecord` schema.
|
||||
|
||||
The timed section excludes manager construction, fixture preparation,
|
||||
serialization, world scenes, rendering, navigation, and `NpcVisual`. This is a
|
||||
simulation-throughput baseline, not a complete frame-time or memory profile.
|
||||
|
||||
## Results
|
||||
|
||||
| Case | NPCs | Seeded history | Median us/tick | Ticks/s | Simulated realtime | Arrival share | End JSON | Events added |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| Population 6 | 6 | 0 | 55.74 | 17,942.05 | 21,530.5x | 5.2% | 0.06 MiB | 209 |
|
||||
| Population 60 | 60 | 0 | 621.87 | 1,608.07 | 1,929.7x | 6.4% | 0.56 MiB | 2,090 |
|
||||
| Population 600 | 600 | 0 | 15,187.74 | 65.84 | 79.0x | 14.0% | 5.67 MiB | 20,950 |
|
||||
| History 600 | 60 | 600 | 677.12 | 1,476.85 | 1,772.2x | 5.6% | 0.71 MiB | 2,090 |
|
||||
| History 6,000 | 60 | 6,000 | 1,421.19 | 703.64 | 844.4x | 2.7% | 2.04 MiB | 2,090 |
|
||||
|
||||
All three samples in each case produced the same deterministic checksum and
|
||||
state counters. The benchmark runner rejects the report if they diverge.
|
||||
|
||||
## Findings
|
||||
|
||||
Population scaling is the first measured knee. Raising the fixture from 60 to
|
||||
600 NPCs increases median tick cost by about 24.4x for 10x the population.
|
||||
Immediate-arrival completion accounts for only 14.0% of the 600-NPC timed
|
||||
section, so most of that growth remains inside the ordinary simulation tick.
|
||||
|
||||
The benchmark is not a function profiler, but code inspection identifies a
|
||||
bounded first candidate: while food is scarce, each applicable idle decision
|
||||
calls `RelationshipSystem.get_trusted_starving_subject()`, which rebuilds a map
|
||||
by scanning every NPC. Building the same stable-ID population view once per
|
||||
tick should remove repeated work without changing action semantics or tie
|
||||
breaks. The benchmark and checksum give that change a concrete comparison.
|
||||
|
||||
History also has a visible but less urgent cost. At a fixed 60 NPCs, increasing
|
||||
seeded objective events from 600 to 6,000 raises tick cost about 2.1x and final
|
||||
serialized state from 0.71 MiB to 2.04 MiB. The 180 seeded known-event
|
||||
references age out during the measured window, while the objective event log
|
||||
remains complete. Event-log indexing or archiving therefore remains a later,
|
||||
separate decision.
|
||||
|
||||
Organic event growth matters independently of CPU time. The 600-NPC case adds
|
||||
20,950 objective events and grows serialized state by 5,235,086 bytes over only
|
||||
200 measured ticks. Future long-session work should measure event retention and
|
||||
save cost explicitly rather than treating tick throughput as the whole scale
|
||||
problem.
|
||||
|
||||
## Reference target and next slice
|
||||
|
||||
The first local reference target is at least **50 measured ticks per second for
|
||||
600 data-only full-fidelity NPCs** on this Apple M1 Max workload, with identical
|
||||
deterministic state and checksum. Baseline 01 reaches 65.84 ticks per second.
|
||||
This is a local comparison target, not a cross-machine CI timing assertion or a
|
||||
claim about 600 rendered and navigating characters.
|
||||
|
||||
The next slice should build one reusable per-tick population view for the
|
||||
existing trusted-starving-subject consumer, preserve exact selection and
|
||||
continuation results, and rerun this ledger. Spatial partitioning and
|
||||
active/abstract LOD should wait until that bounded change shows what cost
|
||||
remains.
|
||||
Reference in New Issue
Block a user