Rijad Zuzo
100 lines
4.8 KiB
Markdown
100 lines
4.8 KiB
Markdown
# Simulation scaling baseline 01
|
|
|
|
This is the first reviewed Milestone 8 measurement of the current
|
|
full-fidelity, data-only simulation. It establishes a reproducible reference
|
|
before introducing batching, spatial partitions, or simulation LOD.
|
|
|
|
The machine-readable samples and checksums are stored beside this note in
|
|
[`simulation_scaling_baseline_01.json`](simulation_scaling_baseline_01.json).
|
|
|
|
## Capture context
|
|
|
|
- captured: 2026-07-16;
|
|
- engine: Godot 4.7 stable;
|
|
- host: Apple M1 Max with 64 GB memory on macOS, with 10 processors reported
|
|
by Godot;
|
|
- benchmark seed: `8088`;
|
|
- workload: `full_fidelity_headless_arrival`;
|
|
- samples: three fresh managers per case, using the median;
|
|
- cadence: 10 warmup ticks, then 200 measured ticks at 1.2 simulated seconds
|
|
per tick.
|
|
|
|
Run the reviewed capture from the project root:
|
|
|
|
```bash
|
|
/Applications/Godot.app/Contents/MacOS/Godot \
|
|
--headless --path "$PWD" \
|
|
--script res://tools/benchmark_simulation_scaling.gd -- \
|
|
--host-label="Apple M1 Max, 64 GB" \
|
|
--output=res://docs/benchmarks/simulation_scaling_baseline_01.json
|
|
```
|
|
|
|
## Workload contract
|
|
|
|
Every NPC remains a named `SimNPC` and receives the normal needs, schedule,
|
|
task, opportunity, and action-selection work each tick. The pantry starts
|
|
empty, so ordinary food-supply decisions exercise the current population-wide
|
|
queries. Travel completes through the same deterministic immediate-arrival
|
|
convention used by existing headless continuation scenarios. Seeded history is
|
|
made of valid immutable storage-deposit events plus at most three known-event
|
|
references per NPC, and every prepared fixture must parse through the normal
|
|
`SimulationStateRecord` schema.
|
|
|
|
The timed section excludes manager construction, fixture preparation,
|
|
serialization, world scenes, rendering, navigation, and `NpcVisual`. This is a
|
|
simulation-throughput baseline, not a complete frame-time or memory profile.
|
|
|
|
## Results
|
|
|
|
| Case | NPCs | Seeded history | Median us/tick | Ticks/s | Simulated realtime | Arrival share | End JSON | Events added |
|
|
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
|
| Population 6 | 6 | 0 | 55.74 | 17,942.05 | 21,530.5x | 5.2% | 0.06 MiB | 209 |
|
|
| Population 60 | 60 | 0 | 621.87 | 1,608.07 | 1,929.7x | 6.4% | 0.56 MiB | 2,090 |
|
|
| Population 600 | 600 | 0 | 15,187.74 | 65.84 | 79.0x | 14.0% | 5.67 MiB | 20,950 |
|
|
| History 600 | 60 | 600 | 677.12 | 1,476.85 | 1,772.2x | 5.6% | 0.71 MiB | 2,090 |
|
|
| History 6,000 | 60 | 6,000 | 1,421.19 | 703.64 | 844.4x | 2.7% | 2.04 MiB | 2,090 |
|
|
|
|
All three samples in each case produced the same deterministic checksum and
|
|
state counters. The benchmark runner rejects the report if they diverge.
|
|
|
|
## Findings
|
|
|
|
Population scaling is the first measured knee. Raising the fixture from 60 to
|
|
600 NPCs increases median tick cost by about 24.4x for 10x the population.
|
|
Immediate-arrival completion accounts for only 14.0% of the 600-NPC timed
|
|
section, so most of that growth remains inside the ordinary simulation tick.
|
|
|
|
The benchmark is not a function profiler, but code inspection identifies a
|
|
bounded first candidate: while food is scarce, each applicable idle decision
|
|
calls `RelationshipSystem.get_trusted_starving_subject()`, which rebuilds a map
|
|
by scanning every NPC. Building the same stable-ID population view once per
|
|
tick should remove repeated work without changing action semantics or tie
|
|
breaks. The benchmark and checksum give that change a concrete comparison.
|
|
|
|
History also has a visible but less urgent cost. At a fixed 60 NPCs, increasing
|
|
seeded objective events from 600 to 6,000 raises tick cost about 2.1x and final
|
|
serialized state from 0.71 MiB to 2.04 MiB. The 180 seeded known-event
|
|
references age out during the measured window, while the objective event log
|
|
remains complete. Event-log indexing or archiving therefore remains a later,
|
|
separate decision.
|
|
|
|
Organic event growth matters independently of CPU time. The 600-NPC case adds
|
|
20,950 objective events and grows serialized state by 5,235,086 bytes over only
|
|
200 measured ticks. Future long-session work should measure event retention and
|
|
save cost explicitly rather than treating tick throughput as the whole scale
|
|
problem.
|
|
|
|
## Reference target and next slice
|
|
|
|
The first local reference target is at least **50 measured ticks per second for
|
|
600 data-only full-fidelity NPCs** on this Apple M1 Max workload, with identical
|
|
deterministic state and checksum. Baseline 01 reaches 65.84 ticks per second.
|
|
This is a local comparison target, not a cross-machine CI timing assertion or a
|
|
claim about 600 rendered and navigating characters.
|
|
|
|
The next slice should build one reusable per-tick population view for the
|
|
existing trusted-starving-subject consumer, preserve exact selection and
|
|
continuation results, and rerun this ledger. Spatial partitioning and
|
|
active/abstract LOD should wait until that bounded change shows what cost
|
|
remains.
|