gamedev-the-steward/docs/FEATURE_TESTING_AND_BENCHMARKS.md

Feature slice: tests, quality, and performance evidence

The project uses deterministic headless scenarios, GUT unit tests, formatting/ lint checks, and reviewed benchmark ledgers. Tests are part of the architecture: they prove authority boundaries, conservation, migration, parity, and deterministic continuation.

Quality gate

Run from the repository root:

./tools/quality.sh --changed   # fast iteration
./tools/quality.sh             # full gate before commit
git diff --check

The gate currently runs Godot 4.7 platform/import checks, gdformat, gdlint, headless project checks, standalone scenarios, GUT, and a dedicated Compatibility presentation smoke. It uses an isolated Godot profile under logs/quality/godot_profile. On macOS, Godot's certificate-store diagnostic, Terrain3D interpolation deprecation, and the exact known dummy-shader exit diagnostic are classified by the gate; new error classes must not be hidden.

Scenario families

Concern	Representative tests
Definitions and packs	content_catalog_test, simulation_definitions_test, test_activity_content_catalog_contract
Targets and interaction parity	world_target_registry_test, context_scoped_world_targets_test, resource_node_player_parity_test, test_activity_action_command_service
Economy and resources	food_storage_loop_test, resource_regrowth_test, animal_food_delivery_test
Animals and enemies	animal_definition_contract_test, animal_pair_vertical_slice_test, test_combatant_factory, conflict_war_system_test
Events and social facts	test_world_event_store, test_social_foundations, witnessed_knowledge_consequence_test, relationship_consequence_test
Situations and dialogue	test_situation_system, emergent_jajce_slice_test, conversation_domain_test, dialogue_manager_conversation_presenter_test, dialogue_mode_controller_test
Save/migration	test_emergent_state_validation, simulation_state_serialization_test, save_slot_persistence_test, test_commitment_lifecycle_service
Regional records	test_regional_world_state, test_regional_scheduler, test_regional_caravan_service, test_regional_chunked_persistence, test_regional_chunked_file_store
Presentation	jajce_world_scaffold_test, jajce_presentation_quality_test, jajce_runtime_integration_test, test_presentation_relevance_policy, creature_visual_path_retry_test

When a change affects state, target selection, saves, navigation reachability, player/NPC parity, deterministic continuation, or a fact-representing UI, extend a scenario rather than relying on a parser/lint pass.

Benchmark runners

# local simulation population/history fixture
godot --headless --path "$PWD" \
  --script res://tools/benchmark_simulation_scaling.gd

# loaded finite-resource spatial discovery
godot --headless --path "$PWD" \
  --script res://tools/benchmark_loaded_resource_discovery.gd

# regional records and scheduler (20 caravans / 50k cohorts)
godot --headless --path "$PWD" \
  --script res://tools/benchmark_regional_scale.gd

Reviewed reports live in docs/benchmarks/ beside machine-readable JSON. A benchmark must record seed, Godot version, host label, workload/schema ID, sample count, warmup, simulated ticks, checksums, and exclusions. Structural headless measurements cannot be presented as rendered frame-time claims.

Determinism and conservation checklist

For a new system, add tests for:

• same seed/run produces the same canonical checksum;
• save/load at the important boundary continues identically;
• execution budgets {1, 8, unlimited} preserve order and result where a scheduler is involved;
• loaded, unloaded, and load/unload presentation produce identical authority;
• people, cargo, inventory, storage, unique items, and event IDs conserve;
• stale offers, duplicate IDs, unknown definitions, forged facts, and scope mismatches fail without partial mutation;
• important identities remain pinned through relevance/aggregation decisions.

Current evidence and gaps

The current full suite passes 154/154 tests with 2,047 assertions. Baseline 03 measures 600 full-fidelity data-only NPC/combatant records at about 90.6 ticks/s on the reviewed development host. The regional baseline proves the target record counts and scheduler parity. Neither measures a complete regional economy or weak-PC rendering.

The next performance evidence should capture High and Low at a fixed gameplay camera on a named weak-PC reference: p50/p95/p99 CPU/GPU frame time, draw calls, primitives, memory/VRAM, active detailed visuals, and navigation queue depth. Keep those measurements in a reviewed ledger instead of adding brittle hardware limits to CI.

Test authoring conventions

• Use stable IDs and fixed seeds.
• Prefer exact assertions over prose or frame timing.
• Make test fixtures isolate duplicate stable-ID registries and world scopes.
• Use fake presenters for semantic dialogue; reserve plugin/parser tests for the adapter boundary.
• Keep benchmark fixtures independent of production managers unless the test is explicitly an integration proof.
• Run focused tests while editing, then the full gate before committing.