Draft: Make navmesh building fast.

Related to #9103

Cold transitions waited on unrelated navmesh jobs and slow Recast rasterization and ledge filtering. While requiredTilesPresent is active, queues favor capacity-freeing removals, jobs, and database reads for every active AgentBounds and cache generation required by the loading barrier. Changing worldspace clears local Recast state. Including the generation in job identity keeps stale work from matching new tiles.

Tile construction still runs on the configured idle-priority background worker. OpenMW pins Recast fork commit 2033b3202ed6. The fork preserves input order while batching column merges, flattens ledge traversal, and skips clipping when bounds make it unnecessary. Inputs, defaults, the mod, and cache-clear scope are unchanged.

What is measured

Fixture is the "Garden of Dreams" mod on my POTI install, a save in "The Garden, Corpse Fields", scripted teleports between three interiors. Defaults: wait until min distance to player = 5, async nav mesh updater threads = 1, write to navmeshdb = true. Before every cold run only the tiles rows for the three Garden worldspaces are deleted from navmesh.db, assets and the OS page cache stay warm. The wall numbers below are the loading-barrier wait, i.e. until every required tile for every active AgentBounds in its current cache generation within the configured radius exists. The engine keeps building after that, it never waits for the whole worldspace. CPU and memory come from openmw-navmeshtool with only the two Recast object files swapped, which isolates the Recast half from the queue change.

Wall wait

measurement master branch result
White Cliffs → Tower required-tile transition wait 145.967 s 17.541 s 8.32x
canonical cold rotation, four stops, four 15 s dwells included ≈229.4 s 89.257 s 2.57x
same rotation, OpenMW change alone on base Recast 157.345 s

The rotation split says where the wait went: the OpenMW scheduling and barrier changes take the route from master ≈229.4 s to 157.345 s, then the Recast change takes it to 89.257 s.

Isolated Recast CPU

openmw-navmeshtool on ^The Garden, Tower$, identical OpenMW objects in both binaries, only RecastFilter.cpp.o and RecastRasterization.cpp.o swapped (base Recast vs pinned 2033b32), fresh DB, 1,061 tile attempts, 803 stored, /usr/bin/time -v.

workers base user+sys fixed user+sys result
1 (game default, mean of 2) 48.145 s 31.940 s 1.51x
15 (navmeshtool default here, mean of 2) 76.180 s 45.505 s 1.67x

Memory

The Recast change trades per-tile scratch for locality: packed spans merged per column in contiguous arrays before the linked lists are emitted, a flat snapshot for the ledge filter. Scratch lives for one tile build and is released after, each concurrent job carries its own copy.

workers base peak RSS fixed peak RSS delta
1 (game default, mean of 2) 370,124 KiB 444,918 KiB +73.0 MiB
15 (mean of 2) 658,538 KiB 1,041,068 KiB +373.6 MiB

Correctness

I put together a suite of local checks for serialized spans and every filter, and an A/B tool which normalizes only database-local IDs, then compares every remaining prepared-navmesh byte. I kept a "grand-tour" (teleport across all vanilla + expansions major cities) A/B aside from dev iterations to ensure I'm not faking correctness unknowingly and validated at the end.

gate result
deterministic Recast sampler 243/243 exact
Recast suite with the review supplement, Release and ASan+UBSan 52 cases, 25,065 assertions
focused OpenMW navigator tests, including barrier/capacity regressions 199/199
OpenMW component suite on the rebased head 1,447/1,447
Narsis roaming-group arrival, navmeshes and paths ready at the barrier 10/10
navmeshtool Tower A/B, prepared navmeshes 803/803 byte-identical
grand-tour prepared navmeshes 4,052/4,052 exact
Recast fork CI on 2033b32 Build and Tests green

CI currently and temporarily points at my GH fork so you can easily checkout / check the performance gains and correctness. Once the matching MR is accepted I will update the pointer back to the openMW fork repo. I'll keep this MR as draft until then to ensure nightly builds don't end up pulling from a (my) random GH.

Edited by Thomas Portal

Merge request reports

Loading
Loading