Draft: Make navmesh building fast.
Related to #9103
Cold transitions waited on unrelated navmesh jobs and slow Recast rasterization and ledge filtering. While requiredTilesPresent is active, queues favor capacity-freeing removals, jobs, and database reads for every active AgentBounds and cache generation required by the loading barrier. Changing worldspace clears local Recast state. Including the generation in job identity keeps stale work from matching new tiles.
Tile construction still runs on the configured idle-priority background worker. OpenMW pins Recast fork commit 2033b3202ed6. The fork preserves input order while batching column merges, flattens ledge traversal, and skips clipping when bounds make it unnecessary. Inputs, defaults, the mod, and cache-clear scope are unchanged.
What is measured
Fixture is the "Garden of Dreams" mod on my POTI install, a save in "The Garden, Corpse Fields", scripted teleports between three interiors. Defaults: wait until min distance to player = 5, async nav mesh updater threads = 1, write to navmeshdb = true. Before every cold run only the tiles rows for the three Garden worldspaces are deleted from navmesh.db, assets and the OS page cache stay warm. The wall numbers below are the loading-barrier wait, i.e. until every required tile for every active AgentBounds in its current cache generation within the configured radius exists. The engine keeps building after that, it never waits for the whole worldspace. CPU and memory come from openmw-navmeshtool with only the two Recast object files swapped, which isolates the Recast half from the queue change.
Wall wait
| measurement | master | branch | result |
|---|---|---|---|
| White Cliffs → Tower required-tile transition wait | 145.967 s | 17.541 s | 8.32x |
| canonical cold rotation, four stops, four 15 s dwells included | ≈229.4 s | 89.257 s | 2.57x |
| same rotation, OpenMW change alone on base Recast | 157.345 s |
The rotation split says where the wait went: the OpenMW scheduling and barrier changes take the route from master ≈229.4 s to 157.345 s, then the Recast change takes it to 89.257 s.
Isolated Recast CPU
openmw-navmeshtool on ^The Garden, Tower$, identical OpenMW objects in both binaries, only RecastFilter.cpp.o and RecastRasterization.cpp.o swapped (base Recast vs pinned 2033b32), fresh DB, 1,061 tile attempts, 803 stored, /usr/bin/time -v.
| workers | base user+sys | fixed user+sys | result |
|---|---|---|---|
| 1 (game default, mean of 2) | 48.145 s | 31.940 s | 1.51x |
| 15 (navmeshtool default here, mean of 2) | 76.180 s | 45.505 s | 1.67x |
Memory
The Recast change trades per-tile scratch for locality: packed spans merged per column in contiguous arrays before the linked lists are emitted, a flat snapshot for the ledge filter. Scratch lives for one tile build and is released after, each concurrent job carries its own copy.
| workers | base peak RSS | fixed peak RSS | delta |
|---|---|---|---|
| 1 (game default, mean of 2) | 370,124 KiB | 444,918 KiB | +73.0 MiB |
| 15 (mean of 2) | 658,538 KiB | 1,041,068 KiB | +373.6 MiB |
Correctness
I put together a suite of local checks for serialized spans and every filter, and an A/B tool which normalizes only database-local IDs, then compares every remaining prepared-navmesh byte. I kept a "grand-tour" (teleport across all vanilla + expansions major cities) A/B aside from dev iterations to ensure I'm not faking correctness unknowingly and validated at the end.
| gate | result |
|---|---|
| deterministic Recast sampler | 243/243 exact |
| Recast suite with the review supplement, Release and ASan+UBSan | 52 cases, 25,065 assertions |
| focused OpenMW navigator tests, including barrier/capacity regressions | 199/199 |
| OpenMW component suite on the rebased head | 1,447/1,447 |
| Narsis roaming-group arrival, navmeshes and paths ready at the barrier | 10/10 |
| navmeshtool Tower A/B, prepared navmeshes | 803/803 byte-identical |
| grand-tour prepared navmeshes | 4,052/4,052 exact |
Recast fork CI on 2033b32 |
Build and Tests green |
- I have read the CONTRIBUTING guidelines
- My code runs locally
- My code passes CI on my fork
CI currently and temporarily points at my GH fork so you can easily checkout / check the performance gains and correctness. Once the matching MR is accepted I will update the pointer back to the openMW fork repo. I'll keep this MR as draft until then to ensure nightly builds don't end up pulling from a (my) random GH.