Wrap all memory allocating Lua functions in protected calls

Fixes #8099 (closed), I think. Assuming I got everything and didn't make any mistakes.

I think this is how you're supposed to prevent your process from dying when Lua throws an error in a non-Lua (i.e. C++) context. Doing this correctly across the board and keeping it correct seems non-trivial...

The important part of this MR is the LuaState::invokeProtectedCall function which pushes an arbitrary lambda function onto the Lua stack (after checking if there is space for it) and executes it using lua_pcall. This way, if anything inside the lambda throws or triggers a Lua (OOM) error, we get a return status we can actually check. This behaviour is similar to what happens when we run a script through sol's protected function. The downside of this approach is that everything that can push onto the stack needs to happen through invokeProtectedCall or its equivalent, meaning a lot of code changes.

To ease the transition, I've moved a few methods from LuaState to LuaView and made the latter available only through invokeProtectedCall. I've also removed access to LuaState::mSol to turn the most obvious way of accessing a sol::state_view into a call to invokeProtectedCall which ensures usage of that state_view is safe. LuaState::unsafeState provides a way around this, but with a name and comment that will hopefully make people think twice before using it.

I tried to keep the changes to the tests minimal. As such not all of them are memory safe, but I figure a test that crashes counts as a failure all the same. I also don't really expect memory issues there.

Edited by Evil Eye

Merge request reports

Loading