Lua: Create Lua threads to fix exception handling for nested calls

Lua, when compiled for C, uses setjmp and longjmp for exception handling, and creates a linked list L->errorJmp of struct lua_longjmp that is added to by every lua_pcall and similar function (ultimately by luaD_rawrunprotected.) The kazlib portable exception handling used in epan for Wireshark exceptions also uses setjmp and longjmp for exception handling.

Dissection of a frame occurs within a Wireshark TRY block. In the case of nested Lua dissector calls, a Wireshark exception that is unhandled by the inner Lua call but caught in the outer Lua call can result in a lua_error that longjmps to the most recent entry in the L->errorJmp list. If the two calls share lua_State, then that means longjmping to the jmp_buf that was setjmped in a luaD_rawrunprotected that has already exited (because of the longjmp caused by the uncaught Wireshark exception). That is undefined behavior and likely a crash.

To address this, use lua_newthread in order to create lua_States for each call that share global variables but have a separate execution stack and list of errorJmps. We then must ensure that the newly created thread is closed and garbage collected regardless of Wireshark exception.

This probably allows #15655 to be addressed without having to stick TRY..EXCEPT wrappers everywhere.

Fix #20767 (closed).

Merge request reports

Loading