Utils::server_run() should not return while Tango threads are still running
When a device server receives a SIGTERM it is caught by the ThSig thread which calls Utils::shutdown_ds
. At the end of Utils::shutdown_ds
we shutdown the ORB by calling CORBA::ORB::shutdown()
. When CORBA::ORB:shutdown
has finished the main thread returns from its call to CORBA::ORB::run
and the returns ultimately back to the user from the Utils::server_run
call.
Typically, the main thread will then return from main
and the C++ runtime will begin calling dtors of static objects. While this is going on the ThSig thread is still running and potentially accessing these static objects. This is UB and can lead to sefgfaults, see e.g. https://gitlab.com/tango-controls/cppTango/-/jobs/6927235040.
Crashes relating to this issue seem to be more common on the CI with the !1197 (merged) MR that introduced telemetry. I guess telemetry is slowing things down enough that we are running into segfaults which we weren't before.
We need to make sure that the ThSig thread (and any other Tango threads, e.g. the PollThread) have finished before we return from Utils::server_run
.