Skip to content

Fix Spring not working with macOS

Stan Hu requested to merge sh-fix-spring-macos into master

What does this MR do and why?

Previously the macOS-specific initializer was only run in Puma's before_fork block, but this code only gets executed when Puma is in clustered mode. However, Spring also needs this initializer because it works by forking the application.

We can simplify the initializer by always running it in macOS. This works because:

  1. Spring loads the application first, running all the initializers, and then forks the process.
  2. Puma preloads the application via preload_app! so the initializer runs before the process forks.
  3. Sidekiq and Rake tasks don't fork, but it doesn't hurt to run the initializer.

In addition, it appears that we need to add a call to CFTimeZoneGetName since otherwise we get the following error under Spring:

objc[77147]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[77147]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.

This was discovered by attaching a debugger to the forked process with the grpc gem compiled with debug symbols: GRPC_CONFIG=dbg gem install grpc -v 1.55.0

Debug backtrace:

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
  * frame #0: 0x0000000187124d4c libsystem_kernel.dylib`__abort_with_payload + 8
    frame #1: 0x0000000187149e50 libsystem_kernel.dylib`abort_with_payload_wrapper_internal + 104
    frame #2: 0x0000000187149de8 libsystem_kernel.dylib`abort_with_reason + 32
    frame #3: 0x0000000186de9504 libobjc.A.dylib`_objc_fatalv(unsigned long long, unsigned long long, char const*, char*) + 128
    frame #4: 0x0000000186de9484 libobjc.A.dylib`_objc_fatal(char const*, ...) + 44
    frame #5: 0x0000000186dbed54 libobjc.A.dylib`initializeNonMetaClass + 1088
    frame #6: 0x0000000186dd90e8 libobjc.A.dylib`initializeAndMaybeRelock(objc_class*, objc_object*, locker_mixin >&, bool) + 156
    frame #7: 0x0000000186dbe5c4 libobjc.A.dylib`lookUpImpOrForward + 884
    frame #8: 0x0000000186dbdf64 libobjc.A.dylib`_objc_msgSend_uncached + 68
    frame #9: 0x0000000187254288 CoreFoundation`-[__NSTimeZone name] + 28
    frame #10: 0x00000001487ce180 grpc_c.bundle`absl::lts_20230125::time_internal::cctz::local_time_zone() at time_zone_lookup.cc:140:29
    frame #11: 0x00000001487af40c grpc_c.bundle`absl::lts_20230125::LocalTimeZone() at time.h:1173:19
    frame #12: 0x00000001487af3d0 grpc_c.bundle`absl::lts_20230125::FormatTime(t=Time @ 0x000000016d1890e8) at format.cc:89:44
    frame #13: 0x00000001484cee2c grpc_c.bundle`grpc_core::StatusToString(this=0x000000016d189e38, type_url=(ptr_ = "created_time", length_ = 12), payload=0x00000002a65a9728)::$_0::operator()(absl::lts_20230125::string_view, absl::lts_20230125::Cord const&) const at status_helper.cc:323:45
    frame #14: 0x00000001484ce9d8 grpc_c.bundle`decltype(f=0x000000016d189e38, args=0x000000016d189b68, args=0x00000002a65a9728)::$_0 const&>()(std::declval(), std::declval())) absl::lts_20230125::base_internal::Callable::Invoke(grpc_core::StatusToString(absl::lts_20230125::Status const&)::$_0 const&, absl::lts_20230125::string_view&&, absl::lts_20230125::Cord const&) at invoke.h:185:12
    frame #15: 0x00000001484ce994 grpc_c.bundle`decltype(f=0x000000016d189e38, args=0x000000016d189b68, args=0x00000002a65a9728)::$_0 const&, absl::lts_20230125::string_view, absl::lts_20230125::Cord const&>::type::Invoke(std::declval(), std::declval(), std::declval())) absl::lts_20230125::base_internal::invoke(grpc_core::StatusToString(absl::lts_20230125::Status const&)::$_0 const&, absl::lts_20230125::string_view&&, absl::lts_20230125::Cord const&) at invoke.h:212:10
    frame #16: 0x00000001484ce950 grpc_c.bundle`void absl::lts_20230125::functional_internal::InvokeObject(ptr=VoidPtr @ 0x000000016d189b78, args=(ptr_ = "type.googleapis.com/grpc.status.time.created_time", length_ = 49), args=0x00000002a65a9728) at function_ref.h:74:7
    frame #17: 0x00000001487331c8 grpc_c.bundle`absl::lts_20230125::FunctionRef::operator(this=0x000000016d189c60, args=(ptr_ = "type.googleapis.com/grpc.status.time.created_time", length_ = 49), args=0x00000002a65a9728)(absl::lts_20230125::string_view, absl::lts_20230125::Cord const&) const at function_ref.h:132:12
    frame #18: 0x00000001487330fc grpc_c.bundle`absl::lts_20230125::Status::ForEachPayload(this=0x000000016d18a318, visitor=FunctionRef @ 0x000000016d189c60)>) const at status.cc:191:7
    frame #19: 0x00000001484cc0f0 grpc_c.bundle`grpc_core::StatusToString(status=0x000000016d18a318) at status_helper.cc:295:10
    frame #20: 0x000000014868734c grpc_c.bundle`grpc_error_get_status(error=(rep_ = 11380889169), deadline=(millis_ = 401216), code=0x00000002a65ab2a4, message="", http_error=0x0000000000000000, error_string=0x00000002a65ab2c8) at error_utils.cc:105:32
    frame #21: 0x00000001485f7ec4 grpc_c.bundle`grpc_core::FilterStackCall::SetFinalStatus(this=0x000000029dfb9290, error=(rep_ = 11380889169)) at call.cc:1055:5
    frame #22: 0x00000001485f8dd8 grpc_c.bundle`grpc_core::FilterStackCall::RecvTrailingFilter(this=0x000000029dfb9290, b=0x000000029dfb9a88, batch_error=(rep_ = 0)) at call.cc:1220:7
    frame #23: 0x00000001485fa600 grpc_c.bundle`grpc_core::FilterStackCall::BatchControl::ReceivingTrailingMetadataReady(this=0x000000029dfba3e0, error=(rep_ = 0)) at call.cc:1466:10
    frame #24: 0x00000001486055d8 grpc_c.bundle`grpc_core::FilterStackCall::StartBatch(this=0x000000029dfba3e0, bctl=0x000000029dfba3e0, error=)::$_7::operator()(void*, absl::lts_20230125::Status) const at call.cc:1807:9
    frame #25: 0x0000000148605590 grpc_c.bundle`grpc_core::FilterStackCall::StartBatch(bctl=0x000000029dfba3e0, error=)::$_7::__invoke(void*, absl::lts_20230125::Status) at call.cc:1807:9
    frame #26: 0x00000001485f9a50 grpc_c.bundle`grpc_core::Closure::Run(location=0x000000016d18a7f0, closure=0x000000029dfba090, error=(rep_ = 0)) at closure.h:303:5
    frame #27: 0x0000000148284490 grpc_c.bundle`recv_trailing_metadata_ready(arg=0x000000029dfba238, error=(rep_ = 0)) at deadline_filter.cc:150:3
    frame #28: 0x00000001485f9a50 grpc_c.bundle`grpc_core::Closure::Run(location=0x000000016d18a918, closure=0x000000029dfba260, error=(rep_ = 0)) at closure.h:303:5
    frame #29: 0x000000014802cae0 grpc_c.bundle`grpc_core::ClientChannel::FilterBasedCallData::RecvTrailingMetadataReadyForConfigSelectorCommitCallback(arg=0x000000029dfba1f0, error=(rep_ = 0)) at client_channel.cc:2341:3
    frame #30: 0x00000001484f3b18 grpc_c.bundle`exec_ctx_run(closure=0x000000029dfba2c8) at exec_ctx.cc:45:3
    frame #31: 0x00000001484f3998 grpc_c.bundle`grpc_core::ExecCtx::Flush(this=0x000000016d18bfa8) at exec_ctx.cc:72:9
    frame #32: 0x00000001484eb3b4 grpc_c.bundle`grpc_event_engine::experimental::PollEventHandle::NotifyOnLocked(this=0x000000014df04080, st=0x9b28257b27750069, closure=0x000000000005aaa9) at ev_poll_posix.cc:412:21

How to set up and validate locally

  1. Check out master.
  2. Make sure spring is not running: killall spring
  3. Run a feature spec, such as: bin/spring rspec spec/features/admin/admin_runners_spec.rb.
  4. You should see something like +[__NSTimeZone initialize] may have been in progress in another thread when fork() was called towards the end of the spec.
  5. Check out this branch and repeat the test.
  6. gitlab-ctl restart rails.
  7. Attempt to create a new file and other work in the GitLab UI.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Stan Hu

Merge request reports