• Steven Estes's avatar
    [#25] Enable Go main routines using YottaDB Go wrapper to intercept fatal... · ab78909a
    Steven Estes authored
    [#25] Enable Go main routines using YottaDB Go wrapper to intercept fatal YottaDB issues via panic/recover interface
    Note this change also has a dependency on YottaDB r1.30 containing [YottaDB/DB/YDB#561].
    This change is implemented by a significant change to YottaDB C code and smaller changes to
    this wrapper. This wrapper has the following changes:
    1.  Instituted an atomic check of wrapper file variable (ydbInitialized) to see if YottaDB
        engine initialization is necessary and to initialize our signal monitoring goroutines.
    2.  If false, a call is made to ydb_main_lang_init() (see main commit message for issue
        YDB#561) which records both the language being initialized for (in this case Go) and an
        entry point to call to "panic" or equivalent. Note we must call ydb_main_lang_init()
        pre-emptively to initialize the Go wrapper instead of letting initialization default
        to running just ydb_init(). This call is what informs the runtime that we will be using
        alternate signal handling.
    3.  Initialization proceeds normally except since the language is Go, YottaDB does not
        install any signal handlers except one for the signal YDBSIGNOTIFY (equated to SIGUSR2
        at this point) that is used to tell the signal thread (ydb_stm_thread) that a signal
        is pending.
    4.  Once normal initialization concludes, we start up one goroutine for each signal YottaDB
        wants to be notifed about. A single channel is created for signal notification but each
        goroutine asks for notification of the signal it is monitoring and specifying this one
        channel as where it gets notified. Each goroutine also creates a notification channel
        which it records in an array. This is a shutdown notification channel that we can send
        shutdown notifications to for fatal signals and is used to synchronize shutdowns of
        these signal handling goroutines.
    5.  Once a goroutine is notified a given signal has occurred, it is then sent to YottaDB by
        a function call to the runtime (ydb_sig_dispatch).
    6.  There is a new external entry point in the wrapper named YDBWrapperPanic(int) that is
        intended for use by the wrapper but is usable by anyone. It will drive a panic for the
        signal number it is called with as a parameter.
    7.  I noticed a couple of issues with fetching text for given errors so restructured the
        NewError() function in error.go to better deal with these rare issues.
    8.  Also noticed some issues with driving ydb_exit() in the Exit() routine we suggest
        being always used with "defer" to make sure things shutdown nicely. Added a timer so
        it has limited time to do what it needs to do before the module finishes anyway.
        Previously, this had hung in rare instances.
    9.  The wrapper initially had a lot of references to "Golang" instead of "Go". The website
        is golang.org but this is to make the url more unique. The actual name of the language
        is "Go" and not "Golang" so it should be referred to in writing as such. So all references
        to "Golang"/"golang" that weren't part of a url in the wrapper were changed to Go/go
    The bulk of the changes are in the new init.go module.
    The changes in most routines are adding the atomic check for the initialization flag. Below
    are brief descriptions by module of the remaining changes done:
    - error.go - Had several instances in testing returning errors that had no text associated
      with them or got stuck in loops. Expanded NewError() such that it should not have these
      issues any more and will, at worst, at least given an unknown error type error message.
    - init.go - New module - Contains the new initialization done in the Go wrapper to
      support alternate sighandling mode in the YottaDB engine. This includes driving the
      new initialization in YottaDB, initializing the goroutines to monitor for the signals
      that YottaDB cares about, and the routine those goroutines run to initialize signal
      monitoring, notifying YottaDB when a signal occurs as well as an orderly shutdown when
      things go wrong.
    - init_cgo.c - New module - Necessary workarounds to cgo silliness to be able to pass the
      address of a Go callback routine through to C so C can call YDBWrapperPanic(). We need
      two routines because cgo won't allow us to pass a direct pointer to YDBWrapperPanic()
      to C so we need a C routine (gowrapper_panic_callback) that CAN call YDBWrapperPanic()
      to make that call and then we need a routine to fetch the address of
      gowrapper_panic_callback(). Tried to put these inside init.go but there are known
      problems doing this (after I googled it) so they have to be in a separate file.
    - misc.go - To fix a rare event (usually involving signals or other failures), ydb_exit()
      (as called by Exit()), may deadlock waiting for the engine lock so create a timer that
      times out after a defined number of seconds (currently defined as 10 seconds in yottadb.go)
      and lets the shutdown continue.
    - scripts/gen_error_codes.sh - Add a line so YDB_LOCK_DEFER definition gets pulled into
      error_codes.go from libyottadb.h.
    - util.go - Fix some panic msgs to include the YDB: prefix. Also, change MessageT() slightly
      to prevent loops or extraneous messages by adding some hardcoded checks for a couple of
      errors that would prevent their text being fetched if the engine had crashed.
    - variadic.go - Fix some panic messages to include a "YDB: " prefix.
    - yottadb.go - Made changes to the default initial allocation sizes of control blocks
      created on-the-fly by the EasyAPI. The values were reduced to reduce storage requirements
      which had arbitrarily been too large for most uses. The values will increase as needed
      during the lifetime of the process.
buffer_t_array.go 21 KB