API for Multi-threaded applications to use the single-threaded YottaDB engine
Final Release Note
Even though the YottaDB data management engine is single-threaded, YottaDB has Simple API functions to support multi-threaded applications. An overview is in the blog post YottaDB Support for Multi-threaded Applications and details are in the Multi-Language Programmers Guide. [#351 (closed)]
Description
Although the YottaDB engine is not thread-safe, and is unlikely to be thread-safe in the near future, it is desirable to allow multi-threaded applications to be able to use YottaDB. This Issue proposes a version of the Simple API for multi-threaded applications to use. If this API is implemented, in the even more distant future, when the YottaDB engine starts using multiple threads, no application code changes will be needed.
The proposed API is a set of *_st() functions that matches the Simple API *_s() functions, e.g, ydb_data_st(), matching ydb_data_s().
Under the covers, this is how they would work. The YottaDB runtime would execute in a thread it uses exclusively. In a C global area, ydb_init_st() would register the thread on which YottaDB is running, as well as a process-private queue in a C global area for communication.
When C application code (in a different thread from the YottaDB runtime) executes YottaDB function such as ydb_get_st(), a small YottaDB stub running in the same thread as the C application code puts an execution request in the queue. When YottaDB completes the access, it puts the result back in the queue from where the stub picks it up and returns it to the calling application code.
Local variables are interesting. Should they be considered shared across the threads of a process or should be considered specific to a thread? One possible solution is to extend the permitted names of local variables to allow them to also start with “#”. M local variables would be thread-private, and local variables starting with “#” would be shared across threads. Because of intereactions with the NEW command, we may opt to disable M code from running in a multi-threaded environment except for trigers, which are in M.
M locks are even more interesting, because an M lock can be:
- held by a thread, and visible only within the process (such that only the holding thread can release it);
- held by a thread and visible outside the process (with two options, depending on whether only the holding thread can release or any thread can release it); and
- held by the process, visible outside, with any thread able to release it.
Again, there are multiple design choices. One possible solution, again allowing for lock resource names to also start with “#”:
- Locks with names starting with “#” are process private (i.e., visible across threads but not outside the process). Only the thread owing such a lock can release it (YottaDB is able to scavenge abandoned locks).
- When a thread attempts to acquire a traditional M lock, the acquisition succeeds only if it already has, or is able to acquire, the traditional M lock name with a “#” prefix.
- When a thread attempts to release a traditional M lock, it is able to do so only if either it owns the traditional M lock name prefixed with “#” or if no thread owns that process-private lock. So a thread that acquires a traditional M lock, but wishes to allow any thread to release it would acquire the traditional M lock and then release the prefixed lock.
ydb_tp_st() presents even more interesting options and design choices whose discussion is best left till after the discussion of M local variables.