Fixes to multiple issues affecting databases with READ_ONLY set
Final Release Note
Multiple issues with READ_ONLY features from GTM-8735 in the upstream code base that generated errors while accessing the help database were addressed and corrected. More details in the Description. As a consequence of this fix, each YottaDB process that accesses functionality that accesses a read-only database in $ydb_dist (e.g., the %PEEKBYNAME() utility program) consumes an additional OS semaphore. Monitor semaphore usage (e.g., with ipcs -s
) and ensure that your application has sufficient semaphores. (#150 (closed))
Description
GT.M V6.3-003 (which is merged in the YottaDB mainline from r1.20 onwards) introduced a new feature as part of GTM-8735. Below is the release note for that feature.
MUPIP SET -{FILE|REGION} recognizes the -[NO]READ_ONLY qualifier to indicate whether GT.M should treat an MM access method segment as read only for all users, including root. This designation augments UNIX authorizations and prevents any state updates that normally might require an operational action for a database with no current accessing (attached) processes. The GT.M help databases have -READ_ONLY set by default. Previously, a database such as the gtmhelp database in the GT.M distribution typically never received a data update but nevertheless could require a ROLLBACK, RECOVER or RUNDOWN to ensure a proper at-rest state. MUPIP emits an error on attempts to set -READ_ONLY on databases with the BG access method, or to set the access method to BG on databases with -READ_ONLY set. (GTM-8735)
Various issues were found in the above change during internal testing and code review at YottaDB.
-
The help database has the READ_ONLY flag set as well as read-only permissions on the file to all users (with the default install of YottaDB/GT.M, root owns the files in the install directory). In this setup, if a process opens the help database (e.g.
zhelp, $zpeek, $$^%peekbyname
etc.) and concurrently an argumentlessmupip rundown
runs, it would incorrectly delete the process-private semaphore of the help database created by the live process. Later when the process that opened the help database halts, it would get a DBFILERR/GVRUNDOWN error (with error detail "Unable to remove semaphore"). -
If one tries accessing the help database (with read-only permissions) after running an operational command that requires standalone access to the database file (e.g.
mupip integ -file $gtm_dist/gtmhelp.dat
), one gets a CRITSEMFAIL/SYSCALL/ENO22 error. From that point on, the help database is inaccessible. The only workaround to get it back and running was to turn the read_only feature off and back on (i.e. do amupip set -noread_only -file $gtm_dist/gtmhelp.dat
followed by amupip set -read_only -file $gtm_dist/gtmhelp.dat
, both of which require root access). -
If a process accesses the help database and halts cleanly, the ftok semaphore (ipcs -a shows a key of the form
0x2b......
) was left around. -
If more than one process does a
view "statshare"
(or sets thegtm_statshare
environment variable to 1) and opens a database file with READ_ONLY characteristic (it does not matter if it has read-only or read-write permissions) at the same time, the last process to halt out of the database issues a CRITSEMFAIL/SYSCALL/NOTALLDBRNDWN/GVRUNDOWN error. -
A mupip command to turn off the READ_ONLY feature and set the access method to BG (i.e.
mupip set -read_only -access_method=BG
) incorrectly issues a READONLYNOBG error. The access method specified is BG which is not compatible with READ_ONLY but should be compatible with the specified NOREAD_ONLY flag. -
A
mupip set -noread_only
turned off the read_only characteristic of the database file. But it did not ensure that no other processes were accessing the database. This meant that processes which started out when the READ_ONLY characteristic was turned on in the database file header could be running while themupip
command changed this characteristic. If one runs further ``mupip``` commands (e.g. to change access method from MM to BG etc.), it is possible for the process that still thinks the database file is READ_ONLY to incorrectly issue errors (e.g. GVGETFAIL, GVDATAFAIL etc.) while trying to read from a database that is otherwise good (but is out of date with respect to this process). It is not clear exactly what other race/timing issues are possible but clearly this is a situation best prevented in the first place.
Draft Release Note
Can be derived from the description section.