THIS IS A DRAFT. YottaDB r1.30 HAS NOT YET BEEN RELEASED.

YottaDB r1.30

Binary Distributions

sha256sum	file

Release Note Revision History

Revision	Date	Summary
1.00	DATE	r1.30 Initial Release

Contact Information

YottaDB LLC

40 Lloyd Avenue, Suite 104 Malvern, PA 19355, USA [email protected] +1 (610) 644-1898

Support

Customers

Contact your YottaDB support channel.

Others

For free (to you) support from members of communities who run widely available applications on YottaDB, please use an application-specific list where appropriate.

• For requests other than to the communities below, post an Issue and include the words "Help Wanted" in the summary.
• For requests specific to the use of YottaDB from node.js via nodem, QewdJS, or EWD.js, post to the Enterprise Web Developer community.
• For requests specific to the use of YottaDB with VistA flavors, post to the Hardhats list.
• For requests specific to the use of YottaDB with M other than for applications above, post to the comp.lang.mumps list.

r1.30

Overview

YottaDB is a major release with important new functionality that is required by the Octo 1.0 production release and the production release of the Rust wrapper.

Note: YottaDB r1.30 anticipates release of production releases of Octo and the Rust wrapper. These release notes will be updated when they are released.

The most important new functionality, required by Octo to distinguish between the empty string ("") and the absence of data (NULL), is the intrinsic special variable, $ZYSQLNULL, which can be used as a local variable subscript or node value. (484)

Other highlights include:

• The ydbinstall / ydbinstall.sh scripts include --enclugin, --octo, and --posix command line options to install respectively, the encryption plugin, Octo, and the POSIX plugin. (458), (457), (521)
• The call to $ZJOBEXAM() allows for specification of the data to dumped. So, for example, the dumping of local variables can be suppressed unless required, as local variables can contain confidential information. (482)
• For faster numeric base conversion, $ZCONVERT() converts between decimal and hexadecimal. (485)
• A new implementation of $RANDOM(). (384)
• $ZYHASH() returns the 128-bit MurmurHash3 hash of a string. (390)
• Simple API functions use nanosecond timers internally. (388)
• To facilitate migration to YottaDB from big endian versions of the upstream code base, YottaDB can automatically convert global directories from big to little endian formats. Note that all YottaDB platforms are little endian. (524)
• CentOS 8 is a Supported platform with its own binary distribution.

YottadB r1.30 is upward compatible with YottaDB r1.28, as well as with GT.M V6.3-008.

As with any YottaDB release, there are numerous other enhancements and fixes, some specific to YottaDB and others inherited from the upstream GT.M V6.3-008. See the Change History.

Platforms

A platform is a combination of a CPU architecture and an operating system. A platform is Supported, Supportable, or Unsupported.

• Supported means that we have the platform in our development environment and test each release on that platform.
• Supportable means that although we do not necessarily have such a platform in our environment, we have no reason to believe that the software will not run on it.
• All others are Unsupported.

CPU Architecture	Supported OS Version(s)	Notes
64-bit x86	Ubuntu 18.04 LTS; Red Hat Enterprise Linux 7.x; Debian GNU/Linux 10 (Buster); CentOS 8.x	There are separate binary distributions for each OS version, owing to differences in library versions of those distributions.
64-bit ARM (Raspberry Pi 3 Model B)	Ubuntu 18.04 LTS	See below.
32-bit ARM (Raspberry Pi 3 Model B)	Raspbian GNU/Linux 10 (Buster)	See below.
32-bit ARM (Raspberry Pi Zero)	Raspbian GNU/Linux 10 (Buster)	See below.

Recent releases of major GNU/Linux distributions with contemporary kernels, glibc and ncurses are Supportable. Specific notes:

• On Ubuntu releases after 18.04 LTS, YottaDB needs the libtinfo5 package to be installed.
• On Arch Linux and other leading edge distributions, YottaDB may need to be recompiled from source code owing to library and tool chain versions significantly newer than those used in building the distribution.
• While YottaDB is Supportable on other ARMv6, ARMv7-A, and ARMv8-A CPUs, owing to variations in the implementations of ARM microarchitectures, we recommend that you ensure the software runs correctly before committing to any specific hardware other than those Supported. Please contact [email protected] if you want a specific combination of OS and CPU microarchitecture to be Supported.

Installation

See our Get Started page to use YottaDB.

We strongly recommend that you install YottaDB r1.30 in a newly created directory, different from those of prior YottaDB releases and any GT.M versions you may have installed on the system.

Removing an installed YottaDB release

Assuming $ydb_dist points to the directory where YottaDB is installed:

• Cleanly shut down all application processes using that release.
• Execute mupip rundown && mupip rundown -relinkctl
• Ensure that there are no gtcm* or gtmsecshr processes active.
• Use sudo lsof | grep $ydb_dist to ensure there are no open files.
• Delete the directory with sudo rm -rf $ydb_dist

Upgrading to YottaDB r1.30

As YottaDB r1.30 is upward compatible from YottaDB r1.28 and GT.M V6.3-008, the minimal upgrade steps are:

• Install YottaDB r1.30.
• Install plugins you use. Note that the ydbinstall / ydbinstall.sh shell scripts can install the encryption and POSIX plugins after installing YottaDB. (457) and (458)
• Recompile object code, and recreate shared libraries where appropriate.
• If you are using encryption, compile and install the reference implementation plugin (if not done by the ydbinstall / ydbinstall.sh script) or your customized plugin.
• Cleanly shut down the application and ensure that the database files are shut down using MUPIP RUNDOWN from the prior release.
• Switch journal files with the new YottaDB release.
• Start using the new YottaDB release.

If the database has triggers defined with non-canonical numbers, or numbers specified as strings with any version prior to r1.28, or if you are unsure, extract the trigger definitions, delete existing triggers, and reload the trigger definitions. Issue [#430] from r1.28 has a series of steps you can copy and execute. There is no need to do this if upgrading from r1.28.

To upgrade from older GT.M releases, first upgrade to GT.M V6.0-000 or later and follow the steps above, or contact your YottaDB support channel for assistance and guidance.

A more sophisticated upgrade technique is:

• Install YottaDB r1.30.
• Create a new replicated instance of your application (on the same system or a different system).
• Assuming the existing instance is A, and the new instance is B, upgrade B to r1.30 and start replicating from A to B.
• Once B catches up, switchover so that B is in a primary role replicating to A.
• Once you are satisfied with B, remove (or upgrade) A.

Change History

r1.30

YottaDB r1.30 includes the following enhancements and fixes beyond YottaDB r1.28.

ID	Category	Summary
(384)	Languages	Reimplement $RANDOM()
(388)	Other	Nanosecond resolution timer for Simple API times
(390)	Languages	$ZYHASH(string[,salt]) function
(457)	Other	ydbinstall option to install POSIX plugin after installing YottaDB
(458)	Other	ydbinstall option to install encryption plugin after installing YottaDB
(470)	Other	$ydb_dist copied to gtm_dist during initialization
(476)	Languages	$ZSIGPROC() accepts signal names
(482)	Languages	Control contents of process dump generated by $ZJOBEXAM()
(484)	Languages	$ZYSQLNULL intrinsic special variable and $ZYISSQLNULL() function
(485)	Languages	$ZCONVERT() converts integers between decimal and hexadecimal
(488)	Languages	Environment translation for extended references; global directory translation
(492)	Languages	UTF-8 mode $TRANSLATE() with multi-byte string literals as the second or third parameter works correctly when executed from shared library
(493)	System Administration	More informative default first line from MUPIP EXTRACT in GO and ZWR formats
(494)	System Administration	Spaces in labels in GO and ZWR format MUPIP EXTRACT
(495)	Other	Option to recompile only if .m file is newer than corresponding .o file
(496)	Other	Object code files linkable into shared libraries by ld version 2.33 and later
(503)	Database	DRD not incremented for MM access method
(504)	Database	On ARMv6 and ARMv7-A platforms, syslog messages recording database update failures work correctly
(511)	Languages	$[Z]TRANSLATE() correctly reports error when second and third arguments are undefined
(513)	Languages	$VIEW("REGION","^*") returns name of default region
(515)	Languages	ydb_zstatus() has a return value
(518)	Languages	Support for 64-bit integer types on 64-bit platforms when calling in to M
(519)	Languages	SOCKET device timeouts happen at specified times
(520)	Languages	Repeatedly switching between $ZTRAP and $ETRAP works correctly
(521)	Other	ydbinstall script option to install Octo as a YottaDB plugin
(523)	Languages	IF '@ works when operand is a constant or glvn evaluating to a constant
(524)	Database	Auto convert global directory to endianness of current architecture
(525)	Languages	SILENT^%RSEL restores value of $IO
(532)	Other	ydbinstall with --utf8 default installs encryption plugin
(534)	Languages	Process-terminating signals handled correctly in Simple API multi-threaded functions
(545)	Languages	LOCK command works correctly after a prior LOCK command interrupted by an error
(547)	Languages	Extrinsic function calls where the caller and label are hundreds of thousands or more lines apart work correctly on 32-bit ARMv7 CPUs
(549)	Languages	Environment variables are expanded in $ZROUTINES
(550)	Languages	On successful commit of outer transactions, only updates of inner transactions that return status other than YDB_OK or YDB_TP_RESTART are rolled back
(553)	Languages	Boolean expressions involving the NOT operator (') and side effects compile when ydb_boolean=1
(554)	Languages	Boolean expressions with side effects compile when used in a context requiring an integer and ydb_boolean or ydb_side_effects are 1
(562)	Languages	Setting $ZROUTINES to "..." or "...." issues a ZROSYNTAX error
(566)	Languages	Comments in Call-In and External call tables
(568)	System Administration	MUPIP commands that output to a pipe leave terminal settings untouched
(569)	System Administration	MUPIP LOAD option to ignore mode of MUPIP EXTRACT that created ZWR extract
(576)	Languages	$PIECE() in a database trigger returns correct results when invoked from SimpleAPI
(577)	Languages	Normal termination on receipt of SIGTERM when inside a transaction
(578)	Languages	$ZSTATUS for LVUNDEF error quotes string subscripts
(579)	System Administration	Standard action for $ydb_procstuckexec to invoke
(580)	Languages	timer_id parameter of ydb_timer_cancel(), ydb_timer_cancel_t(), and ydb_timer_start_t() is type intptr_t
(583)	Languages	CITABOPN error details why fopen() call failed
(584)	Languages	$ZSTATUS after OPEN command errors identifies the file name
(586)	Database	No SYSCALL error in rare cases when MUPIP STOP sent to YottaDB process
(589)	Languages	Buffered IO writes from external calls are flushed by YottaDB when it closes IO devices
(590)	Languages	Identify source checkin commit hash in $ZRELDATE
(592)	System Administration	JOB'd processes default to same name as parent
(594)	Languages	Robust results from ydb_incr_s()/ydb_incr_st() and language wrapper APIs
(595)	Other	yottadb -version provides a detailed report on the YottaDB build
(596)	Database	$ZTDELIM returns piece separator for KILL and ZKILL triggers
(607)	Database	Upgrading a database from r1.22 or r1.24 upgrades the flush_trigger_top field
(609)	Database	MUPIP STOP can cause DBMBPINCFL or DBMBMINCFRE errors in database files of frozen instances

GT.M V6.3-008

YottaDB r1.30 incorporates enhancements and fixes from GT.M V6.3-008. In some cases, fixes from the GT.M release notes are not shown here because those issues may have been identified and already been fixed in earlier releases of YottaDB, or they were independently fixed by the YottaDB team (for example, (511]) is a more complete fix than (GTM-9093), the latter being only a partial fix for the same issue).

ID	Category	Summary
(GTM-8911)	Other	Defer allocation of memory for TP processing until the region participcates in a TP transaction
(GTM-9000)	Other	^%PEEKBYNAME accepts an optional argument to specify a global directory
(GTM-9012)	Database	Processes automatically release online freeze only if they do not find enough global buffers.
(GTM-9067)	Database	Avoid WCBLOCKED caused by delayed I/O
(GTM-9077)	Languages	SOCKET Devices properly report TLS errors
(GTM-9078)	Database	Avoid potential deadlock caused by delay I/O with Instance Freeze
(GTM-9079)	Languages	Prevent inappropriate error status from compile of certain literal arguments to an XECUTE
(GTM-9082)	Other	Fix to auto-upgrade for database file header error in V6.3-007
(GTM-9083)	Database	[NON]TPRESTART messages present numeric codes with a hexadecimal representation
(GTM-9084)	System Administration	TLS reference implementation partially supports OpenSSL 1.1.1
(GTM-9089)	Database	NULSUBSC error provides more accurate context
(GTM-9092)	Other	IN^%YGBLSTAT function checks for process connection to GT.M; also an enhancement and a fix
(GTM-9097)	Other	Prevent Source Server hang when processing corrupted journal files
(GTM-9098)	System Administration	MUPIP JOURNAL reports the correct numbers when issuing a continuity check failure
(GTM-9099)	Languages	The compiler warns of argumentless DOs without a valid block
(GTM-9100)	Languages	Resume from Out-Of-Band Operations during XECUTEd indirect extrinsics nesting interruptable actions
(GTM-9110)	Other	limit GT.M commands from the shell to 32KiB

Database

• The DRD count stored in the database fileheader, and reported by ZSHOW "G" and $VIEW("GVSTAT",<region>) is not incremented for database accesses when using the MM access method. Previously, it could be incremented, even though DRD is not meaningful for the MM access method. [#503]

• Syslog messages recording failures in certain database update operations work correctly. Previously such failures could cause segmentation violations (SIGSEGV). This issue was restricted to ARMv6 and ARMv7-A platforms and was only observed in the development environment; it was never reported by a user. [#504]

• When GDE opens a global directory file, it converts the global directory file to the endianness of the current architecture. Previously, this would result in an ZGBLDIRACC error. After conversion, if the global directory needs upgrading to the current release of YottaDB, existing logic performs such a conversion. Although all YottaDB platforms are little endian, this facilitates upgrading to YottaDB from versions of the upstream software on big endian platforms. [#524]

• MUPIP STOP (SIGTERM/SIG-15 signal) of a YottaDB process that has updated one or more database files works correctly. Previously it was possible in very rare cases for the process to terminate with a SYSCALL error. [#586]

• Within a KILL or ZKILL trigger the intrinsic special variable $ZTDELIM returns the piece separator (specified by -delim in the trigger definition). Previously, $ZTDELIM was maintained only for SET triggers. [#596]

• The flush_trigger_top field is set correctly when a database is auto-upgraded from YottaDB r1.22 or r1.24. [#607]

• MUPIP STOP (i.e. SIGTERM / kill -15) of a YottaDB process while it has a database file open that is part of a frozen instance leaves the database in a clean state. Previously, it was possible in rare cases for such database files to get DBMBPINCFL or DBMBMINCFRE structural integrity errors. [#609]

• When initiating a MUPIP FREEZE -ONLINE -AUTORELEASE, processes release the freeze if they cannot find global buffers to do their work. Previously, they occasionally released the freeze prematurely. (GTM-9012)

• Processes handle very long delays in database and journal file writes appropriately. Previously, they could incorrectly issue a WCBLOCKED message in such cases, resulting a variety of incorrect behavior, including a segmentation violation (SIG-11). (GTM-9067)

• Processes avoid a possible deadlock between a database critical resource and a database buffer flush wait caused by an extended period of significantly delayed I/O when using an Instance Freeze for certain journaling conditions. (GTM-9078)

• TPRESTART and NONTPRESTART messages report non-graphic codes as "0xnn". The numeric codes are associated with restarts caused by implementation details, rather than database conflicts and are normally infrequent. Previously they reported these as octal values or as periods (.) depending on the configuration of the device presenting the report. (GTM-9083)

• The NULSUBSC error now provides more accurate context; previously, it always indicated an update. (GTM-9089)

Languages

• The underlying implementation of $RANDOM() has been changed with no change to the API. Note that $RANDOM() should never be used when cryptographic quality random numbers are needed. [#384]

• The M function $ZYHASH(string[,salt]) returns the 128-bit MurmurHash3 of string as a hexadecimal string prefixed with "0x". For salt values of 0 (the default) through 2,147,483,647 (2**31-1) this is equivalent to calling the C API function ydb_mmrhash128() and passing its return value to ydb_mmrhash128_hex(). [#390]

• In $ZSIGPROC(expr1,expr2), expr2 can optionally be a signal name, such as "USR1", optionally prefixed by "SIG", e.g., "SIGUSR1". The strings are case-insensitive. Previously $ZSIGPROC() required numeric signal values. [#476]

• Defaulting to "*", $ZJOBEXAM() takes a second parameter whose string value specifies the ZSHOW output codes as a string to be included in the process dump. For example, $ZJOBEXAM("isvsonly.txt","I") dumps only the intrinsic special variables, and nothing else, to the output file. Previously, the $ZJOBEXAM() process dump was always equivalent to ZSHOW "*". [#482]

• The read-only $[ZYSQ]LNULL intrinsic special variable is conceptually equivalent to a logical value of unknown, and can be assigned as the value or used as a subscript of a local variable.

  When $ZYSQLNULL is an operand, the results are as follows:

  Operator(s)	Result
  Binary !	If the other operand evaluates to true (1), the result is true, otherwise the result is $ZYSQLNULL
  Binary &	If the other operand evalutes to false (0), the result is false, otherwise the result is $ZYSQLNULL
  All binary operators except ! and &	Regardless of the value of the other operand, the result is $ZYSQLNULL
  Unary +	$ZYSQLNULL
  Unary -	$ZYSQLNULL
  Unary '	$ZYSQLNULL

  $TEST continues to have only 2 values : false (0) and true (1). An IF statement whose condition evaluates to $ZYSQLNULL sets $TEST to 0 and does not execute the rest the line. Commands with postconditionals that evaluate to $ZYSQLNULL do not execute the command.

  ZSHOW and ZWRITE of $ZYSQLNULL values show a value of $ZYSQLNULL. WRITE does not show any value for $ZYSQLNULL just as it does with "".

  $ZYSQLNULL can be a subscript of a local variable. In that case, it collates after all other subscripts, i.e., $ORDER() and $QUERY() return that subscript at the very end.

  The function $ZYISSQLNULL() returns 1 if its sole argument has a value $ZYSQLNULL, and 0 otherwise.

  Using $ZYSQLNULL as a subscript or assigning it as the value of a global variable (including implicitly with a MERGE), using it as a subscript in a LOCK/ZALLOCATE/ZDEALLOCATE command, or in a context that expects an integer or a numeric value raises the ZYSQLNULLNOTVALID error. Other than usage as an operand as discussed above, $ZYSQLNULL in a context that expects a string, e.g. $ASCII($ZYSQLNULL,1), is treated like the empty string "". [#484]

• For fast numeric conversion between decimal and hexadecimal, the second and third parameters of $ZCONVERT() can be case-insensitive string values "HEX" or "DEC", the second parameter being the base of the string or number to be converted and the third the base of the output.

  • Unsigned numbers in the range 0 through 0xFFFFFFFFFFFFFFFF can be converted. Decimal return values greater than 999999999999999999 (YottaDB's maximum numeric size) are returned as strings.
  • Hexadecimal numbers are always converted to positive decimal numbers.
  • As conversion from hexadecimal numbers preceded by "-" to decimal is not considered meaningful, if the number to be converted is a “negative” hexadecimal number (e.g., "-F"), the result is 0.
  • Conversion from negative decimal numbers to hexadecimal returns the hexadecimal value of the 2's complement of the number, e.g., the value of $ZCONVERT(-23,"DEC","HEX") is "E9".

  The ^%DH() and ^%HD() utility programs use fast $ZCONVERT() conversion when the values supplied are in the range that $ZCONVERT() supports; they otherwise revert to existing conversion logic.

  $ZCONVERT() raises the INVCONVERT error for unsupported conversions (e.g., a Unicode conversion in M mode, or converting from DEC to DEC or HEX to HEX) and the INVVALUE error when the input number is outside the supported range. [#485]

• The environment variable ydb_env_translate can identify a shared library containing a routine to provide an optional environment translation for extended references. If $ydb_env_translate is undefined, YottaDB can use a shared library identified by $gtm_env_translate; defaulting to no environment translation. Previously, YottaDB only sought a value of the gtm_env_translate environment variable.

  Global directories defined by $zgbldir can also be similarly translated. When $zgbldir is set, if the environment variable ydb_gbldir_translate identifies a shared library with the entry point ydb_gbldir_xlate(), the global directory used is the value assigned to $zgbldir as translated by the routine. ydb_gbldir_xlate() has the same signature as the ydb_env_xlate() routine used for environment translation:

  int ydb_gbldir_xlate(ydb_string_t *in1, ydb_string_t *in2, ydb_string_t *in3, ydb_string_t *out);

  where:

  • in1 references the value being assigned to $zgbldir.
  • in2 is the NULL string - the parameter exists only so that the signature matches that of ydb_env_translate().
  • in3 references $zdirectory the current directory of the process.
  • out is a return value that references the actual global directory file to be used.

  A return value other than zero (0) indicates an error in translation, and is reported as a YottaDB error.

  We thank Thomas Morstein (@ztmr) for working with us on this enhancement. [#488]

• When executed from a routine in a shared library, $TRANSLATE() in UTF-8 mode works correctly when the second or third arguments are multi-byte literals. In YottaDB r1.26 and r1.28, this could result in abnormal process termination with a segmentation violation (SIGSEGV). [#492]

• A first call to $[Z]TRANSLATE() where the first argument is defined but the second and third are undefined results in an error. Previously, this caused abnormal process termination with a segmentation fault (SIG-11) in UTF-8 mode, and no output in M mode. [#511]

• $VIEW("REGION","^*") returns the name of the region in the global directory to which the * namespace maps (as seen in a $ydb_dist/yottadb -run GDE SHOW -NAME output). Previously there was no easy way to determine the default region name. [#513]

• ydb_zstatus() has an int return value with a value of YDB_ERR_INVSTRLEN if the buffer supplied is not large enough to hold the message and YDB_OK otherwise. As with other YDB_ERR_INVSTRLEN return codes, ydb_zstatus() copies what can be copied to the buffer (including a null terminator byte) if the length is non-zero. Previously, ydb_zstatus() would attempt to copy the entire message to the buffer even if the buf_len parameter was zero. [#515]

• On 64-bit platforms, ydb_ci(), ydb_cip(), ydb_ci_t(), and ydb_cip_t() support ydb_int64_t and ydb_uint64_t data types. These 64-bit types are not yet supported on 32-bit platforms. [#518]

• Opening a SOCKET device with a timeout times out as specified. Previously the timeout could happen too early or much later than requested. Note that while system load and overhead can always cause a timeout to happen later (which was not the case here), a timeout should never happen sooner. [#519]

• Switching error trapping back and forth by alternating assignments to $ETRAP and $ZTRAP works correctly. Previously YottaDB could lose the error trap assignment, resulting in an empty error trap. [#520]

• IF '@ syntax works when the operand is a constant or a glvn that evaluates to a constant (e.g. IF '@"0", IF '@x). Previously this issued a VAREXPECTED error. [#523]

• <a name="x525"SILENT^%RSEL returns with the current M IO device ($IO) restored to what it was at the time of the call. Previously, it would set it to $PRINCIPAL. [#525]

• Signals that terminate a YottaDB process are handled correctly in multi-threaded functions of the Simple API. For example, invoking the abort() function (which raises a SIGABRT signal) inside a TP callback function correctly generates a core file and terminates the process immediately. Previously such signals could cause the YottaDB process to hang. [#534]

• A LOCK command after a prior LOCK command interrupted by an error works correctly. Previously, it would result in a BADLOCKNEST error. [#545]

• Extrinsic function calls where the calling M line is hundreds of thousands or more M lines away from the called M line work correctly on the ARMv7 architecture (32-bit ARM). Previously, the process would terminate abnormally with a SYSTEM-E-UNKNOWN and/or SIG-11 error. [#547]

• Any environment variables in a string used to set $ZROUTINES. either in an environment variable at process startup or explicitly during execution, are expanded into absolute path names, eliminating the need for application code using $ZROUTINES to do the expansion. Previously, environment variables were not expanded. [#549]

• In applications using the C API (and other language APIs that wrap the C API), updates from outer transactions that invoke nested inner transactions which return YDB_TP_ROLLBACK are committed upon a successful transaction commit; updates of inner transactions whose callback functions return any status to ydb_tp_s() / ydb_tp_st() other than YDB_OK or YDB_TP_RESTART are rolled back. Previously, updates from the enclosing transaction were also rolled back, whereas only the updates of the inner transactions should have been rolled back. The workaround was to return YDB_TP_ROLLBACK through all transaction levels to the top application code level outside a transaction, and for that code to re-invoke YottaDB transaction logic. [#550]

• Boolean expressions involving the NOT operator (') and side effects ($increment(), extrinsic function calls, etc.) compile when the ydb_boolean or ydb_side_effects environment variables are set to 1. Previously, the compilation would hang (loop for ever). [#553]

• Boolean expressions with side effects compile when used in a context that requires an integer (e.g. as an argument to $RANDOM()) and when the ydb_boolean or ydb_side_effects environment variables are set to 1. Previously terminated the process with a fatal GTMASSERT2 error. [#554]

• Setting $ZROUTINES to "..." or "...." either with a SET command or from $ydb_routines or $gtmroutines at process startup issues a ZROSYNTAX error. Previously those meaningless directory specifications were accepted. [#562]

• Call-in tables for calling M routines from other languages, and external-call tables to call out from M code to non-M routines support comments. YottaDB ignores text from a double slash (//) on a line to the end of the line. Previously, there was no support for comments in these tables. [#566]

• MUPIP commands that output to a pipe, such as mupip extract -stdout | less leave terminal settings untouched. Previously MUPIP modified and restored terminal settings even when its output was to a pipe. As a program such as less will modify and restore terminal settings (since it expects to control terminal output), the previous behavior could result in terminal settings left in an unfriendly state in the event the command was terminated, e.g., by pressing Ctrl-C at the terminal, requiring an stty sane to restore terminal settings. [#569]

• The -ignorechset command line option of MUPIP LOAD tells MUPIP to load the extract even if it was created by a MUPIP process in another mode (UTF-8 mode vs. M mode). Effective r1.30 (see 493) YottaDB checks that the modes match, reporting a LOADINVCHSET error if they did not. As using -ignorechset bypasses YottaDB checks, use it only if you are sure that the extract file can be loaded correctly. [#569]

• $ZSTATUS for an LVUNDEF error message quotes string subscripts. Previously, it did not quote string subscripts. [#578]

• %YDBPROCSTUCKEXEC is a standard utility program to capture diagnostics when invoked by the ydb_procstuckexec mechanism. To use it, set the ydb_procstuckexec environment variable to $ydb_dist/yottadb -run %YDBPROCSTUCKEXEC with $ydb_dist expanded to the actual directory where YottaDB is installed, or to yottadb -run %YDBPROCSTUCKEXEC when yottadb is located by $PATH. Also $ydb_dist/libyottadbutil.so must be in the routine search path defined by $ydb_routines at process startup, or in the $ZROUTINES intrinsic special variable. When invoked by yottadb -run %YDBPROCSTUCKEXEC <msg> <callingpid> <blockingpid> <count> as described in the ydb_procstuckexec documentation, it creates in the directory specified by $ydb_log, $gtm_log, $ydb_tmp, or $gtm_tmp (defaulting to /tmp) a file whose name is %YDBPROCSTUCKEXEC_<date>,<time>_<msg>_<callingpid>_<blockingpid>_<count>_<%YDBPROCSTUCEXECpid>.out where the , timestamp is generated by $ZDATE() with a format string of "YEAR.MM.DD,24.60.SS", e.g., %YDBPROCSTUCKEXEC_2020.05.22,17.58.38_ABCD_91998_91987_1_92045.out. The file contains:

  • A summary of the invocation, e.g., Invoked on 91987 by 91998 for 1st time; reason: MUTEXLCKALERT.
  • The command line of the blocking process.
  • The environment of the blocking process.
  • The value of /proc/sys/kernel/yama/ptrace_scope (if non-zero, gdb may not be able to capture the blocking process for a snapshot of its state).
  • A snapshot of the state of the blocking process, captured by gdb.
  • The value returned by $ZSIGPROC() used to send SIGUSR1 (a MUPIP INTRPT) to the blocking pid. If appropriately configured (with ydb_zinterrupt / $ZINTERRUPT, the blocking process will create a dump file of its process state.

  If the calling pid and blocking pid are of different uids, the functionality of %YDBPROCSTUCKEXEC is reduced. Depending on security and system administration considerations of the system, it may be appropriate in such cases to create a small setuid root shell script to invoke %YDBPROCSTUCKEXEC. Depending your specific requirements, it may be appropriate to copy and adapt the standard routine for your environment.

  Previously, YottaDB provided the ydb_procstuckexec mechanism for a blocked process to invoke a monitoring script, but not a specific tool. [#579]

• $PIECE() in trigger logic invoked by ydb_set_s()/ydb_set_st()/ydb_incr_s()/ydb_incr_st() works correctly. Previously, such a $PIECE() invocation could return incorrect results. This was observed in the development / test environment and was never reported by a user. [#576]

• A YottaDB process that does transaction processing (using TSTART / TCOMMIT in M, ydb_tp_s()/ydb_tp_st() in C and languages that use the C API, etc.) terminates normally when it receives a SIGTERM signal (SIG-15 or MUPIP STOP). Previously it could in rare case terminate with a TPLOCK error. This was only observed in in development / test environments, and was never reported by a user. [#577]

• The timer_id parameter of ydb_timer_start_t(), ydb_timer_cancel_t() and ydb_timer_cancel() is of type intptr_t. Previously this was of type int which was inadequate for a 64-bit platform. [#580]

• The CITABOPN error has additional detail on why the fopen() of the call-in table file failed (e.g., %SYSTEM-E-ENO2, No such file or directory). Previously this was not provided, making it hard to troubleshoot this error. [#583]

• $ZSTATUS, after an OPEN command that encounters an error, identifies the associated file name in addition to the error (e.g. %YDB-E-DEVOPENFAIL, Error opening /tmp/x.txt, %SYSTEM-E-ENO13, Permission denied). Previously it only identified the error without the associated file name (e.g. %SYSTEM-E-ENO13, Permission denied). [#584]

• YottaDB flushes all buffered writes done outside of its runtime logic (e.g., a puts() from C code called by a base M program) as part of closing open IO devices. Previously it was possible for some non-M buffered output to be lost when the output was piped to other programs, for example when the output of yottadb -run was piped to another program. [#589]

• The third piece of $ZRELDATE is the hexadecimal hash assigned by the version control system to identify the last source code commit. Previously, $ZRELDATE only had two pieces. [#590]

• ydb_incr_s()/ydb_incr_st() and language wrappers that use those functions, such as IncrE() implemented by the Go wrapper, work correctly on local variable nodes. Previously, it was possible in rare cases for a subsequent operation accessing such a local variable node (e.g. ydb_get_s()) to abnormally terminate with a SIG-11 or to report incorrect results. [#594]

• SOCKET Devices properly report TLS errors. Since GT.M V6.3-003 (YottaDB r1.20) TLS related errors from SOCKET devices could result in a segmentation violation (SIG-11). (GTM-9077)

• YottaDB appropriately maintains the exit status for modules with literal arguments to XECUTE containing GOTO, NEW, QUIT, (nested) XECUTE and indirection. In GT.M V6.3-007 (YottaDB r1.28), such XECUTE arguments could cause incorrect $ZCSTATUS from ZCOMPILE and ZLINK, or a non-zero $status in the shell immediately after a MUMPS command performing a compilation. (GTM-9079)

• The compiler reports a warning when an argumentless DO has no subsequent block of the appropriate level; previously it silently ignored such inoperable DO commands. (GTM-9099)

• Execution resumes correctly after out-of-band actions, such as responding to MUPIP INTRPT, <CTRL-C>, and $ZTIMER. Previously when XECUTE'ng an operation that could run a long time (for example, LOCK) with an indirect argument that used an extrinsic, and the code invoked by the extrinsic performed a FOR or one of the set of potentially long commands (JOB, LOCK, MERGE, READ, ZATTACH, ZEDIT, ZPRINT, ZSHOW, ZSYSTEM, ZWRITE, or a format control mnemonic), when resuming from the out-of-band action, the process could terminate abnormally with a segmentation violation (SIG-11) or GTMASSERT2 fatal error. (GTM-9100)

System Administration

• The default first output line of a GO or ZWR format MUPIP EXTRACT command is YottaDB MUPIP EXTRACT followed by the command line used to generate the extract, including the full path to the mupip executable, followed by UTF-8 if the process ran in UTF-8 mode. Previously, the default first line was GT.M MUPIP EXTRACT. [#493]

• The entire text of a string specified for the -label command line parameter for GO and ZWR format MUPIP EXTRACT commands is reproduced on the first line of the extract. Previously only text up to the first space was reproduced. Note that quote characters should not be specified in the label, e.g. -label="hello \"world" is not valid. [#494]

• The -ignorechset command line option of MUPIP LOAD tells MUPIP to load the extract even if it was created by a MUPIP process in another mode (UTF-8 mode vs. M mode). Effective r1.30 (see 493) YottaDB checks that the modes match, reporting a LOADINVCHSET error if they did not. As using -ignorechset bypasses YottaDB checks, use it only if you are sure that the extract file can be loaded correctly. [#569]

• By default, JOB'd processes have the same process name displayed by ps. Previously, the default was mumps. [#592]

• The TLS reference implementation plugin supports the use of OpenSSL 1.1.1, but not TLSv1.3 certificates. The configuration file pointed to by $ydb_crypt_config $gtmcrypt_config or passed on the WRITE /TLS command recognizes SSL_OP_NO_TLSv1_3, in the ssl-options list. The reference encryption plugin currently sets this option by default. Note that even without using TLSv1.3 certificates, current versions of OpenSSL report some error conditions differently than prior versions of OpenSSL. Previously, the differences introduced by the TLSv1.3 certificates preferred by OpenSSL 1.1.1 caused YottaDB to report OpenSSL errors. (GTM-9084)

• YottaDB correctly prints 64-bit sequence numbers in error messages; previously sequence numbers only represented the lower 32 bits of the appropriate value. (GTM-9098)

Other

• Simple API functions such as ydb_lock_s() internally use nanosecond timers. Previously, although the API specified time in nanoseconds, the underlying timer was a millisecond timer. Note that even if timer resolution is in nanoseconds, the accuracy is always determined by the underlying hardware and operating system, as well as factors such as system load. [#388]

• The --posix option of the ydbinstall script downloads and installs the POSIX plugin after first successfully installing YottaDB. Note that this option requires Internet access, and also that gcc already be installed. [#457]

• The --encplugin option of the ydbinstall script downloads and installs the encryption plugin after first successfully installing YottaDB. Note that this option requires Internet access, and also that gcc and supported encryption libraries already be installed, as YottaDB includes no cryptographic software. [#458]

• When YottaDB is initialized by a non-M process using either call-ins, the Simple API, or by calling ydb_init() explicitly, the gtm_dist environment variable is set to $ydb_dist. Previously, it was possible under these circumstances for $gtm_dist to be not set, or to point to a directory other than $ydb_dist. [#470]

• With the environment variable ydb_recompile_newer_src set to 1, t[rue], or y[es], a ZLINK/DO/GOTO/ZBREAK/ZGOTO/ZPRINT/$TEXT recompile the .m file only if it has a newer modification time than the corresponding .o file. The default behavior is for the .m file to be recompiled if its modification time is later than OR equal to that of the corresponding .o file. [#495]

• Object code files generated by YottaDB can be linked into shared libraries by ld version 2.33 and later. Previously ld version 2.33 and later, found on distributions with newer toolchains, such as Ubuntu 19.10 & up, and Arch Linux, would not link YottaDB object code files into shared libraries. [#496]

• The --octo command line option of the ydbinstall script downloads and installs Octo a YottaDB plugin for SQL access to application data in YottaDB global variables. [#521]

Note: Until the production version of Octo is released, this option will download and build Octo from source.

• yottadb -version provides a detailed report on the YottaDB build, e.g.,

  $ yottadb -version
  YottaDB release:         r1.30
  Upstream base version:   GT.M V6.3-008
  Platform:                Linux x86_64
  Build date/time:         2020-07-17 16:58
  Build commit SHA:        5f0e5c9b368c18405e2b4a34dc85f4901cfde9ae
  $ 

  [#595]

• With the --utf8 default option, the ydbinstall script successfully installs the encryption plugin. Previously, when installing the encryption plugin, it failed with an ICUSYMNOTFOUND error. [#532]

• YottaDB defers allocation of transaction processing structures for a particular region until that region is directly part of a TP transaction. This prevents allocation of structures that may not be needed by the application. Previously, YottaDB allocated transaction processing structures for all open regions at the first transaction in any region, and from then on, when opening any new region. (GTM-8911)

• %PEEKBYNAME() accepts the path to the directory where gtmhelp.gld and gtmhelp.dat reside, as an optional fourth parameter gldpath. These files contain the data required for %PEEKBYNAME() to execute properly. If not provided, %PEEKBYNAME() picks up the data from $ydb_dist, the default location. Previously, %PEEKBYNAME() did not accept this option. (GTM-9000)

Note: As GT.M users have reported to YottaDB unwanted syslog messages that seem to be associated with this functionality, we do not recommend using this without extensive testing and/or guidance from YottaDB.

• The auto-upgrade processes performs on database file headers works appropriately. In GT.M V6.3-007 (YottaDB r1.28) due to GTM-9052, it failed to upgrade the flush_trigger_top field, which could require a MUPIP SET -TRIGGER_FLUSH=n if flushing with that version seems inappropriate, where n is a number of dirty buffers at which processes should start flushing more aggressively. (GTM-9082)

• $$IN^%YGBLSTAT(<pid>,<global directory>,<region>) returns a TRUE (1) value if the process is sharing statistics in the region, a FALSE (0) if it is not, and an empty string if the pid is invalid or there is no sharing for a region of the specified name. If region is empty or an asterisk, the extrinsic returns a TRUE if the process is sharing statistics in any region, and a FALSE otherwise. If the global directory is not empty the function attempts to use it, but if it is unavailable the function fails into the invoking environment's specified $ETRAP or $ZTRAP. There are other ways of getting the same kind of indication but they require more code. Previously ^%YGBLSTAT did not provide this extrinsic function. Also, ^%YGBLSTAT returns an empty string for any process that has not shared statistics on any region sought by the invoking arguments. The most interesting class of such processes are probably non-YottaDB processes, but it also includes YottaDB processes that are not sharing. Previously the utility returned an empty string only for nonexistent processes. In addition, ^%YGBLSTAT deals appropriately with the current global directory; previously an explicit or implicit invocation of the STAT extrinsic with no global directory argument inappropriately used the original global directory of the reporting process rather than its current global directory. (GTM-9092)

• The Source Server issues a JNLFILEREAD error when processing corrupted journal files; previously, journal files corrupted by external actions could cause the Source Server to hang indefinitely. (GTM-9097)

• YottaDB issues an error, typically CLISTRTOOLONG or CLIERR, for a command line that exceeds 32KiB; in V6.3-007 (YottaDB r1.28) a regression allowed this condition to cause a segmentation violation (SIG-11), This issue was identified in the development environment and was reported by a user. (GTM-9110)

Error Messages

AIOQUEUESTUCK, Waited mmmm minutes for AIO work queue to complete (cr = rrrr)

All Components Error: A process is taking over mmmm minutes for asynchronous IO (AIO) activity to complete on cache record rrrr. A zero value for rrrr indicates that the process is waiting for all AIO activity to complete.

Action: Check for trouble in the I/O subsystem. The process continues to wait for AIO activity to complete.

BOOLEXPRTOODEEP,Boolean expression depth exceeds maximum supported limit of 2047

Compile / Run Time Error: The nesting depth of a Boolean expression exceeds 2047, the YottaDB limit.

Action: Fix the coding issue. Exceeding the nesting depth limit of 2047 is a pathological error that is most likely to occur in generated code.

BUFSPCDELAY, Request for bbbb blocks in region rrrr delayed

All Components Warning: A process is taking longer than expected to obtain bbbb free database buffers for region rrrr. If bbbb is zero, then the process was attempting to free one particular buffer.

Action: Check for trouble in the I/O subsystem. The process continues its attempt to obtain the free buffer(s).

COMMITWAITSTUCK Pid wwww waited tttt minute(s) for nnnn concurrent YottaDB process(es) to finish commits in database file dddd

Run Time Error: This error message indicates that a process could not finish a database transaction commit and timed out waiting for other concurrent processes to finish. The process will continue to wait.

Action: Check the operator log for accompanying COMMITWAITPID messages. Every concurrent YottaDB process reporting COMMITWAITSTUCK messages would have accompanying COMMITWAITPID message(s). If so, review those messages. If not, report to your YottaDB support channel with system log and operator log information.

DONOTBLOCK, Argumentless DO not followed by a block

Compile Time Warning: This indicates the compiler detected an argumentless DO with no subsequent block with an appropriate level, and optimized it away.

Action: Generally this indicates a coding issue where the block is missing or has the wrong level indication, but it may also be unnecessary - correct as appropriate.

GTMSECSHRPERM, The gtmsecshr module in ydb_dist (DDDD) does not have the correct permission and uid (permission: PPPP, and UID: UUUU)

Run Time Error: This indicates that a client did not start GTMSECSHR, installed in DDDD, because the executable was not owned by root (UUUU is the actual owner) and/or did not have setuid and/or execute permissions (actual permissions are PPPP).

Action: Arrange to provide the GTMSECSHR executable with the proper characteristics. The executable must be SETUID root with execute permissions for the current user.

INDRCOMPFAIL, Compilation of indirection failed

Run Time Error: This indicates that an indirection or XECUTE command failed due to syntax errors.

Action: Review the code and make sure the indirection or XECUTE string has valid syntax and contains no non-graphic characters. Consider using $ZWRITE to identify any such characters.

INVVALUE, VVVV is invalid DEC value for $ZCONVERT(). Range is -9223372036854775808 to 18446744073709551615\ INVVALUE, VVVV is invalid HEX value for $ZCONVERT(). Range is 1 to 16 unsigned hexadecimal digits

Run Time Error: This indicates that the value VVVV is not valid input for $ZCONVERT() in the specified base.

Action: If the input value is expected to be within the range supported by $ZCONVERT(), the actual results from a coding issue or from input that is not validated, correct the issue. If input values are legitimately expected outside the range supported by $ZCONVERT() use the %DH and %HD utility programs.

INVZCONVERT, Translation between XXXX and XXXX is not supported

Run Time Error: This indicates that the base from which a number is to be converted is the same as that to which it is to be coverted. Conversion is not meaningful when the bases are the same.

Action: Review the code, and modify as necessary to ensure that $ZCONVERT() is not called when a number is already in the desired base.

LOADINVCHSET, Extract file CHSET xxx is incompatible with ydb_chset/gtm_chset.

MUPIP Information: This indicates that a MUPIP LOAD operation did not take place because the value of the environment variable ydb_chset or gtm_chset at the time of creating the extract file was not the same as the current value of ydb_chset or gtm_chset.

Action: Determine whether to change the current character set or retry the EXTRACT with a different character set. Alternatively, you can edit the extract file so the EXTRACT file header matches the ydb_chset environment variable. This enables an M mode MUPIP LOAD to treat the input as a byte stream or a UTF-8 mode MUPIP LOAD, which either detects BADCHAR errors or not, depending on the setting of the ydb_badchar environment variable.

NULSUBSC, XXXX Null subscripts are not allowed for database file: YYYY

Run Time/MUPIP Error: This indicates that a global variable specified a null subscript in a database file YYYY, which does not accept null subscripts. The leading context (XXXX) identifies more about the event or the location of the issue.

Action: Look for the source of the null subscript(s) and consider whether they are appropriate or due to a coding error. If they are appropriate, use MUPIP SET -NULL_SUBSCRIPTS, and remember to make the same adjustment with GDE CHANGE REGION -NULL_SUBSCRIPTS to ensure the next time you recreate a database that the characteristic persists.

REPLJNLCLOSED, Replication in jeopardy as journaling got closed for database file dddd. Current region seqno is xxxx[XXXX] and system seqno is yyyy[YYYY]

Run Time Warning: This message indicates that YottaDB turned OFF journaling and switched replication from ON to WAS_ON on the specified database. Other preceding messages identify the cause (for example, lack of disk space while writing to journal file, permissions issue while auto-switching to new journal files, and so on). The message also displays the region (xxxx decimal and XXXX hexadecimal) and journal (yyyy/YYYY) sequence numbers. From this point, replicating updates on the primary to the secondary might, or might not, work depending on the backlog on the primary until replication/journaling gets turned back ON.

Action: First, correct the cause (lack of disk space, permission issues, and so on) that turned journaling OFF.

Execute the MUPIP SET REPLICATION=ON or MUPIP BACKUP REPLICATION=ON command to turn replication (and journaling) ON and switch to a new set of journal files. This command can work while processes are concurrently updating the database and causes YottaDB to journal subsequent updates in both the journal file and journal pool (rather than only in the journal pool as it does when replication is in the WAS_ON state).

Execute the MUPIP REPLIC -SOURCE -SHOWBACKLOG command. Note down the value of "sequence number of last transaction written to journal pool".

Execute the above command at regular intervals and note down the value of "sequence number of last transaction sent by source server."

If the "sequence number of last transaction sent by source server" is greater than "sequence number of last transaction written to journal pool", it means that the source server successfully sent all journal records during the time interval when journaling was turned OFF. In this case, no further action is required.

On the other hand, if the "sequence number of last transaction sent by source server" is less than "sequence number of last transaction written to journal pool" and reports the same value across repeated SHOWBACKLOG commands, then check the source server log file for any error messages - most likely a NOPREVLINK error from the source server. This means the source server could not locate the corresponding journal records required from the journal files to replicate a particular sequence number and therefore, it failed to synchronize the primary and secondary. In this case, take an online backup of the primary, restore it on the secondary and start the secondary with the UPDATERESYNC qualifier to synchronize the secondary with the primary.

ZYSQLNULLNOTVALID, $ZYSQLNULL cannot be used as an integer, numeric, gvn subscript/value or lock subscript

Compile / Run Time Error: $ZYSQLNULL can only be used in expressions, or as the value of a local variable subscript or node, and is used incorrection.

Action: Correct the coding issue resulting in the incorrect $ZYSQLNULL usage.

Legal Stuff

Copyright © 2020 YottaDB LLC

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts and no Back-Cover Texts.

YottaDB® and Octo® are registered trademarks of YottaDB LLC. GT.M™ is a trademark of Fidelity National Information Services, Inc. Other trademarks belong to their respective owners.

This document contains a description of YottaDB and the operating instructions pertaining to the various functions that comprise the software. This document does not contain any commitment of YottaDB LLC. YottaDB LLC believes the information in this publication is accurate as of its publication date; such information is subject to change without notice. YottaDB LLC is not responsible for any errors or defects.

YottaDB r1.28

Binary Distributions

sha256sum	file
3f1968f882e6f54f89a12b3eb8a4bbf6d769e5fe46c69272fcddc93cb0e102d3	yottadb_r128_debian10_x8664_pro.tgz
6a44f544dfa3b753bee7f13480699cc5b6f5aa108af06f99d872bd6f50c9f078	yottadb_r128_linux_aarch64_pro.tgz
4c04380fd35f15a1fc1669e9d0519d770b834f2335db4a3b2250611bf99f43ef	yottadb_r128_linux_armv6l_pro.tgz
6c8e61c37d63eae071f2e8a2d57f232237dff7f9e19f286c125e4faab7b01243	yottadb_r128_linux_armv7l_pro.tgz
f5df150d7659fc4b050b89cbc2e9f6a01e117ed1b5e1ccb133b9ebb70d9d40d3	yottadb_r128_linux_x8664_pro.tgz
6200c81f349cf9c4e7ee5120530248bcdffff611621416d57b3cd8f9693c29c2	yottadb_r128_rhel7_x8664_pro.tgz
22e8289e136c165b68b6c9657dcc9702a1d223587c36ae8874ceb56f7b7ddf20	yottadb_r128_centos8_x8664_pro.tgz

Release Note Revision History

Revision	Date	Summary
1.00	September 11, 2019	r1.28 Initial Release
1.01	March 31, 2020	Added CentOS 8 Build

Contact Information

YottaDB LLC

40 Lloyd Avenue, Suite 104 Malvern, PA 19355, USA [email protected] +1 (610) 644-1898

Support

Customers

Contact your YottaDB support channel.

Others

For free (to you) support from members of communities who run widely available applications on YottaDB, please use an application-specific list where appropriate.

• For requests other than to the communities below, post an Issue at https://gitlab.com/YottaDB/DB/YDB/issues and include the words "Help Wanted" in the summary.

• For requests specific to the use of YottaDB from node.js via nodem, QewdJS or Enterprise Web Developer, post to the Enterprise Web Developer community.

• For requests specific to the use of YottaDB with VistA flavors, post to the Hardhats list.

• For requests specific to the use of YottaDB with M other than for applications above, post to the comp.lang.mumps list.

r1.28

Overview

For users of YottaDB using the Go API, r1.28 is a major release because it is required for production grade access to YottaDB from Go. For others, it is a minor release with a small set of enhancements anf fixes as detailed in the complete release notes.

YottadB r1.28 is upward compatible with YottaDB r1.26, and thus upward compatible with GT.M V6.3-006, and GT.M V6.3-007.

Platforms

A platform is a combination of a CPU architecture and an operating system. A platform is Supported, Supportable, or Unsupported. Supported means that we have the platform in our development environment and test each release on that platform. Supportable means that although we do not necessarily have such a platform in our environment, we have no reason to believe that the software will not run on it. All others are Unsupported.

CPU Architecture	Supported OS Version(s)	Notes
64-bit x86	Ubuntu 18.04 LTS; Red Hat Enterprise Linux 7.6; Debian GNU/Linux 10 (Buster)	There are separate binary distributions for each OS version, owing to differences in library versions of those distributions.
64-bit ARM (Raspberry Pi 3 Model B)	Ubuntu 18.04 LTS	While YottaDB is Supportable on other ARMv8-A CPUs, owing to variations in the implementations of ARM microarchitectures, we recommend that you ensure the software runs correctly before committing to any specific hardware other than those Supported. Please contact [email protected] if you want a specific combination of OS and CPU microarchitecture to be Supported.
32-bit ARM (Raspberry Pi 3 Model B)	Raspbian GNU/Linux 10 (Buster)	While YottaDB is Supportable on other ARMv7-A CPUs, owing to variations in the implementations of ARM microarchitectures, we recommend that you ensure the software runs correctly before committing to any specific hardware other than those Supported. Please contact [email protected] if you want a specific combination of OS and CPU microarchitecture to be Supported.
32-bit ARM (Raspberry Pi Zero)	Raspbian GNU/Linux 9.1	While YottaDB is Supportable on other ARMv6 CPUs, owing to variations in the implementations of ARM microarchitectures, we recommend that you ensure the software runs correctly before committing to any specific hardware other than those Supported. Please contact [email protected] if you want a specific combination of OS and CPU microarchitecture to be Supported.

Recent releases of major GNU/Linux distributions with contemporary kernels, glibc and ncurses are Supportable. Specific notes:

• On Ubuntu releases after 18.04 LTS, YottaDB needs the libtinfo5 package to be installed.
• On Arch Linux and other leading edge distributions, YottaDB may need to be recompiled from source code owing to library and tool chain versions significantly newer than those used in building the distribution.

Installation

See our Get Started page to use YottaDB.

We strongly recommend that you install YottaDB r1.28 in a newly created directory, different from those of YottaDB r1.26 and any GT.M versions you may have installed on the system.

Removing an installed YottaDB release

Assuming $ydb_dist points to the directory where YottaDB is installed:

• Cleanly shut down all application processes using that release.
• Execute mupip rundown && mupip rundown -relinkctl
• Ensure that there are no gtcm* or gtmsecshr processes active.
• Use sudo lsof | grep $ydb_dist to ensure there are no open files.
• Delete the directory with sudo rm -rf $ydb_dist

Upgrading to YottaDB r1.28

As YottaDB r1.28 is upward compatible from YottaDB r1.26, GT.M V6.3-006 and GT.M V6.3-007, the minimal upgrade steps are:

• Install YottaDB r1.28.
• Install plugins you use.
• Recompile object code, and recreate shared libraries where appropriate.
• If you are using encryption, compile and install the reference implementation plugin or your customized plugin.
• Cleanly shut down the application and ensure that the database files are shut down using mupip rundown from the prior release.
• Switch journal files with the new YottaDB release.
• Start using the new YottaDB release.

If the database has triggers defined with non-canonical numbers, or numbers specified as strings with any version prior to r1.26, or if you are unsure, extract the trigger definitions, delete existing triggers, and reload the trigger definitions. Issue [#430] from r1.26 has a series of steps you can copy and execute.

To upgrade from older GT.M releases, first upgrade to GT.M V6.0-000 or later and follow the steps above, or contact your YottaDB support channel for assistance and guidance.

A more sophisticated upgrade technique is:

• Install YottaDB r1.28.
• Create a new replicated instance of your application (on the same system or a different system).
• Assuming the existing instance is A, and the new instance is B, upgrade B to r1.28 and start replicating from A to B.
• Once B catches up, switchover so that B is in a primary role replicating to A.
• Once you are satisfied with B, remove (or upgrade) A.

Change History

r1.28

YottaDB r1.28 includes the following changes from YottaDB r1.26.

ID	Category	Summary
(#469)	Language	$FNUMBER() formatting code "." facilitates number formatting
(#471)	Other	Sourcing ydb_env_set and ydb_env_unset provides more information in the event of an error
(#472)	Language	ydb_data_s()/ydb_data_st() return YDB_DATA_ERROR in case of error
(#475)	Language	ydb_lock_incr_s(), ydb_lock_decr_s(), and ydb_lock_s() check for too many subscripts
(#477)	Language	NEW accepts $TEST as an argument
(#480)	Language	Incrementing a variable whose initial value is zero or undefined with a seven or more digit increment returns the correct string value

Database

Language

• To facilitate application internationalization, $FNUMBER() accepts the following formatting code (i.e., as the second argument):

  • ".": inserts periods (".") every third position to the left of the decimal within the number and uses a comma (",") as the decimal separator.

  [#469]

• When ydb_data_s() or ydb_data_st() get an error, the return value in *ret_value is set to YDB_DATA_ERROR. Previously the return value was zero which is a valid return value. [#472]

• ydb_lock_incr_s(), ydb_lock_decr_s(), and ydb_lock_s() validate that the number of subscripts provided does not exceed the maximum. Previously specifying too many subscripts resulted in uninformative assert failures and buffer overflows. [#475]

• The NEW command accepts $TEST as an argument. Previously, it did not. YottaDB would like to thank SP.ARM for contributing this enhancement. [#477]

• Using $INCREMENT(), ydb_incr_s(), or ydb_incr_st() to increment a variable whose initial value is zero or undefined with an increment of seven digits or more returns the correct string value. Previously, while the numeric value of the result was correct, the string value was the empty string (""). Depending on how the result was used, it could have been wrong. [#480]

System Administration

Other

• With the %YDBENV program (invoked while sourcing the ydb_env_set and ydb_env_unset files), the error output includes the output of ZSHOW "*" in addition to the single line error message. Previously, it was only the single line error message, which provided less information when troubleshooting. [#471]

More Information

Error Messages

Legal Stuff

Copyright © 2019 YottaDB LLC

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts and no Back-Cover Texts.

YottaDB® and Octo® are registered trademarks of YottaDB LLC. GT.M™ is a trademark of Fidelity National Information Services, Inc. Other trademarks belong to their respective owners.

This document contains a description of YottaDB and the operating instructions pertaining to the various functions that comprise the software. This document does not contain any commitment of YottaDB LLC. YottaDB LLC believes the information in this publication is accurate as of its publication date; such information is subject to change without notice. YottaDB LLC is not responsible for any errors or defects.

YottaDB r1.26

Binary Distributions

sha256sum	file
b32032401ff252cbd673b91cfa10a825ead02925397e84ff0c643570af83ac10	yottadb_r126_debian10_x8664_pro.tgz
8a58a48530eb75600da5cb9f5ae7148347b546dc4d3469f6d6647e0c258612a0	yottadb_r126_linux_aarch64_pro.tgz
92f57e33e956dfa57b3fc1737186a4516dc484deacdf7099a98779cfed31a5b7	yottadb_r126_linux_armv6l_pro.tgz
ea0fd2a281e72de204870cbcd6f2c84fca51067d7fa57221e3c2380fad8ef282	yottadb_r126_linux_armv7l_pro.tgz
71c128db3ce84b94151af61bcd1a62e28e1d5a0306dec1f0c1862c2acb98e232	yottadb_r126_linux_x8664_pro.tgz
20984e82613307589200135d2a58b26667cc1fa7170d3e6732d27ea2fbddeb73	yottadb_r126_rhel7_x8664_pro.tgz

Release Note Revision History

Revision	Date	Summary
1.00	June 27, 2019	r1.26 Initial Release

Contact Information

YottaDB LLC

40 Lloyd Avenue, Suite 104

Malvern, PA 19355, USA

[email protected]

+1 (610) 644-1898

Support

Customers

Contact your YottaDB support channel.

Others

For free (to you) support from members of communities who run widely available applications on YottaDB, please use an application-specific list where appropriate.

• For issues specific to the use of YottaDB from node.js via nodem, QewdJS or Enterprise Web Developer, post to the Enterprise Web Developer community.

• For issues specific to the use of YottaDB with VistA flavors, post to the Hardhats list.

• For issues specific to the use of YottaDB with M other than for applications above, post to the comp.lang.mumps list.

• If you are not sure where to post, or for requests other than to the above communities, post an issue at https://gitlab.com/YottaDB/DB/YDB/issues and include the words "help wanted" in the summary.

r1.26

Overview

r1.26 is a major release that brings important new functionality to YottaDB.

Functions of the C Simple API to support multi-threaded applications, which were considered field test grade in r1.24 are considered production grade and fully Supported in r1.26. Their performance is also now comparable to functions for single-threaded applications.

There are numerous other enhancements, including:

• Sourcing ydb_env_set addresses an expanded set of use cases, including setting up and automatically recovering a database (e.g., when coming up after a crash, for example). This brings to applications in all languages functionality that was previously available to M applications through the ydb script in a more limited form. ydb_env_set also automatically sets environment variables to access YottaDB plugins that comply with the plugin architecture standard.

• More seamless integration between C and M code.

• Debian GNU/Linux 10 (Buster) on x86_64 is a Supported platform.

• The $ZTIMEOUT intrinsic special variable allows an application to create and manage software watchdog timers.

As with any YottaDB release, there are numerous additional enhancements for functionality, performance, usability, and robustness, as well as fixes. These are detailed in the complete release notes.

YottadB r1.26 is upward compatible with both YottaDB r1.24, GT.M V6.3-006, and GT.M V6.3-007.

Platforms

A platform is a combination of a CPU architecture and an operating system. A platform is Supported, Supportable, or Unsupported. Supported means that we have the platform in our development environment and test each release on that platform. Supportable means that although we do not necessarily have such a platform in our environment, we have no reason to believe that the software will not run on it. All others are Unsupported.

CPU Architecture	Supported OS Version(s)	Notes
64-bit x86	Ubuntu 18.04 LTS; Red Hat Enterprise Linux 7.6; Debian GNU/Linux 10 (Buster)	There are separate binary distributions for each OS version , owing to differences in library versions of those distributions.
64-bit ARM (Raspberry Pi 3 Model B)	Ubuntu 18.04 LTS	While YottaDB is Supportable on other ARMv8-A CPUs, owing to variations in the implementations of ARM microarchitectures, we recommend that you ensure the software runs correctly before committing to any specific hardware other than those Supported. Please contact [email protected] if you want a specific combination of OS and CPU microarchitecture to be Supported.
32-bit ARM (Raspberry Pi Zero)	Raspbian GNU/Linux 9.1	While YottaDB is Supportable on other ARMv6 CPUs, owing to variations in the implementations of ARM microarchitectures, we recommend that you ensure the software runs correctly before committing to any specific hardware other than those Supported. Please contact [email protected] if you want a specific combination of OS and CPU microarchitecture to be Supported.

Recent releases of major GNU/Linux distributions with contemporary kernels, glibc and ncurses are Supportable. Specific notes:

• On Ubuntu releases after 18.04 LTS, YottaDB needs the libtinfo5 package to be installed.
• On Arch Linux) and other leading edge distributions, YottaDB may need to be recompiled from source code owing to library and tool chain versions significantly more recent than those used in building the distribution.

Installation

See our Get Started page to use YottaDB.

We strongly recommend that you install YottaDB r1.26 in a newly created directory, different from those of YottaDB r1.24 and any GT.M versions you may have installed on the system.

Removing an installed YottaDB release

Assuming $ydb_dist points to the directory where YottaDB is installed:

• Cleanly shut down all application processes using that release.
• Execute mupip rundown && mupip rundown -relinkctl.
• Ensure that there are no gtcm* or gtmsecshr processes active.
• Use sudo lsof | grep $ydb_dist to ensure there are no open files.
• Delete the directory with sudo rm -rf $ydb_dist.

Upgrading to YottaDB r1.26

As YottaDB r1.26 is upward compatible from YottaDB r1.24, GT.M V6.3-006 and GT.M V6.3-007, the minimal upgrade steps are:

• Install YottaDB r1.26.
• Recompile any object code, and recreate shared libraries where appropriate.
• If you are using encryption, compile and install the reference implementation plugin or your customized plugin.
• Cleanly shut down the application and ensure that the database files are shut down using mupip rundown from the prior release.
• Switch journal files with the new YottaDB release.
• Start using the new YottaDB release.

If the database has triggers defined with non-canonical numbers, or numbers specified as strings (Issue [#430]), or if you are unsure, you should extract the trigger definitions, delete existing triggers, and reload the trigger definitions. The Issue [#430] has a series of steps you can copy and execute.

To upgrade from older GT.M releases, first upgrade to GT.M V6.0-000 or later and follow the steps above, or contact your YottaDB support channel for assistance and guidance.

A more sophisticated upgrade technique is:

• Install YottaDB r1.26.
• Create a new replicated instance of your application (on the same system or a different system).
• Assuming the existing instance is A, and the new instance is B, upgrade B to r1.26 and start replicating from A to B.
• Once B catches up, switchover so that B is in a primary role replicating to A.
• Once you are satisfied with B, remove (or upgrade) A.

Change History

r1.26

YottaDB r1.26 includes the following changes from YottaDB r1.24.

ID	Category	Summary
(#111)	DB	MUPIP JOURNAL ROLLBACK and MUPIP JOURNAL RECOVER BACKWARD correctly rundown database files after abnormal process terminations
(#370)	Language	Provide a mechanism to set the C→M call-in table
(#382)	Other	Clear locks abandoned by processes terminating abnormally
(#405)	Other	Debian 10 is a Supported platform
(#416)	Admin	ydb_env_set in M mode when $ydb_dist is a symbolic link sets ydb_dist to the fully resolved real path
(#417)	Language	ydb_subscript_next_s(), ydb_subscript_next_st(), ydb_subscript_previous_s(), and ydb_subscript_previous_st() return YDB_ERR_NODEEND when there is no successor / predecessor subscripted local variable node
(#418)	Language	ydb_file_id_free()/ydb_file_is_identical()/ydb_file_name_to_id() and their _t variants should issue PARAMINVALID error if input filename/fileid pointer is NULL
(#419)	DB	Source Server processes do not help flush journal buffers for a frozen instance
(#420)	Language	Performance of Simple API functions for multi-threaded applications comparable to those for single-threaded applications; both faster
(#421)	Admin	Automatically manage environment variables for plugins that meet the standard
(#423)	DB	On AARCH64/ARM64 platform, MUPIP INTRPT works correctly
(#424)	Language	$ZTRIGGER("ITEM",...) works correctly in workload with significant concurrent updates
(#425)	Language	ydb_message() and ydb_message_t() return YDB_ERR_PARMINVALID when second parameter is NULL
(#428)	Language	ydb_timer_start() timer_id parameter type is intptr_t
(#429)	Admin	ydb_env_set and ydb_env_unset handle an expanded set of "out of the box" use cases
(#430)	DB	$ZTRIGGER() and MUPIP TRIGGER accept subscripts with decimal points and treat as identical different descriptions of the same subscript
(#431)	Language	LVUNDEF error in SimpleAPI reports variable name; also string subscripts are appropriately quoted
(#434)	Language	ydb_exit() reports INVDBEXIT error when called from C code invoked from M code
(#435)	Admin	source command using ydbinstall script leaves user in deleted directory
(#440)	Language	Line in a routine with XECUTE of a literal works correctly
(#446)	Language	Call-ins where the return value is a string checked for overflow
(#447)	Language	*ret_subs_used is 0 when returning YDB_ERR_NODEEND
(#449)	Language	$order(gvn,-1), $zprevious(gvn), and $query(gvn,-1) work correctly with concurrent use of $increment()
(#450)	Language	A large number of calls to $QUERY(lvn,-1) does not lead to a STACKCRIT error
(#452)	Language	M routines returning no value and passed strings by invoking C code return execute correctly
(#453)	Other	yottadb is the main executable to which mumps is a symbolic link
(#454)	Other	Default value of ydb_routines if not set on yottadb/mumps process startup
(#455)	Other	ydbcrypt_interface.h and ydb_tls_interface.h available $ydb_dist
(#456)	Language	ZWRITE of a global issues DBFILERR when specified database file does not exist
(#460)	Language	Maximum M source code line length is 32,766 bytes
(#462)	Admin	MUPIP STOP of processes with top level code written in a language other than M terminates processes cleanly
(#463)	Language	Subsequent calls to YottaDB from threads spawned by a TP callback function work correctly
(#464)	DB	SIGINT (Ctrl-C) of an application whose top level is a language other than M terminates the process at the earliest safe point
(#467)	Language	Processes whose top level is a language other than M report a GTMSECSHRPERM

Database

• MUPIP JOURNAL ROLLBACK and MUPIP JOURNAL RECOVER BACKWARD correctly rundown database files after abnormal process terminations (e.g., resulting from a kill -9, or an OS out-of-memory process termination) that leave shared memory segments in place. Previously, there was a small window of a few instructions in the terminated process that could cause the subsequent MUPIP JOURNAL process to hang indefinitely. Note that YottaDB strongly recommends against using kill -9 to terminate processes, and to ensure adequate system resources. ((#111))

• During an instance freeze (indicated by mupip replicate -source -freeze), Source Server processes do not flush journal buffers to journal files, as they do when an instance freeze is not on. Previously, Source Servers could attempt to flush journal buffers, and hang until the freeze was released, which in turn could result in a replication backlog not being cleared during that time.((#419))

• On the AARCH64/ARM64 platform, MUPIP INTRPT works correctly. Previously, it was possible for the M program receiving the interrupt to issue random errors. ((#423))

• $ZTRIGGER() and MUPIP TRIGGER load triggers for numeric subscripts with a decimal point. Previously, they raised parse errors that "." was an invalid character in the subscript. Additionally, trigger definitions for subscripted global variable references are better checked for equality. For example, trigger definitions for ^x(2), ^x(2.0) and ^x("2") are treated the same, as all 3 specifications map to the same node in the database file. Previously such specifications caused multiple triggers to be created, resulting in a single update of ^x(2) invoking multiple triggers, a potentially unintended consequence.

  If you suspect that an existing application database has triggers defined with non-canonical numbers, numbers specified as strings or if you are not sure, extract, delete, and reload all triggers after upgrading to r1.26. For example:

  mupip trigger -select /tmp/triggers.define # extract the current trigger definitions
  echo "-*" >/tmp/triggers.delete # create trigger file to delete all triggers
  mupip trigger -triggerfile=/tmp/triggers.delete # delete triggers
  mupip trigger -triggerfile=/tmp/triggers.define # reload trigger definitions

  ((#430))

• A SIGINT (Ctrl-C) of an application whose top level is a language other than M, terminates the process at the earliest safe point. Previously, a premature process termination at an unsafe point could result in structural damage to database files being updated by the process. ((#464))

Language

• The Simple API Utility Functions ydb_ci_tab_open() / ydb_ci_tab_open_t() and ydb_ci_switch() / ydb_ci_switch_t() provide a mechanism for a process to switch between different call-in tables. This allows for more modular applications by allowing each plugin or library to have its own call-in table; previously, a process had a single call-in table. The call-in table specified by the environment variable ydb_ci is available at process startup without the need to explicitly open it (i.e., its behavior is unchanged). ((#370))

• ydb_subscript_next_s(), ydb_subscript_next_st(), ydb_subscript_previous_s(), and ydb_subscript_previous_st() return YDB_ERR_NODEEND and leave *ret_value unmodified when provided with a subscripted local variable as input, and there is no successor or predecessor node. In YottaDB r1.24, the functions could sometimes return YDB_ERR_LVUNDEF incorrectly or YDB_OK, and could set *ret_value to the empty string.((#417))

• ydb_file_id_free()/ydb_file_id_free_t(), ydb_file_is_identical()/ydb_file_is_identical_t(), and ydb_file_name_to_id()/ydb_file_name_to_id_t() issue a PARAMINVALID error in case the input file name or file id parameter is NULL. Previously, they could cause the process to terminate abnormally with a SIG-11 if the file id parameter was NULL, and returned YDB_OK in case the file name parameter was NULL. ((#418))

• The performance of Simple API functions to support multi-threaded applications (those whose names end in _st e.g,. ydb_get_st()) is comparable to those that support single-threaded applications. Previously, they were markedly (on the order of a half order of magnitude) slower. Functions to support single-threaded applications are also faster. ((#420))

• $ZTRIGGER("ITEM",expr2) works correctly in a workload with significant concurrent updates to database triggers from multiple processes. Previously, it could incorrectly return a parse error in the input string expr2 even if no such error existed. ((#424))

• If the *msg_buff parameter is NULL, ydb_message() and ydb_message_t() return YDB_ERR_PARAMINVALID. Previously, this would result in process termination with a segmentation fault. Additionally, YDB_ERR_INVSTRLEN errors returned by ydb_message() and ydb_message_t() result in a subsequent ydb_status() returning the full YDB-E-INVSTRLEN error string. Previously, it returned an additional SYSTEM-W-UNKNOWN string. ((#425))

• The timer_id parameter of ydb_timer_start() is of type intptr_t. Previously, this was of type int which was inadequate for a 64-bit platform. ((#428))

• A YDB_ERR_LVUNDEF return status from ydb_get_s()/ydb_get_st() records the non-existent local variable name and any specified subscripts in the error string obtained by a subsequent ydb_zstatus() call. Previously ydb_zstatus() returned an error string that did not contain the local variable name but contained the subscripts. Additionally, string subscripts are reported within double-quotes. Previously they were reported without the surrounding double-quotes. ((#431))

• ydb_exit() returns with an error YDB_ERR_INVYDBEXIT when called from C code was invoked from M code. Previously, this abnormally terminated the process with a SIG-11. ((#434))

• Routine lines with an XECUTE of a literal string followed by one or more commands (i.e., the XECUTE is not the last command in the line) work correctly. Previously, an optimization of XECUTE of literal strings in routines caused such lines to execute incorrectly. For example, the line W !,"start" X "F J=1:1:0 W !,J" W !,"stop" in a routine works correctly, whereas previously it would print "start" but not "stop". The workaround was to assign the literal to a variable. Direct mode was unaffected by this issue. ((#440))

• Call-ins where the return value is a string are checked for overflow (where possible) and return an INVSTRLEN error if the return area is not large enough, where previously it did not. Note that for string parameters, use of the ydb_string_t type is highly recommended as it facilitates checking for buffer overflows. A char * type does not facilitate such checks and is best avoided. ((#446))

• The parameter *ret_subs_used, which is intended to return results, is set to zero (0) when a call to ydb_node_next_s(), ydb_node_next_st(), ydb_node_prev_s(), or ydb_node_prev_st() has a return code of YDB_ERR_NODEEND. Previously, this value remained untouched under this case. ((#447))

• $order(gvn,-1), $zprevious(gvn), and $query(gvn,-1) work correctly with $increment() used by concurrent processes. Previously, it was possible in such an environment for processes to fail on rare occasions with a SIG-11 or to return incorrect results. ((#449))

• A sequence of a large number of calls to $QUERY(lvn,-1) does not lead to a STACKCRIT error. Previously, it could. ((#450))

• M routines that are called from C code with string (ydb_string_t * type) parameters and which return no value (i.e., the call-in table has a void return type) execute correctly. Previously, $[Z]PIECE() and $[Z]LENGTH() functions using these parameters could occasionally return incorrect results, resulting in incorrect execution of the M routines. ((#452))

• When preceded by a name level $ORDER(), $ZPREVIOUS(), or DO ^%GD, ZWRITE of a global issues a DBFILERR error when the database file for the global does not exist. Previously, it was possible for this to abnormally terminate the process with a KILLBYSIGSINFO1. ((#456))

• The maximum length for lines of M source code, and the maximum length of a single line of output when ZSHOW directs its output to a local variable, is 32,766 bytes. Previously, it was 8192 bytes. ((#460))

• Subsequent calls to YottaDB (e.g., ydb_set_st()) from threads spawned by a TP callback function (invoked from ydb_tp_st()) work correctly. Previously, this use case could cause the process to terminate abnormally with a SIG-11. ((#463))

• Processes whose top level is a language other than M report a GTMSECSHRPERM error if $ydb_dist/gtmsecshr is not owned by root or does not have the setuid bit set. Previously, such calls terminated the process abnormally with a SIG-11. ((#467))

System Administration

• Sourcing $ydb_dist/ydb_env_set in M mode sets the ydb_dist environment variable to the fully resolved real path when $ydb_dist is a symbolic link, bringing M mode behavior in line with that of UTF-8 mode. Previously, it left that environment variable unaltered in M mode. ((#416))

• The ydb_env_set file when sourced also defines environment variables ydb_xc_* / GTMXC_* for $ydb_dist/plugin/*.xc files with M to C call-out tables. Previously, ydb_env_set ignored the call-out table files. ((#421))

• Under the directory specified by $ydb_dir (defaulting to $HOME/.yottadb), the ydb_env_set file when sourced:

  • Ensures a standard environment for YottaDB, creating one should an environment not exist.
  • Creates new database files for any regions that do not have database files, under the assumption that missing database files correspond to temporary regions with scratch globals.
  • If the database was not shut down cleanly (e.g., when a system is rebooted after a crash), recovers the database using MUPIP JOURNAL RECOVER BACKWARD if the database does not have replication turned on and MUPIP JOURNAL ROLLBACK BACKWARD if replication is turned on. All database regions to be recovered must have before image journaling enabled and on at the time of the crash.
  • Sets reasonable values for ydb_* environment variables and their gtm* counterparts (the latter so that application code and scripts that query the latter continue to work correctly).
  • Defines environment variables ydb_xc_* / GTMXC_* for $ydb_dist/plugin/*.xc files with M to C call-out tables.
  • Adds $ydb_dist to $PATH, as well as $ydb_dist/plugin/bin if it exists.
  • Sourcing ydb_env_set saves environment variables that it sets so that sourcing ydb_env_unset subsequently restores those variables to their prior values.

  Previously:

  • Sourcing ydb_env_set created an environment if one did not exist, and provided values to environment variables that were not set, letting the ydb script perform any needed recovery. However, that only served those using YottaDB using the M language, since those using YottaDB using other Supported languages would not use the ydb script.
  • Environment variables had to be explicitly added for M-to-C call-out tables in $ydb_dist/plugin/*.xc files.
  • Sourcing ydb_env_set created aliases for mupip and mumps (gde continues to be an alias).

  While the behavior of the combination of ydb_env_set and ydb_env_unset is not upward compatible (for example, it sets ydb_routines and gtmroutines to the environment under $ydb_dir where previously it made consistent existing values of $ydb_routines and $gtmroutines), they are intended to be upward compatible for common "out of the box" usage; with the expectation that more complex scenarios would use application-specific scripting. In addition to improved support for non-M users, they handle a larger number of situations that previously would have required application-specific scripting. ((#429))

• Sourcing the ydbinstall / ydbinstall.sh script reports an error. Previously, it completed the installation, but then left the shell in a directory that had been deleted. Note that the file is designed to be run as its own process, not sourced by an existing shell process. ((#435))

• MUPIP STOP of processes with top level code written in a language other than M terminates processes cleanly. Previously, a CALLINTROLLBACK error could be issued if the process was in a transaction (e.g., C functions ydb_tp_s() / ydb_tp_st()) or in called M code (e.g., using C functions ydb_ci(), ydb_cip(), ydb_ci_t() and ydb_cip_t()). ((#462))

Other

• The LKE CLNUP command clears the lock space of locks abandoned by processes that exited abnormally. In addition to the standard -region and -all flags, the command has two optional command line flags:

  • While the command by default runs once and terminates, the -periodic=n qualifier instructs LKE CLNUP to run in a loop, performing a cleanup every n seconds, a lighter weight operation than invoking the LKE command every n seconds from a shell script.
  • The -integ option instructs the command to also validate the data structures in the lock space for structural integrity.

  Note that processes release their locks on normal exit. Also, there is technically no need to run LKE CLNUP because a process whose attempt to acquire a lock is blocked by an abandoned lock will clean up the blocking abandoned locks. ((#382))

• Debian 10 (Buster) on x86_64 is a Supported platform for YottaDB. Previously, it was Supportable, requiring manual installation or use of the --force-install flag for the ydbinstall script. This was previously reported as Issue 354 on GitHub. ((#405))

• The main executable of YottaDB is called yottadb to which mumps is a symbolic link. Either name can therefore be used. ((#453))

• If the environment variable ydb_routines is not set when the yottadb / mumps process starts, or if it is set to the empty string (""), YottaDB sets it in the environment to $ydb_dist/libyottadbutil.so if it exists, and to $ydb_dist if it does not, and then uses that value. Previously, it used a value of ".". While this is not backward compatible, the new behavior better protects against inadvertent execution of the wrong routine. Please let us know if this change to an edge case affects you. ((#454))

• To facilitate the compilation of YottaDB plugins, the files ydbcrypt_interface.h and ydb_tls_interface.h are made available both in $ydb_dist as well as in the tarball of the reference implementation of the encryption plugin at $ydb_dist/plugin/source.tar, with symbolic links gtmcrypt_interface.h and gtm_tls_interface.h respectively. Previously the files, named gtmcrypt_interface.h and gtm_tls_interface.h, were only provided in the tarball of the reference implementation of the encryption plugin. ((#455))

GT.M V6.3006

ID	Category	Summary
(GTM-4263)	Language	MUMPS commands accept prompted names in response to "What file: "
(GTM-6135)	Language	$ZTIMEOUT manages a process wide timed interrupt
(GTM-7952)	Language	Improve memory and signal management for external calls
(GTM-8017)	Other	^%TRIM accepts characters other than and to trim
(GTM-8178)	Language	Normalize YottaDB compiler invocations from ZCOMPILE, ZLINK, auto-ZLINK and the MUMPS command
(GTM-8518)	Admin	MUPIP REPLICATE -EDITINSTANCE requires standalone access and supports -CLEANSLOTS
(GTM-8933)	DB	YottaDB limits the number of errors from processes attempting to open a statsDB
(GTM-8947)	Language	Performance enhancement for $TRANSLATE() when arguments two and three are literals
(GTM-8993)	DB	A process that fails to open a statsDB does not establish the location it used
(GTM-8998)	Language	External calls can return all available types
(GTM-9005)	Admin	Appropriate exit status from MUPIP LOAD
(GTM-9011)	Admin	MUPIP SET accepts -KEY_SIZE or -RESERVED_BYTES in the same command as -RECORD_SIZE
(GTM-9017)	Other	Prevent segmentation violation when invoking $ydb_procstuckexec
(GTM-9024)	DB	Improve LOWSPC reporting
(GTM-9025)	Other	Restore conversion performance in percent routines for smaller numbers
(GTM-9031)	Other	Update Cmake build scripts to be compatible with current Cmake releases
(GTM-9038)	Other	YottaDB excludes TLS 1.0/1.1 unless configured to use older SSL/TLS protocols

Database

• If misconfigured processes get the same error when opening a statsDB, YottaDB throttles the messages such that every hundredth message goes to the operator log; previously every process reported the issue. (GTM-8933 )

• A process which cannot open a statsDB disables itself from maintaining the shared statistics, but does not disable subsequently starting processes, which better enables changes to an incorrectly configured environment. Previously, an initializing process that could not access a statsDB could also effectively require all processes using that database to restart in order to enable statistics sharing for the region. (GTM-8993)

• When the database reaches the 88% size threshold, and for every 1% increase in size and beyond, YottaDB reports the blocks used in the LOWSPC warning as the sum of the data blocks and the local bit map blocks. Previously, YottaDB attempted to report the total blocks used as just the data blocks to match the 'total' field output by MUPIP INTEG. Additionally, YottaDB prints an accurate message about the percent usage when one of these threshold sizes is reached. (GTM-9024)

Language

• MUMPS commands without an argument accept an appropriate response after the "What file: " prompt. The "What file: " prompt may be more appropriate to a mumps -run than for compilation only. Previously, YottaDB appeared to ignore all input at this prompt and optionally allowed MUPIP to accept file or region on a separate line. (GTM-4263)

• The $ZTIMeout=([timeout]\[:labelref]) Intrinsic Special Variable (ISV) controls a single process wide timer. The optional timeout in seconds specifies with millisecond accuracy how long from the current time the timer interrupts the process. If the specified timeout is negative, YottaDB cancels the timer. If the timeout is zero, YottaDB treats it as it would a DO of the vector. The optional labelref specifies a code vector defining a fragment of M code to which YottaDB transfers control as if with a DO when the timeout expires. If the timeout is missing, the assignment must start with a colon and only changes the vector, and in this case, if the vector is the empty string, YottaDB removes any current vector. Note that YottaDB only recognizes interrupts, such as those from $ZTIMEOUT at points where it can properly resume operation, for example, at the beginning of a line, when waiting on a command with a timeout, or when starting a FOR iteration. When a ztimeout occurs, if the last assignment specified no vector, YottaDB uses the current $ETRAP or $ZTRAP with a status warning of ZTIMEOUT. YottaDB rejects an attempted KILL of $ZTIMeout with an error of %YDB-E-VAREXPECTED, and an attempted NEW of $ZTIMeout with an error of %YDB-E-SVNONEW. (GTM-6135)

Example:

   YDB>zprint ^ztimeout
   ztimeout
     ; Display $ztimeout
       write !,$ztimeout               ; display $ZTIMeout - in this case the initial value -1
     ; set with a vector (do ^TIMEOUT)
       set $ztimeout="60:do ^TIMEOUT"  ; timeout of 1 minute. After timeout expires, XECUTEs do ^TIMEOUT
       write !,$ztimeout               ; displays the remaining time:vector until timeout
     ; set without a vector
       set $ztimeout=120               ; set the timeout to 2 minutes without changing the vector
       set $ztimeout="1234do ^TIMEOUT" ; missing colon creates a timeout for 1234 seconds
       set $ztimeout="10:"             ; set the timeout to 10 seconds and vector to current etrap or ztrap
       set $ztimeout=-1                ; set cancels the timeout
     ; Note that set to 0 triggers an immediate timeout
       set $ztimeout=0                 ; triggers the current vector
       set $ztimeout="0:DO FOO"        ; this has the same effect as DO FOO

   YDB>

• Name-level $ORDER(,-1) and $ZPREVIOUS() return an empty string when they reach the trigger definitions (stored in ^#t) as it is not a normally accessible global. Since the introduction of triggers, if there were trigger definitions, these functions could return ^#t. (GTM-7433)

• YottaDB protects buffers used for external calls and produces an EXTCALLBOUNDS error if the external call attempts to exceed the space requested by the call table definition. Previously, YottaDB did not provide this protection and used a less efficient strategy for managing the space. Additionally, when an external call exceeds its specified preallocation (ydb_string_t * or ydb_char_t * output), YottaDB produces an EXCEEDSPREALLOC error. Previously YottaDB did not immediately detect this condition, which could cause subsequent hard to diagnose failures.

  YottaDB supports call-specific options in external call tables by appending a colon to the end of the line followed by zero or more space separated, case-insensitive keywords. The SIGSAFE keyword attests that the specific call does not create its own signal handlers, which allows YottaDB to avoid burdensome signal handler coordination for the external call. Previously, and by default, YottaDB saves and restores signal setups for external calls. (GTM-7952)

• When the name of a source file is not a valid routine name, YottaDB issues a NOTMNAME error and does not produce an object file. Previously, YottaDB eventually gave an error when attempting use the routine. The -OBJECT compilation qualifier used without the -NAMEOFRTN qualifier implicitly names the first routine to match the name of the object qualifier. Note that, as before, listing files take on the name of the source rather than the name of the routine, and the -NAMEOFRTN or -OBJECT qualifiers in $ZCOMPILE are problematic to use with ZLINK commands as they apply to every ZLINK argument. Previously the qualifier applied to all files specified by the same MUMPS or ZCOMPILE command such that all sources received the same object name, which meant the last file was the only one that endured. Also, ZCOMPILE, as documented, accepts qualifiers in its argument prior to the routine list; previously it did not. ZCOMPILE with a wildcard works reliably; previously it stopped compiling routines after encountering a large source file. In addition, the MUMPS and ZCOMPILE commands default file specifications without a .m file extension to have one and they only compile files with a .m extension; previously they did not, although some other facilities did require a .m file extension for source files. As before, explicit ZLINK of a source (.m) file always places the object in the same directory as the specified source. While we are not aware of customers with a practice of using non .m extensions or module names that are not valid M names at compilation, but subsequently rename the object modules, this change requires revision of such practices. (GTM-8178)

• When the second and third argument of $TRANSLATE are literals, the YottaDB compiler calculates the tables used by the translation. Previously, the tables were always prepared at run-time. (GTM-8947)

• YottaDB can return any of the types documented in the external calls API. For C external calls, to prevent memory leaks when returning any pointer types, YottaDB requires the application to allocate returns of these types using ydb_malloc. Note that using the standard malloc for these types (or not explicitly allocating at all) produces a YDB-F-ASSERT. For Java external calls, the plugin manages any necessary allocations. Previously, YottaDB had the ability to return any of the documented types, however the Programmer's Guide did not document the ability to return anything other than an integer status or a long (for Java calls). Additionally, even though YottaDB had the ability to return a range of types, it did not perform the appropriate check for available space in the string pool when returning ydb_char_t*, ydb_char_t**, or ydb_string_t*, which could lead to unpleasant symptoms including a segmentation violation (SIG-11) or incorrect results. Also, attempting to return null values from a C external call results in a %YDB-E-XCRETNULLREF error, and attempting to return null from a Java external call results in a %YDB-E-JNI error. (GTM-8998)

System Administration

• MUPIP REPLICATE -EDITINSTANCE supports a -CLEANSLOTS qualifier. When specified, MUPIP goes through all slots (currently 16) in the replication instance file, identifies the slots that are inactive, and clears them to make them available for reuse. Also, except in the case where an originating primary instance has crashed, MUPIP REPLICATE grabs a "standalone" resource lock for processing -EDITINSTANCE. Previously, MUPIP did not use a resource lock when acting on an -EDITINSTANCE. (GTM-8518)

• MUPIP LOAD returns non-zero exit status for load errors. Previously, in some cases, it inappropriately returned a 0 (Zero) exit status when it had been unable to load one or more records. (GTM-9005)

• MUPIP SET accepts -KEY_SIZE or -RESERVED_BYTES and -RECORD_SIZE in the same command; in previous versions, they were not incompatible, but MUPIP SET continued to give an error when they were combined. (GTM-9011)

Other

• The ^%TRIM() utility allows the specification of what character(s) to trim from either the left and/or right hand side of a given string. The default trim characters are $CHAR(32,9) (<SP> and <TAB>), these can be overridden by passing a string consisting of the desired characters in the optional second parameter. This functionality has existed for some time but was undocumented and not regularly tested. (GTM-8017)

• YottaDB appropriately invokes $ydb_procstuckexec when it encounters a situation, such as BUFOWNERSTUCK, when another process abnormally holds some resource for too long. Previously, under a rare sequence of events the invocation of ydb_procstuckexec could result in a segmentation violation (SIG-11). (GTM-9017)

• The following utilities: %DH, %DO, %HD %OD and %UTF2HEX have optimizations for the sizes most likely to be used in the YottaDB environment. Previously, GTM-5574 extended the maximum size supported by the conversion utilities but that caused a performance reduction for the most common cases. (GTM-9025)

• The YottaDB Cmake build scripts now work with Cmake v3; previously they used a feature (the debug property) deprecated in that Cmake release. (GTM-9031)

• YottaDB TLS encrypted sockets disallow TLS 1.0 and TLS 1.1 protocols. Previously, YottaDB disallowed only SSLv2 and SSLv3 protocols. If you need to selectively re-enable these protocols, please refer to the ssl_options configuration option in "Creating a TLS Configuration File" (GTM-9038)

GT.M V6.3-007

ID	Category	Summary
(GTM-4796)	Language	Reserved name for socket pool
(GTM-7318)	Admin	Audit Direct Mode facility
(GTM-8130)	Other	Modify %GSEL to deal appropriately with names and wildcards
(GTM-8626)	Other	MUPIP JOURNAL requires different names for different output files
(GTM-8653)	Admin	Prevent potential deadlock between MUPIP JOURNAL ROLLBACK and MUPIP REPLICATE -SOURCE -FREEZE=OFF
(GTM-8665)	Admin	Improve reporting on an interrupted MUPIP JOURNAL -RECOVER/-ROLLBACK
(GTM-8729)	Other	MUPIP JOURNAL support for -NOLOSTTRANS and -NOBROKENTRANS
(GTM-8871)	DB	Processes survive a StatDB out-of-space, and maintain statistics when switching between private and shared
(GTM-8872)	Other	Please see GTM-8871
(GTM-8904)	Other	Revise ^%RCE to not lose files when changing routines on multiple file systems
(GTM-9003)	DB	Critical Resource Management Changes
(GTM-9042)	Delete	Resolve some issues with $ZTIMEOUT
(GTM-9043)	Language	The compiler detects too much concatenation in an expression in a way that does not prevent production of an object file
(GTM-9047)	Language	Improved $ZCSTATUS presentation of compilation results
(GTM-9049)	Language	Adjust WRITE of a concatenation expression in non side effect mode for consistency
(GTM-9053)	Admin	Resource management fixes for a couple of unusual cases
(GTM-9056)	Admin	MUPIP SET for -WRITES_PER_FLUSH and -TRIGGER_FLUSH, both of which persist
(GTM-9058)	Language	JOB error handling changes
(GTM-9061)	Other	^%YGBLSTAT returns an empty string when directed to report on a nonexistent process
(GTM-9065)	Other	GDE treats canonic numeric subscripts as numerics rather than strings
(GTM-9071)	Language	Fix ZMESSAGE to allow Boolean expressions in its argument
(GTM-9072)	Admin	GBLOFLOW message identifies the database file rather than a global
(GTM-9073)	Other	Relationship between Maximum Key Size and Maximum Reserved Bytes
(GTM-9074)	Other	GDE accepts values between 2048 and 8388607 for JOURNAL ALLOCATION
(GTM-9075)	Other	GDE accepts values within quotes and exits on first out of bounds error

Database

• If a YottaDB process receives a SIGBUS (SIG-7) signal when attempting to register itself in a statistics database, it prints a STATSDBMEMERR message in the system log, indicating the need to provide sufficient space for the StatsDB file to expand. The process then reverts to process-private statistics collection. After the cause of the SIGBUS has been addressed, the process may turn statistics database logging back on by doing a VIEW "STATSHARE":"REGION", at which point the process returns to doing updates to the corresponding statistics database. Previously, a YottaDB process that received a SIGBUS while trying to do statistics database logging terminated, producing a core file. Additionally, when a YottaDB process changes between private and shared statistics collection, it copies the current statistics into the appropriate statistics location so collected statistics persist. Previously, YottaDB did not properly copy over process-private statistics, meaning that switching over to a statistics database lost any previously collected statistics. (GTM-8871)

• YottaDB uses facilities provided by the operating system to protect critical database and journal pool resources on Linux. Previously, YottaDB used a combination of multiple operating system facilities and its own logic to provide this protection. Several ZSHOW "G" mnemonics behave differently with this change. CQS, CQT, CYS, and CYT are not maintained and contain zeros. CFT and CFE are maintained, but are only incremented a single time for each observed instance of contention, whereas previously it counted the number of low-level synchronization operations performed, which would typically have been significantly larger. CFS is incremented a single time along with CFT (as the square of one is one.) CAT is maintained as before. $VIEW("PROBECRIT") returns valid CPT and CAT fields, but zero for all other fields. (GTM-9003)

Language

• SOCKET devices use "YGTMSOCKETPOOL" to identify the socket pool; an attempt to OPEN a device of that name produces a DEVNAMERESERVED error. Note this change requires adjustment of any code explicitly referencing the socket pool. Previously, SOCKET devices used the name "socketpool" to designate the socket pool and an OPEN of a device with that name prevented the use of the pool or access to any devices sockets in it. (GTM-4796)

• YottaDB detects the case of more concatenation operands in a row than it can handle when parsing the source code; previously, it detected this at code generation, which meant it always failed to create an object file in this case. (GTM-9043)

• $ZCSTATUS holds an indication of the result of the last ZCOMPILE, ZLINK, $ZTRIGGER() or auto-zlink compilation. One (1) indicates a clean compilation, a positive number greater than one is an error code you can turn into text with $ZMESSAGE(), and a negative number is a negated error code that indicates that YottaDB was not able to produce an object file. The error details appear in the compilation output, so $ZCSTATUS typically contains the code for ERRORSUMMARY. Previously, $ZSTATUS almost always indicated a one (1) except when object file creation failed. $ZTRIGGER() and MUPIP TRIGGER don't install trigger definitions with XECUTE strings that do not compile without error; previously they did. In addition, the value for $ZCSTATUS provided by ZSHOW "I" matches that provided by WRITE $ZCSTATUS; previously ZSHOW provided a zero (0) when it should have provided a one (1). (GTM-9047)

• WRITE does not turn an expression starting with a concatenation operation into separate arguments if the expression is within parentheses. WRITE compilation turns a concatenated sequence into separate arguments which, when not processing in side effect mode, can affect the evaluation of side effects. Therefore the documentation contains the following: "The YottaDB compiler breaks a concatenated WRITE argument into a series of WRITE arguments to eliminate the overhead of the concatenation. If circumstances provide a reason for a single WRITE, perform the concatenation prior to the WRITE." Previously protecting the concatenation with parentheses in non-side effect mode did not suppress this optimization, which made the result inconsistent with separate evaluation. (GTM-9049)

• YottaDB handles JOB errors differently. Errors associated with the specified routine, label, or offset appear in the error file of the JOBbed process in detail in addition to the JOBFAIL error received by the original process. Previously, YottaDB did not report the more specific errors and did not start the JOBbed process. YottaDB detects and reports JOBLVN2LONG errors in the original process; YottaDB does not start the JOBbed process in this case. Previously, the JOBbed process would report JOBLVN2LONG to its error file. (GTM-9058)

• The ZMESSAGE command appropriately handles a Boolean expression within an argument; previously, such an argument tended to cause a segmentation violation (SIG-11). (GTM-9071)

System Administration

• YottaDB supports the ability to log actions initiated from a principal device including MUMPS commands typed interactively, or piped in by a script or redirect, from the principal device ($PRINCIPAL) and/or any information entered in response to a READ from $PRINCIPAL. An action initiated from $PRINCIPAL executes as usual when the Audit Principal Device is disabled, which it is by default. However, when the Audit Principal Device is enabled, YottaDB attempts to send the action out for logging before acting on it. Additionally, the $ZAUDIT Intrinsic Special Variable (ISV) provides a Boolean value that indicates whether or not the Audit Principal Device is enabled. Please see the Additional information for GTM-7318 - Audit Principal Device in this document for details. (GTM-7318)

• MUPIP JOURNAL -ROLLBACK -ONLINE -BACKWARD, on encountering a frozen region when Instance Freeze is ON, releases all its resources and retries the rollback from the start. Previously, this could cause a potential deadlock with MUPIP REPLICATE -SOURCE -FREEZE=OFF. This issue was only observed in the development environment, and was never reported by a user. (GTM-8653)

• MUPIP INTEG reports an interrupted MUPIP JOURNAL -RECOVER/-ROLLBACK operation on the database. Previously, a MUPIP INTEG on such a database did not report an interrupted recovery. Note: The MUPIP dumpfhead command already provided this information. YottaDB reports the "Recover interrupted" field with DSE DUMP -FILEHEADER even when journaling is turned off. Previously, YottaDB reported the "Recovery interrupted" field only with DSE DUMP -FILEHEADER -ALL and only when journaling was turned ON. (GTM-8665)

• YottaDB does additional error checking when managing shared resources associated with relinkctl and replication update operations. Previously rare error conditions could leave the resources in an unintended status, resulting in core dumps and interfering with future relinkctl or replication update operations. This issue was only observed in the development environment, and was never reported by a user. (GTM-9053)

• MUPIP SET {-FILE|-REGION} accepts -TRIGGER_FLUSH=n and -WRITES_PER_FLUSH=n qualifiers; previously, only DSE supported these qualifiers. Also, the trigger_flush value appears in MUPIP DUMPFHEAD as trigger_flush_top and acts as a stable limit; previously, YottaDB tended to lose any user supplied value as it made adjustments intended to improve performance. (GTM-9056)

• GBLOFLOW messages identify the database file that is full; previously, they identified the global node the process was updating when it found no room, and in the case of a TP transaction, the report was for the last update in the transaction, which was not necessarily in the full database file. (GTM-9072)

Other

• The %GSEL utility now silently ignores any subscript in the search string and throws a non-fatal error if the input contains an invalid character. Previously, %GSEL would remove invalid characters and then perform the search. This could cause problems if a subscript was present because the utility removed "(" and ")" from the search pattern but not what was between them. This change also applies to %GCE, %GD, %GO, and %GSE which use GD^%GSEL to search for globals. In addition, when used interactively, it attempts to preserve the original I/O state of the caller; previously, it tended to leave that state disrupted. (GTM-8130)

• MUPIP JOURNAL does not allow any two of -EXTRACT, -LOSTTRANS or -BROKENTRANS to specify the same file name unless they are special files (-stdout or /dev/null). Previously, MUPIP JOURNAL allowed overlapping file specifications, which lead to unexpected behavior including missing files and the specified file was not created. (GTM-8626)

• MUPIP JOURNAL recognizes -NOLOSTTRANS and -NOBROKENTRANS as optional qualifiers which cause it to discard any lost or broken transactions rather than record them in files. Previously, MUPIP JOURNAL always produced files containing any lost and/or broken transactions as there was no such option to discard them. (GTM-8729)

• Please see GTM-8871. (GTM-8872)

• The ^%RCE utility deals appropriately with the case where $ZROUTINES lists source directories where the target directory is on a different file system than the default (first) source directory. Previously, the utility would delete the original, but then fail to move the modified copy to the correct directory. In addition, when used interactively, it attempts to preserve the original I/O state of the caller; previously, it tended to leave that state disrupted. (GTM-8904)

• ^%YGBLSTAT skips non-existent processes and returns an empty string when it is sent after an nonexistent PID. Previously, it could inappropriately return information on the reporting process or defunct processes. In addition, when used interactively, it attempts to preserve the original I/O state of the caller; previously, it tended to leave that state disrupted. Also, the utility versions its local working storage where previously it did not. (GTM-9061)

• GDE treats canonic numeric string subscripts as numeric subscripts, in line with other YottaDB utilities. Previously, GDE treated them as string subscripts. In addition, GDE SHOW -NAME <NAME> prints the name-region mapping of all subscripts of the specified name. Previously, GDE only printed the name-region mapping of the unsubscripted name. (GTM-9065)

• GDE and MUPIP maintain a consistent relationship between Maximum Key Size and Maximum Reserved Bytes. It adheres to the equation:

  Maximum Reserved Bytes = Block Size - Key Size - (Size of Block Header + Size of Record Header + Size of Block id + B-star Record Size)

  which is equivalent to

  Block Size - Key Size - 32

  Previously, they used inconsistent calculations and allowed inconsistent Maximum Key Size. (GTM-9073)

• GDE accepts values between 2048 and 8388607 for JOURNAL ALLOCATION, with 2048 as the default value. Previously, GDE incorrectly accepted values between 200 and 16777216 with 200 as the default value. (GTM-9074)

• For -ACCESS_METHOD and -NULL_SUBSCRIPTS, GDE accepts valid values enclosed within double quotes. Previously, GDE rejected valid values when enclosed within double quotes. If a value of any qualifier is out of bounds, GDE prints VALTOOSMALL/VALTOOBIG and exits immediately without further processing. Previously GDE continued to process that value and the values of other qualifiers and printed potentially confusing messages. (GTM-9075)

More Information

Additional information for GTM-7318 - Audit Principal Device

Direct Mode receives a command line, YottaDB first checks if direct mode auditing or Audit Principal Device (APD) is enabled. If it is disabled, which it is by default, the command executes as usual. If it is enabled, YottaDB establishes a connection (via a UNIX/TCP/TLS socket) to a logger/listener process, and sends the to-be-logged command through the socket to the listener for logging. If sending is successful, YottaDB executes the logged command. If the connection is not successful or sending of the command fails, then YottaDB produces an error and does NOT execute the command.

Enabling Audit Principal Device (APD)

The APD_ENABLE entry in a restrictions definition file turns on APD and enables the logging of all code entered from Direct Mode and optionally any input entered on the principal device ($PRINCIPAL). To enable APD, add a line with the following format to the restriction file:

APD_ENABLE:[comma-separated-list-of-options]:{path-to-sock-file|host:port}[:tls-id]

• The optional "comma-separated-list-of-options" can consist of zero or more of these options:

  • TLS - Enables TLS connectivity between YottaDB and the logger; this option requires the host information (e.g. IP/port or hostname/port)
  • RD - Enables logging of all responses READ from $PRINCIPAL in addition to that entered at the Direct Mode prompt. This option is more comprehensive and captures input that might be XECUTEd, but depending on your application architecture may significantly increase the amount of logged information.

• The "path-to-sock-file" is the absolute path of the UNIX domain socket file for connecting to the logger.

• The "host" is the hostname or numeric IPv4/IPv6 address of the logger; numeric IP addresses must be enclosed in square brackets (i.e. '[' and ']').

• The "port" is the port number the logger listens on.

• The optional "tls-id" is the label of the section within the YottaDB configuration file that contains TLS options and/or certificates for YottaDB to use; APD ignores any "tls-id" if the "TLS" option is not specified.

If parsing the APD_ENABLE line in restriction file or initializing logger information fails, YottaDB enforces all restrictions (default restriction file behavior).

Examples

APD_ENABLE::/path/to/sock/file/audit.sock

Adding this line to the restriction file enables APD. YottaDB connects with the logger via UNIX domain socket using the domain socket file "/path/to/sock/file/audit.sock" and sends all Direct Mode activity from $PRINCIPAL to logger.

APD_ENABLE:RD:[123.456.789.100]:12345

Adding this line to the restriction file enables APD. YottaDB connects with the logger (listening on port 12345 at the IPv4 address 1enable23.456.789.100) via TCP socket and sends all Direct Mode and READ activities from $PRINCIPAL to logger.

APD_ENABLE::loggerhost:56789

Adding this line to the restriction file enables APD. YottaDB connects with the logger (listening on port 56789 at the hostname "loggerhost") using a TCP socket and sends all Direct Mode activities from $PRINCIPAL to logger.

APD_ENABLE:TLS,RD:[1234:5678:910a:bcde::f:]:12345:clicert

Adding this line to the restriction file enables APD. YottaDB connects with the logger (listening on port 12345 at the IPv6 address 1234:5678:910a:bcde::f:) via TLS socket. YottaDB configures its TLS options for APD based on the contents within the section of the configuration file labeled "clicert". YottaDB sends all Direct Mode and READ activities from $PRINCIPAL to logger.

Logging

The "logger" is a separate server-like program responsible for receiving the to-be-logged information from YottaDB and logging it. This separate program must be introduced by the user, either running in foreground or background, in order for logging to actually work. YottaDB distributions include basic example logger programs.

The six fields in the message, separated by semicolons (';'), contain information on the to-be-logged activity. Each to-be-logged message sent to the logger from YottaDB has the following format:

dist=<path>; src={0|1|2}; uid=<uid>; euid=<euid>; pid=<pid>; command=<text>

• The "dist" field, shows the path to location of the sender/user's $ydb_dist (YottaDB executables).
• The "src" field shows zero (0) for input from unknown source, one (1) for Direct Mode input, or two (2) for READ input from $PRINCIPAL.
• The next three fields ("uid", "euid", and "pid") show (respectively) decimal representations of the user ID, effective user ID, and process ID of the process that sent the message.
• The "command" field is the input provided on the YottaDB side.

Examples

dist=/path/to/ydb_dist; src=1; uid=112233445; euid=112233445; pid=987654; command=write "Hello world",!

dist=/usr/library/r126/dbg; src=2; uid=998877665; euid=998877665; pid=123456; command=set a=789

Error Messages

From GT.M V6.3-006

ERRWZTIMEOUT, Error while processing $ZTIMEOUT

Run Time Error: This indicates a problem invoking the current $ZTIMEOUT vector and usually accompanies other error messages

Action: Examine and correct the code vector specified by $ZTIMEOUT, or if there is none, examine the current value for $ETRAP or $ZTRAP. Unlike $ETRAP and code values for $ZTRAP, which are evaluated when they are assigned, compilation of $ZTIMEOUT vectors occurs when the vector is invoked by the expiration of the specified time.

EXCEEDSPREALLOC, Preallocated size ssss for M external call label LLLL exceeded by string of length SSSS.

Call out Error: The code invoked as external routine name LLLL returned a string of length SSSS, but the call table specified a maximum length of ssss for the return.

Action: Revise the external routine to abide by the call table size or change the call table to preallocate a suitably larger size.

EXTCALLBOUNDS, Wrote outside bounds of external call buffer. M label: LLLL.

Call out Fatal: The code invoked as external routine name LLLL violated the bounds of its allocated buffers.

Action: Ensure the non-YottaDB code uses appropriate allocations, pointer management logic and bounds checking.

FILEEXISTS, File xxxx already exists.

This is an existing message with updated text.

MUPIP Error: This indicates that MUPIP discovered a file with the filename xxxx already existing and did not overwrite it while executing the specified command(s). In many cases, this is an expected outcome when the action has an explicit or implicit target of multiple database files which may be in differing states.

Action: If appropriate, rename the already existing file xxxx and reissue the MUPIP command(s), or modify the MUPIP command to name (explicitly/implicitly) a file different from xxxx. If you encountered this error with MUPIP BACKUP, use the -REPLACE qualifier if you want to replace the existing backup files.

FILERENAME, File xxxx is renamed to yyyy.

This is an existing message with updated text.

Run Time Information: This indicates that an existing file xxxx has been renamed to yyyy so that a new file created with the original name does not overwrite the existing one. YottaDB renames files during an automatic journal switch in case no explicit journal file name is specified, in which case the message is sent to the operator log. The utilities (MUPIP, GT.CM) rename files while opening log files or journal extract files and they send the message to the terminal. YottaDB or utilities rename files only if the new file name specified already exists.

Action: This information message confirms the success of the file rename operation. No futher action is necessary unless there are other warning, fatal, and/or error category messages.

GTMSECSHRPERM, The gtmsecshr module in $ydb_dist (DDDD) does not have the correct permission and UID (permission: PPPP, and UID: UUUU).

This is an existing message with updated text.

Run Time Error: This indicates that a client did not start the GTMSECSHR, installed to DDDD, because the executable was not owned by root (UUUU is the actual owner) and/or did not have setuid and/or execute permissions (actual permissions are PPPP).

Action: Arrange to provide the GTMSECSHR executable with the proper characteristics. The executable must be SETUID root with execute permissions for the current user.

JNLCREATE, Journal file xxxx created for <database/region> yyyy with aaaa.

This is an existing message with updated text.

MUPIP Information: This indicates that a journal file xxxx is created for database/region yyyy with NOBEFORE_IMAGES or BEFORE_IMAGES journaling option (aaaa).

Action: This informational message confirms the success of the new journal file creation operation for a region. No futher action is necessary unless there are other warning, fatal, and/or error category messages.

JNLFILOPN, Error opening journal file jjjj for database file dddd.

Run Time/MUPIP Error: This indicates that YottaDB was unable to open journal file jjjj for the specified database file dddd. The Source Server exits with a JNLFILOPN message after six failed attempts to open journal files.

Action: Check the authorizations for the user of the process and the health of the file system holding the journal file.

JNLSPACELOW, Journal file jjjj nearing maximum size, nnnn blocks to go.

This is an existing message with updated text.

Run Time Information: Depending on your settings for ALLOCATION, AUTOSWITCHLIMIT, and EXTENSION journaling options, you may see one to three JNLSPACELOW messages for each generation of a journal file. When the difference between AUTOSWITCHLIMIT and ALLOCATION is an exact multiple of EXTENSION, YottaDB attempts to write the JNLSPACELOW message to the operator log three times as a journal file reaches its maximum size. The first JNLSPACELOW message appears in the operator log when the available free space (blocks) in a journal file is equal to twice the EXTENSION, the second appears when the available free space is equal to EXTENSION, and the third appears when the journal file reaches the maximum size (AUTOSWITCHLIMIT). With EXTENSION=0 or EXTENSION=AUTOSWITCHLIMIT, YottaDB logs the JNLSPACELOW message only once per journal file to the operator log.

Action: The JNLSPACELOW message is an information message and requires no action. However, you can use the JNLSPACELOW messages as part of monitoring journaling space requirements or as an operational practice to a trigger to intervene in journal file management. Use the frequency of JNLSPACELOW messages to proactively monitor how fast a journal file grows and as part of a monitoring alorithm that helps predict how soon the disk is likely to hit a quota limit.

JNLSTATE, Journaling state for <database/region> xxxx is now yyyy.

This is an existing message with updated text.

MUPIP Information: This indicates that journal state for the database/region xxxx is now yyyy.

Action: This information message confirms the success of journal state change operation. No futher action is necessary unless there are other warning, fatal, and/or error category messages.

JNLSWITCHSZCHG, Journal AUTOSWITCHLIMIT [aaaa blocks] is rounded down to [bbbb blocks] to equal the sum of journal ALLOCATION [cccc blocks] and a multiple of journal EXTENSION [dddd blocks].

This is an existing message with updated text.

MUPIP Information: This indicates that the specified AUTOSWITCHLIMIT value was rounded down as little as possible to make it aligned to the ALLOCATION + a multiple of EXTENSION. Any subsequently created journal file will use this value for AUTOSWITCHLIMIT.

Action: If the automatically rounded value for AUTOSWITCHLIMIT is inappropriate, specify an appropriate value for ALIGNSIZE, ALLOCATION, and/or EXTENSION.

JNLSWITCHTOOSM, Journal AUTOSWITCHLIMIT [aaaa blocks] is less than journal ALLOCATION [bbbb blocks] for database file dddd.

This is an existing message with updated text.

MUPIP Error: This indicates that the specified value or the automatically calculated value for AUTOSWITCHLIMIT from a MUPIP SET JOURNAL command is less than the default or specified value of ALLOCATION. This error also indicates that the AUTOSWITCHLIMIT value specified was greater or equal to the ALLOCATION but in turn got rounded down, and this rounded down value is less than the ALLOCATION.

Action: Specify a higher value of AUTOSWITCHLIMIT or specify an ALLOCATION value that is less than the AUTOSWITCHLIMIT.

LOCKCRITOWNER, LOCK crit is held by: PPPP.

Run Time/LKE Information: This shows any current owner of the resource managing M LOCKs.

Action: If a process persists in this state investigate what it's doing and, if appropriate, consider terminating it.

LOCKSPACEINFO, Region: rrrr: processes on queue: pppp/qqqq; LOCK slots in use: llll/kkkk; SUBSCRIPT slot bytes in use: ssss/tttt.

This is an existing message with updated text.

Run Time Error: This indicates that the environment attempted more concurrent M LOCKs than the configured LOCK_SPACE for region rrrr can support. pppp processes are waiting on a lock. llll locks are in use. qqqq and kkkk indicate maximum number of process queue entries, and maximum number of locks respectively.

Action: Analyze the LOCK protocol for efficiency. Use mupip set -region -lock_space=size "rrrr" to increase the lock space for region rrrr. To avoid the same problem the next time you recreate the database, use GDE to make the analogous change to lock_space for the segment mapped to the ffff file in the global directory used to MUPIP CREATE this region.

LOWSPC, WARNING: Database DDDD has less than PPPP% of the total block space remaining. Blocks Used: UUUU Total Blocks Available: AAAA.

This is an existing message with updated text.

Operator log Information: The database has UUUU block in use and is appoaching its current limit of AAAA blocks. When the database reaches the 88% size threshold, and for every 1% increase in size and beyond, YottaDB reports the blocks used in the LOWSPC warning as the sum of the data blocks and the local bit map blocks.

Action: Purge data if possible. Consider a MUPIP REORG to compact the remaining data. Investigate whether migrating to a database created by a current version has a higher limit. Move some data to another, possibly new, region and delete it from this one.

MLKCLEANED, LOCK garbage collection freed aaaa lock slots for region rrrr.

LKE Information: LKE CLNUP was able to free lock slots when requested.

Action: No action required.

MLKHASHRESIZE, LOCK hash table increased in size from aaaa to bbbb and placed in shared memory (id = mmmm).

Operator log Information: YottaDB needed to expand a hash table used for managing LOCK information.

Action: No user action is required, but shared memory monitoring will show an additional shared memory segment with id mmmm.

MLKHASHRESIZEFAIL, Failed to increase LOCK hash table size from aaaa to bbbb. Will retry with larger size.

Operator log Warning: YottaDB needed to expand a hash table used for managing LOCK information needed to be expanded, but the initial attempt failed, necessitating a retry.

Action: A subsequent MLKHASHRESIZE indicates that the retry succeeded and no user action is required.

MLKHASHTABERR, A LOCK control structure is damaged and could not be corrected. Lock entry for LLLL is invalid.

LKE Error: LKE CLNUP -INTEG encountered an out-of-design situation for LOCK LLLL and was unable to repair it automatically.

Action: Immediately report the entire incident context with information from the operator log and any other relevant information to your support channel.

MLKHASHWRONG, A LOCK control structure has an invalid state; LOCK table failed integrity check TTTT.

LKE Error: MLK CLNUP -INTEG encountered damage to the data structures related to LOCK management. The text in TTTT describes whether LKE was able to correct the error or not.

Action: If LKE was not able to correct the error, immediately report the entire incident context with information from the operator log and any other relevant information to your support channel as soon as possible.

MUNOACTION, MUPIP unable to perform requested action.

This is an existing message with updated text.

MUPIP Error: This indicates that MUPIP encountered an error, which prevented the requested action.

Action: Review the accompanying message(s) to identify the cause that prevented MUPIP from performing the requested operation.

NOTMNAME, XXXX is not a valid M name.

Compile Time Error: M names must be ASCII, start with a "%" or an alpha and thereafter contain only alphanumeric characters. In YottaDB M, names are currently functionally limited to 31 characters, in most cases, by truncation.

Action: Correct the (typically) routine name to comply with the supported format. Names are also used for labels and both global and local variables. Note that YottaDB usually truncates names longer than its supported maximum, which YottaDB recommends against having as, while it can provide embedded information, can lead to ambiguity or other unintended behavior.

RESRCWAIT, Waiting briefly for the tttt semaphore for region rrrr (ffff) was held by PID pppp (Sem. ID: ssss).

Run Time Information: A process started a three (3) second wait for an FTOK or access control semaphore. If the process with PID pppp does not release the semaphore before the timeout expires, the waiting process bypasses acquiring the semaphore. tttt identifies the semaphore type: "FTOK" or "access control"; rrrr is the region; ffff is the database file corresponding to region rrrr; ssss is the semaphore ID.

Action: None required.

XCRETNULLREF, Returned null reference from external call LLLL.

Call out Error: The code invoked as external routine name LLLL returned a NULL pointer. While YottaDB accepts returns of a zero (0) value or an empty string, it does not support the return of a NULL pointer.

Action: Revise the external call code to return a pointer to an appropriate value.

ZTIMEOUT, ZTIMEOUT Time expired.

Run Time Warning: This warning message appears when $ZTIMEOUT expires and there were no vectors defined. If no error handlers are defined, YottaDB invokes the default trap which puts the control to Direct Mode.

Action: Check the message(s) for more information on where the timer expired in the current process. If needed, set an appropriate error handler to specify an action associated with $ZTIMEOUT expiry or define a $ZTIMEOUT with a vector.

From GT.M V6.3-007

APDCONNFAIL, Audit Principal Device failed to connect to audit logger

Run Time Error: The facility for logging activity on principal devices is enabled, but is unable to form a connection with its configured logging program. This prevents a process from taking actions configured for logging initiated on its principal device ($PRINCIPAL).

Action: Check to make sure logger program is running and listening/accepting connections. If using a TCP or TLS-enabled logger, make sure the port number the logger is listening/accepting on matches the port number provided in the restriction file. Ensure the provided information (logger's connection info) in the restriction file is correct. Also make sure the line in restriction file is in correct format. If running a TLS-enabled logger, make sure the logger's TLS certificate is signed by a root CA that YottaDB is aware of through the TLS configuration file. Check syslog for more information on the error. After addressing identified issues, restart all processes subject to APD.

APDINITFAIL, Audit Principal Device failed to initialize audit information

Run Time Error: YottaDB was unable to process or initialize the provided information (e.g. IP, hostname, port number, UNIX domain socket file path, or TLS ID) from the restriction file. This prevents a process from taking actions configured for logging initiated on its principal device ($PRINCIPAL).

Action: Check the restriction file to make sure information is in proper format. After addressing identified issues, restart all processes subject to APD.

APDLOGFAIL, Audit Principal Device failed to log activity

Run Time Error: YottaDB was unable to send the to-be-logged activitiy to logger. This prevents a process from taking the action initiated on its principal device ($PRINCIPAL).

Action: Check to make sure that YottaDB is able to successfully connect to the logger program. Check syslog for more information on error.

DEVNAMERESERVED, Cannot use NNNN as device name. Reserved for YottaDB internal usage.

Run Time Error: This error appears when there is an attempt to OPEN a device with the name YGTMSOCKETPOOL. YottaDB internally reserves the name YGTMSOCKETPOOL to identify the socket pool and prevents any other device from using it.

Action: Use a different name for the SOCKET device.

GBLOFLOW, Database file FFFF is full

This is an existing message with updated text.

Run Time/MUPIP Error: This indicates that an error was encountered while extending database file FFFF.

Action: Examine the accompanying message(s) for the cause of the error. If the error is due to insufficient authorization, address that. If the error is due to TOTALBLKMAX (refer to the explaination of that message) or a lack of enough free space on the disk to fit the size of a database file, try performing a KILL of some nodes in the database to get free blocks in the existing allocated space (you may need to KILL several subscripted nodes before you can KILL a name node).

ILLEGALUSE, Illegal use of the special character "?" in %GSEL

Utility Error: This is an illegal use of the special character "?" in %GSEL. The special character "?" is not valid as the first character of a global name search pattern. "?" only valid as the first character of a search pattern when invoking the commands "?D" or "?d".

Action: Review and re-enter a valid search pattern.

INVALIDGBL, Search pattern is invalid

Utility Error: The search pattern used is invalid due to either using invalid characters or improper formatting.

Action: Review and re-enter a valid search pattern.

ORLBKREL, ONLINE ROLLBACK releasing all locking resources to allow a freeze OFF to proceed

MUPIP Information: MUPIP ROLLBACK -ONLINE encountered an Instance Freeze and must release its resources and restart to prevent a possible deadlock.

Action: None required as this is an informational message.

ORLBKRESTART, ONLINE ROLLBACK restarted on instance iiii corresponding to rrrr

MUPIP Information: MUPIP ROLLBACK -ONLINE is restarting on the instance iiii with replication journal pool rrrr.

Action: None required for this informational message.

STATSDBMEMERR, Process attempted to create stats block in statistics database SSSS and received SIGBUS--invalid physical address. Check file system space.

Run Time Error: A process attempted to enable shared statistics collection for the region associated with SSSS, but was unable to find room to add its records, so it cannot contribute to sharing. This message goes to the operator log facility rather than the process as an error, but the process continues without shared statistics.

Action: Adjust the environment so that SSSS can expand, and then, if possible, have the process again attempt to enable sharing.

TOTALBLKMAX, Extension exceeds maximum total blocks, not extending

This is an existing message with updated text.

Run Time Error: This indicates that the database file extension specified implicitly or explicitly (using MUPIP EXTEND) would cause the GDS file to exceed its maximum size. Please refer to the most recent Release Notes for maximum database sizes.

Action: Modify the extension to use a smaller size. This may indicate that you should move some contents of the database file to another file.

TRANS2BIG, Transaction exceeded available buffer space for region rrrr

This is an existing message with updated text.

Run Time Error: This indicates that a transaction updated more blocks than the global buffer could hold for a particular region rrrr or accessed more than the single transaction limit of 64K blocks.

Action: Look for missing TCOMMIT commands; modify the code to reduce the total content or change content of the transaction. If the transaction is as intended and the issue is the number of updates, increase the GLOBAL_BUFFERS for the region using MUPIP SET, or modify the Global Directory to redistribute the relevant globals to more regions. If this occurs on a replicating instance it may indicate either a difference in configuration between the originating and replicating instances, which probably should be addressed, or a transaction that was borderline on the originating instance, but failed on the replicating instance because of difference in the database layout. In the later case, consider examining the application code to see if it's possible to reduce the size of the transaction, or alternatively increase the global buffers on both the instances.

UNIQNAME, Cannot provide same file name (nnnn) for ffff and FFFF

MUPIP Error: The command species the same name, nnnn for both output ffff and output FFFF.

Action: Revise the command to use unique names for different outputs.

Legal Stuff

Copyright © 2019 YottaDB LLC

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts and no Back-Cover Texts.

YottaDB® is a registered trademark of YottaDB LLC. GT.M™ is a trademark of Fidelity National Information Services, Inc. Other trademarks belong to their respective owners.

This document contains a description of YottaDB and the operating instructions pertaining to the various functions that comprise the software. This document does not contain any commitment of YottaDB LLC. YottaDB LLC believes the information in this publication is accurate as of its publication date; such information is subject to change without notice. YottaDB LLC is not responsible for any errors or defects.

YottaDB r1.24

Binary Distributions

sha256sum	file
7e405a94e0be03ac24be5d6c2741f0195414c875b0a72fe8c6fdfb6d9693b375	yottadb_r124_linux_aarch64_pro.tgz
e01dd5cebe5c407d0f32419eab08429765f653df24e7a2b857485d386abaed54	yottadb_r124_linux_armv6l_pro.tgz
01c6f70216a5e07df0cba611c5d549c6caefc13fec43e481d8efe7a4c4679731	yottadb_r124_linux_armv7l_pro.tgz
4010f898fc190dce8e3fe3b73e6e26b76461dd8352c372cc22731d85b0d6c943	yottadb_r124_linux_x8664_pro.tgz
8e125474aed4b1aa0e0a66c527ad5f558da48353bd4b2b82bfd29e58fe1d26dd	yottadb_r124_rhel7_x8664_pro.tgz

Release Note Revision History

Revision	Date	Summary
1.01	February 6, 2019	Added additional error message information
1.00	January 31, 2019	r1.24 Initial Release

Contact Information

YottaDB LLC

40 Lloyd Avenue, Suite 104
Malvern, PA 19355, USA
[email protected]
+1 (610) 644-1898

Support

Customers

Contact your YottaDB support channel.

Others

For free (to you) support from members of communities who run widely available applications on YottaDB, please use an application-specific list where appropriate.

• For issues specific to the use of YottaDB from node.js via nodem, QewdJS or Enterprise Web Developer, post to the Enterprise Web Developer community.

• For issues specific to the use of YottaDB with VistA flavors, post to the Hardhats list.

• For issues specific to the use of YottaDB with M other than for applications above, post to the comp.lang.mumps list.

• If you are not sure where to post, or for requests other than to the above communities, post an issue at https://github.com/YottaDB/YottaDB/issues and include the words "help wanted" in the summary.

r1.24

Overview

r1.24 is a major release that brings much new functionality to YottaDB.

pkg-config is a standard tool on Linux systems for managing package versions, for example, to query the location and latest installed version of a package. YottaDB is now part of this ecosystem: the ydbinstall script creates a pkg-config file for YottaDB at /usr/share/pkgconfig/yottadb.pc. In turn, this means that an application can write a script to use YottaDB without the script needing to be updated when a new YottaDB release is installed on a system, or if a YottaDB release is installed in different directories on different systems, e.g.

source $(pkg-config --variable=prefix yottadb)/ydb_env_set # setup environment
gcc -I $ydb_dist -L $ydb_dist -o myprog myprog.c -lyottadb
./myprog

or

source $(pkg-config --variable=prefix yottadb)/ydb_env_set # setup environment
mumps -run mypro

Fitting into the standard ecosystem becomes increasingly important as we release YottaDB wrappers providing standard APIs for a variety of languages starting with Go.

Released as field test grade functionality are Simple API functions that support multi-threaded applications. Although the YottaDB engine is itself single-threaded, it supports multi-threaded applications as described in our blog post YottaDB Support for Multi-threaded Applications and in the Simple API section of the Multi-Language Programmers Guide. An API with support for threads allows YottaDB to create wrappers for inherently multi-threaded language implementations such as Go, as described in the Programming in Go section of the Multi-Language Programmers Guide. Note that r1.24 is otherwise considered a production release. This means that we consider YottaDB r1.24 to be suitable for use in production except for functionality to support multi-thread applications which are suitable for use in development and testing, but not in production. In the next release, if not sooner, we intend for that functionality to be suitable for production use, like the rest of YottaDB.

64-bit ARM (aarch64) is a supported CPU architecture, running Ubuntu 18.04 LTS, tested on Raspberry Pi 3. If you need another distribution of Linux to be Supported, or testing on another hardware implementation of the architecture, please contact us. YottaDB thanks community member Steve Johnson for contributing the port.

r1.24 comes with numerous & noteworthy functional & performance enhancements, fixes (including fixes to issues in the upstream GT.M code base), as described below with full details on Gitlab.

YottadB r1.24 is upward compatible with both YottaDB r1.22 and GT.M V6.3-005.

Platforms

A platform is a combination of a CPU architecture and an operating system. A platform is Supported, Supportable, or Unsupported. Supported means that we have the platform in our development environment and test each release on that platform. Supportable means that although we do not necessarily have such a platform in our environment, we have no reason to believe that the software will not run on it. All others are Unsupported.

CPU Architecture	Supported OS Version(s)	Notes
64-bit x86	Ubuntu 18.04 LTS; Red Hat Enterprise Linux 7.6	There are separate binary distributions for Ubuntu and Red Hat, owing to differences in library versions of those distributions.
64-bit ARM (Raspberry Pi 3 Model B)	Ubuntu 18.04 LTS	While YottaDB is Supportable on other ARMv8-A CPUs, owing to variations in the implementations of ARM microarchitectures, we recommend that you ensure the software runs correctly before committing to any specific hardware other than those Supported. Please contact [email protected] if you want a specific combination of OS and CPU microarchitecture to be Supported.
32-bit ARM (Raspberry Pi 3 Model B; BeagleBone Black Wireless)	Raspbian GNU/Linux 9.1; Stretch IoT (non GUI)	While YottaDB r1.24 is Supportable on other ARMv7-A CPUs, owing to variations in the implementations of ARM microarchitectures, we recommend that you ensure the software runs correctly before committing to any specific hardware other than those Supported. Please contact [email protected] if you want a specific combination of OS and CPU microarchitecture to be Supported.
32-bit ARM (Raspberry Pi Zero)	Raspbian GNU/Linux 9.1	While YottaDB r1.24 is Supportable on other ARMv6 CPUs, owing to variations in the implementations of ARM microarchitectures, we recommend that you ensure the software runs correctly before committing to any specific hardware other than those Supported. Please contact [email protected] if you want a specific combination of OS and CPU microarchitecture to be Supported.

Recent releases of major GNU/Linux distributions with contemporary kernels, glibc and ncurses are Supportable. Specific notes:

• On Ubuntu releases after 18.04 LTS, YottaDB needs the libtinfo5 package to be installed.
• On Arch Linux and possibly other leading edge distributions, YottaDB may need to be recompiled from source code owing to library and tool chain versions significantly more recent than those used in building the distribution.

Installation

See our Get Started page to use YottaDB.

We strongly recommend that you install YottaDB r1.24 in a newly created directory, different from those of YottaDB r1.22 and any GT.M versions you may have installed on the system.

Removing an installed YottaDB release

Assuming $ydb_dist points to the directory where YottaDB is installed:

• Cleanly shut down all application processes using that release.
• Execute mupip rundown && mupip rundown -relinkctl.
• Ensure that there are no gtcm* or gtmsecshr processes active.
• Use sudo lsof | grep $ydb_dist to ensure there are no open files.
• Delete the directory with sudo rm -rf $ydb_dist.

Upgrading to YottaDB r1.24

As YottaDB r1.24 is upward compatible from both YottaDB r1.22 and GT.M V6.3-005, the minimal upgrade steps are:

• Install YottaDB r1.24.
• Recompile any object code, and recreate shared libraries where appropriate.
• If you are using encryption, compile and install the reference implementation plugin or your customized plugin.
• Cleanly shut down the application and ensure that the database files are shut down using mupip rundown from the prior release.
• Switch journal files with the new YottaDB release.
• Start using the new YottaDB release.

To upgrade from older GT.M releases, first upgrade to GT.M V6.0-000 or later and follow the steps above, or contact your YottaDB support channel for assistance and guidance.

A more sophisticated upgrade technique is:

• Install YottaDB r1.24.
• Create a new replicated instance of your application (on the same system or a different system).
• Assuming the existing instance is A, and the new instance is B, upgrade B to r1.24 and start replicating from A to B.
• Once B catches up, switchover so that B is in a primary role replicating to A.
• Once you are satisfied with B, remove (or upgrade) A.

Change History

r1.24

YottaDB r1.24 includes the following changes from YottaDB r1.22.

ID	Category	Summary
(#95)	DB	MUPIP LOAD reports 0 records loaded on an empty ZWR or GO format extract file
(#96)	DB	AIMG records include DSE CHANGE command effecting the change
(#113)	Admin	ydb_lct_stdnull environment variable
(#114)	Admin	GDE defaults to standard null collation
(#183)	Language	ydb_incr_s() accepts NULL ret_value parameter
(#191)	DB	Force access through GNP server even when client and server are on the same host
(#230)	Language	$ZSEARCH() context of -1 is always new
(#253)	Language	Attempting to update a database mapped through multiple global directories and for which a process does not have write access issues a DBPRIVERR
(#255)	Admin	ydbinstall script checks for Supported or known Supportable platform and issues a warning otherwise
(#264)	Other	Source Server issues FILTERNOTALIVE when external filter program terminates abnormally
(#267)	Language	MUPIP BACKUP issues a FILENAMETOOLONG error when the absolute path length of the target backup filename is too long
(#272)	Admin	MUMPS, DSE, LKE, MUPIP BACKUP etc. honor the ydb_poollimit environment variable
(#275)	Other	LISTENING sockets can be passed between processes
(#277)	Language	$$set^%PATCODE works in UTF-8 mode
(#280)	Other	WRITE /WAIT on a SOCKET device with no sockets returns immediately
(#282)	DB	Source Server clears as much backlog as possible when reading from journal files of instance frozen because of errors in journal file switching
(#284)	DB	MUPIP JOURNAL -EXTRACT on concurrently updated journal file works
(#290)	DB	During MUPIP FREEZE -ONLINE, exiting YottaDB processes detach from database files without hanging
(#293)	Admin	SET $ZGBLDIR inside a Update Process trigger correctly switches global directory
(#295)	Language	ZSTEP inside trigger ignored completely if TRIGGER_MOD restriction is in place
(#297)	DB	LOCK commands work correctly in a rare case of multiple hash collisions
(#309)	Admin	ydb_env_set leaves gtm_prompt untouched
(#310)	Other	DEVOPENFAIL messages in the syslog correctly identify the device name and system error number in case of errors during OPEN of a PIPE device
(#312)	DB	Fixes to issues affecting processes part of multiple replication instances
(#313)	DB	MUPIP FTOK -JNLPOOL and -RECVPOOL recognize instance-file-name on command line
(#315)	Language	ZCOMPILE omits warnings during compilation if $ZCOMPILE includes "-nowarning"
(#318)	Other	Changes to ZSYSTEM_FILTER/PIPE_FILTER in restrict.txt accepted even when within one second
(#321)	Language	Journal records fed to replication filters include timestamps
(#324)	Language	Errors inside indirection usage while in the direct mode using $ETRAP return control to the direct mode prompt
(#329)	Language	Compiling M program with literal optimization in UTF-8 mode works correctly
(#333)	Other	$VIEW("PROBECRIT") CPT time resolution is nanoseconds
(#338)	Admin	Improvements to the ydbinstall.sh script
(#341)	DB	Journal updates separated in time by more than an epoch interval guaranteed to have an EPOCH journal record between them
(#344)	Language	Simple API calls subsequent to a ydb_zwr2str_s() call work correctly
(#345)	Other	OPEN of a PIPE device with the STDERR deviceparameter set to the name of an already-open device issues a STDERRALREADYOPEN error
(#346)	Other	Certain cases of invalid collation specification in directory tree records handled correctly
(#347)	Admin	ydbinstall.sh checks at startup for all utilities it needs
(#348)	DB	OPEN of SOCKET device previously closed after TPTIMEOUT works correctly
(#349)	DB	MUPIP REORG on a database file with non-zero RESERVED_BYTES retains structural integrity of file
(#350)	Admin	Change terminal characteristics only during READ or Direct Mode
(#351)	Language	API for Multi-threaded applications to use the single-threaded YottaDB engine
(#352)	Other	Unhandled error from C to M call-in is returned to caller after Simple API call that sets a spanning node
(#353)	Language	$INCREMENT() supported for a global that has NOISOLATION turned on
(#354)	Language	$ZSTATUS reports correct actual and maximum string length following ERR_INVSTRLEN return from Simple API call
(#356)	Language	Extended global reference correctly reports NETDBOPNERR error when $zgbldir / $ydb_gbldir not set
(#357)	Admin	SIGINT/SIGQUIT (kill -2 or kill -3) handled when sent to a mumps process waiting for spawned process to complete
(#358)	DB	Avoid DBIOERR error in forked child process when using simpleAPI in database file using ASYNCIO
(#360)	Language	$ZEDITOR reflects exit status of the last ZEDIT
(#361)	Other	Avoid SI replication REPLINSTNOHIST after Receiver Server shutdown before any data is replicated
(#362)	DB	Ensure logical consistency between primary and secondary instances after a switchover following kill -9 of processes on secondary
(#363)	Language	Issue NUMOFLOW for overlarge numeric expressions encountered during literal optimization
(#364)	DB	For a frozen instance Source Server shutdown reports whether or not it deleted the Journal Pool ipcs
(#365)	DB	Use prior journal type in MUPIP SET JOURNAL; default depending on access method
(#369)	Admin	VIEW "SETENV" and VIEW "UNSETENV" commands to set and unset environment variables in M
(#371)	Language	$SELECT() stops evaluating tvexprs or exprs after the first true tvexpr is found
(#372)	Language	Report syntax error for SET command where first setleft is an invalid ISV and later setleft uses indirection
(#374)	Language	Ensure $TEXT() strips line terminators for object code compiled with -EMBED_SOURCE flag
(#375)	Other	Drop support for SSL 2.0, SSL 3.0 and TLS1.0 in the reference TLS implementation shipped with YottaDB
(#376)	Other	Support TLS 1.3 in the reference TLS implementation that ships with the encryption plugin
(#383)	Language	ydb_tp_s() returns negative GBLOFLOW error code
(#385)	Admin	Recompile .m file if .o has same timestamp
(#392)	Admin	Environment variables ydb_linktmpdir/gtm_linktmpdir default to ydb_tmp/gtm_tmp
(#393)	Admin	YDB_RELEASE in libyottadb.h specifies YottaDB release
(#394)	Language	ydb_subscript_next_s() for nonexistent local returns next subscript "0"
(#395)	Admin	/tmp/yottadb/$ydb_ver has read-write-execute permissions for all users permitted to run YottaDB
(#396)	Language	ydb_node_next_s() and ydb_node_previous_s() report PARMINVALID when output ydb_buffer_t structure has NULL buf_addr
(#397)	Admin	Access through SimpleAPI issues ZGBLDIRACC if ydb_app_ensures_isolation is set to a non-null value but global directory does not exist
(#401)	Language	Correct behavior when a call-in invokes an external call that uses Simple API
(#402)	Language	gtm_init(), gtm_ci() and gtm_cip() entry points for backward compatibility with GT.M

Database

• MUPIP LOAD reports 0 records loaded on an empty ZWR or GO format extract file. Previously, it reported MAXSTRLEN and LDBINFMT errors. [#95]

• AIMG records (After-Images of database blocks) reported by MUPIP JOURNAL -EXTRACT -DETAIL include the DSE CHANGE command that caused the block state to change. Previously they did not. [#96]

• GDE CHANGE -SEGMENT -FILE and GDE ADD -SEGMENT -FILE accept a database file name specification of the form @: where is the local host name or a remote host name where the GT.CM GNP server runs. Any database access to such a region/segment by a client process now goes through the GNP server even if is the current host. The : syntax where a database access goes through a GNP server only if is not the current host remains supported. The optional "@" prefix allows a client to force every database access to go through a GNP server, thereby allowing client processes running different releases of YottaDB to access a database file. [#191]

• When an instance is frozen due to an error switching journal files (for example, file system permissions) a Source Server clears the backlog as much as possible and then waits for the instance freeze to clear. Previously, the Source Server could terminate with a FILEDELFAIL or RENAMEFAIL. [#282]

• MUPIP JOURNAL -EXTRACT on a journal file that is being concurrently updated creates an extract file reflecting the state of the journal file when the command started. Previously, the extract command could terminate abnormally with JNLBADRECFMT, JNLUNXPCTERR, and, rarely, GTMASSERT2 errors, sometimes with a core file. [#284]

• When a MUPIP FREEZE -ONLINE is active, exiting YottaDB processes detach from database files without hanging. Previously, they could hang. [#290]

• LOCK commands work correctly when more than 31 subscripts at a given lock name subscript level (across all processes concurrently using a particular database region) hash to the same value. $$^%PEEKBYNAME("sgmnt_data.lock_hash_bucket_full_cntr",<region>) now returns the number of times such a rare event was seen (and handled correctly) in . Previously, effective YottaDB r1.20, it was possible for LOCK commands in this rare case to spin and incorrectly seize ownership of the lock resource from a concurrently holding process. [#297]

• Database file updates to a process that is part of multiple replication instances works correctly in several cases. Previously it was possible for the process to incorrectly

  • issue REPLINSTMISMTCH errors;
  • terminate with a fatal GTMASSERT2 error;
  • terminate with a fatal KILLBYSIGSINFO1/SIGMAPERR/SIG-11 error;
  • freeze the wrong instance;
  • cause a memory leak when using PEEKBYNAME on the Journal Pool; or
  • attach to a Journal Pool even though no update was attempted on a corresponding database file.

  [#312]

• MUPIP FTOK -JNLPOOL and MUPIP FTOK -RECVPOOL operate on the specified instance-file-name. Previously they ignored the input instance-file-name and always operated on the instance file name specified by the ydb_repl_instance or gtm_repl_instance environment variable. [#313]

• Two updates to the journal file are separated by n seconds where n is greater than the epoch_interval for that region, then are guaranteed to have at least at least one intervening EPOCH record in the journal file. Previously, this setting was not honored in some cases (e.g. when an idle/free epoch was written) resulting in delaying the EPOCH record by as much as 6 seconds. [#341]

• OPEN of a SOCKET device that was previously closed after a TPTIMEOUT error while in a READ command works correctly. Previously this would abnormally terminate the process with a GTMASSERT2 fatal error. [#348]

• MUPIP REORG on a database file with non-zero RESERVED_BYTES retains the integrity of the database file. Previously, it was possible for cause a structurally damaged database file (for example DBINVGBL integrity error). [#349]

• A child process forked from a parent using the Simple API to update a database file that has ASYNCIO turned on can successfully update that database file. Previously, such a child process raised a DBIOERR. [#358]

• MUPIP JOURNAL -ROLLBACK -BACKWARD -FETCHRESYNC keeps user data in sync between primary and secondary sides of a replication connection when processes on the secondary are terminated with kill -9 followed by a switchover. Previously, this could cause primary and secondary database instances to have different content. Note that YottaDB strongly recommends against use of kill -9. [#362]

• MUPIP REPLICATE -SOURCE -SHUTDOWN reports whether or not it deleted the Journal Pool shared memory and semaphores even if the instance is frozen. Previously, it reported this information only if the instance was not frozen. [#364]

• The -[NO]BEFORE_IMAGES option of the MUPIP SET JOURNAL command is optional. If neither option is specified and journaling is currently disabled in the database file header (i.e. DSE DUMP -FILEHEADER lists "Journal State" as "DISABLED"), MUPIP SET -JOURNAL defaults to BEFORE_IMAGES for databases with the BG access method, and to NOBEFORE_IMAGES for databases with the MM access method. If neither option is specified and journaling is not disabled in the database file header (i.e. "Journal State" is not "DISABLED"), the prior journal type from the database file header is used for the new journal file. Previously, one of BEFORE_IMAGES or NOBEFORE_IMAGES option was required. [#365]

Language

• ydb_incr_s() accepts a NULL ret_value parameter, slightly simplifying application programs and marginally improving performance for applications that need to increment a node, but do not need to use the value. [#183]

• A value of -1 as the optional intexpr second parameter of $ZSEARCH() always returns the first file name matched by the pattern searched for, i.e., it does not use a stream. Previously, a value of -1 returned a ZSRCHSTRMCT error. [#230]

• Attempting to update a database file for which a process does not have write access, and which is mapped through multiple global directories issues a DBPRIVERR error. In prior versions, such an attempted update would abnormally terminate the process with a KILLBYSIGINFO1 error (SIG-11). [#253]

• MUPIP BACKUP issues a FILENAMETOOLONG error when the absolute path length of the target backup filename is too long (approximately 255 bytes). Previously, this resulted in an abnormal termination of the backup process with a KILLBYSIGSINFO1 fatal error (signal 6/SIGABRT). [#267]

• $$set^%PATCODE loads user defined pattern tables in UTF-8 mode, returning 1 in case of a successful load. Previously it always failed in UTF-8 mode, returning 0. [#277]

• When the TRIGGER_MOD restriction is in place, ZSTEP is completely ignored. Previously it would be executed on completion of the trigger code. [#295]

• ZCOMPILE omits warnings during compilation if $ZCOMPILE includes -nowarning. Previously, the compilation incorrectly issued warnings even with this setting. Note that an application can set $ZCOMPILE directly or through the environment variables ydb_compile/gtmcompile. [#315]

• Journal update records in MUPIP JOURNAL EXTRACT format fed to replication filters include time stamps. Previously the time stamps were zero. [#321]

• Errors inside indirection usage (@ syntax) while in the direct mode using $ETRAP return control to the direct mode prompt after executing the $ETRAP error handler. Previously, such errors caused the mumps process to abruptly terminate. [#324]

• Compiling M programs with $select() containing invalid $char() usages in UTF-8 mode correctly issues a INVDLRCVAL error. Previously, in some cases where the M line could be optimized (due to literal usage in the $CHAR() function which corresponded to an invalid codepoint), the compilation would abnormally terminate with a GTMASSERT2 fatal error. [#329]

• Simple API calls subsequent to a ydb_zwr2str_s() call work. Previously, Simple API calls subsequent to a ydb_zwr2str_s() call would fail with a SIMPLEAPINEST error. [#344]

• Even though the YottaDB data management engine is single-threaded, YottaDB has Simple API functions to support multi-threaded applications. An overview is in the blog post YottaDB Support for Multi-threaded Applications and details are in the Multi-Language Programmers Guide. [#351]

• $INCREMENT() is supported for global variables that have NOISOLATION turned on. Previously this reported GVINCRISOLATION errors, albeit not consistently. Note that for variables that have NOISOLATION turned on YottaDB expects application logic to ensure Isolation within a transaction without requiring YottDB to do so. [#353]

• $ZSTATUS reports the correct actual and maximum string length values in all cases following a YDB_ERR_INVSTRLEN error returned by a Simple API function call e.g. ydb_get_s(). Previously, $ZSTATUS could contain a 0 string length and max length value in the INVSTRLEN. [#354]

• An extended reference correctly issues a NETDBOPNERR error if it points to a remote database file that cannot be opened (for example, the GNP server is not started on the remote host). Previously, if $zgbldir was not set (or the ydb_gbldir / gtmgbldir environment variables were not set), a similar extended reference would terminate with a KILLBYSIGSINFO1 fatal error (SIG-11). [#356]

• $ZEDITOR reports the exit status of the last edit session invoked by a ZEDIT command. Previously, it reported 0 even if the editor invocation exited with a non-zero status. [#360]

• YottaDB issues NUMOFLOW errors for literal expressions which contain large numeric values stored as strings. Previously, in some cases (e.g., 0!("1E47"*10))YottaDB would return potentially incorrect values rather than issue a NUMOFLOW error. [#363]

• $SELECT stops evaluating tvexprs or exprs once a true tvexpr is encountered even if evaluating later tvexprs or exprs would result in a NUMOFLOW or INVDLRCVAL errors. Previously a NUMOFLOW or INVDLRCVAL runtime error would be incorrectly issued (e.g., if $select(1:1,1:1E47)). [#371]

• Indirection in a setleft correctly generates a compilation error if it is preceded by a setleft which has an invalid ISV usage (e.g. set ($iv,@y(2))=1). Previously such a usage would cause process termination with a fatal GTMASSERT2 error during compilation. [#372]

• $TEXT() returns source lines without line terminators. Previously, if a .m source file had line terminators and was compiled with the -EMBED_SOURCE compilation flag, $TEXT() returned source lines with line terminators in some cases and without them in others. [#374]

• ydb_tp_s() returns a negative error code (YDB_ERR_GBLOFLOW) in case of a GBLOFLOW error. Previously it incorrectly returned the positive value of this error code (i.e. -YDB_ERR_GBLOFLOW). [#383]

• ydb_subscript_next_s() and ydb_subscript_previous_s() for a nonexistent local variable return the null string (i.e. ret_value.len_used is 0) as the next subscript. Previously, it used to incorrectly return the next subscript as the string literal "0" (i.e. ret_value.len_used is 1). [#394]

• ydb_node_next_s() and ydb_node_previous_s() report a PARAMINVALID error when a ydb_buffer_t structure passed via the *ret_subsarray parameter has a NULL buf_addr member. In this case, *ret_subs_used is the index into the *ret_subsarray array where the NULL buf_addr was encountered. Previously this usage caused the process to terminate with a fatal KILLBYSIGSINFO1 error (SIG-11). [#396]

• Application code that calls in to M code (using ydb_ci() / ydb_cip()) which in turn calls out to code that in turn calls the Simple API works. Previously, such usage could result in errors and otherwise behave incorrectly. [#401]

• Routines gtm_ci(), gtm_cip(), and gtm_init() are available for C code to call M code providing upward compatibility to allow non-M applications using FIS GT.M to run unchanged with YottaDB. Previously, while YottaDB provided upward compatibility for M applications, non-M applications had to replace the calls with ydb_ci(),ydb_cip(), and ydb_init() respectively. [#402]

Other

• The Source Server issues a FILTERNOTALIVE error in case an external filter (invoked with the -FILTER qualifier at process startup) terminates abnormally. Previously, the Source Server could loop indefinitely waiting for a response from the terminated filter process. [#264]

• Sockets in LISTENING state can now be passed to other processes using a JOB command or the WRITE /PASS with WRITE /ACCEPT. Previously this used to hang. [#275]

• WRITE /WAIT on a SOCKET device with no sockets returns immediately. Previously it would loop until the command timed out (indefinitely if there was no timeout). [#280]

• DEVOPENFAIL messages in the syslog correctly identify the device name and system error number in case of errors during OPEN of a PIPE device. Previously, reported incorrect (garbage) device name and/or system error number/text in some cases. [#310]

• With GT.M V6.3-005, one can specify M code to run a filter that decides whether to allow a restriction or not and to control what commands actually get run when a user specifies ZSYSTEM or a PIPE OPEN command (see GTM-8877 for details). While testing this feature out, we noticed that if the $ydb_dist/restrict.txt or $gtm_dist/restrict.txt file is updated multiple times within the same second, where the first update included one M labelref for ZSYSTEM_FILTER/PIPE_FILTER and the second update included a different M labelref, the second (and later) M labelref did not take effect when a ZSYSTEM or PIPE OPEN command ran afterwards. The first M labelref still took effect incorrectly. [#318]

• $VIEW("PROBECRIT") CPT time resolution is nanoseconds. Previously, it was reported in nanoseconds, but it was a multiple of 1000, effectively making it microsecond resolution. [#333]

• OPEN of a PIPE device with the STDERR deviceparameter set to the name of an already-open device issues a STDERRALREADYOPEN error. Previously such usages resulted in unexpected device state and behavior (e.g. unexpected errors). [#345]

• When a directory tree record has an invalid collation specification (a structural error), MUPIP INTEG issues an INVSPECREC error with appropriate context (block number and offset where the error was noticed), while DSE and application programs assume no collation and proceed. Previously, in some cases of an INVSPECREC error, it was possible for all the above commands to loop for ever. [#346]

• A C→M call-in that encounters an error which is not handled by the error trap handler ($ETRAP or $ZTRAP) returns the error code back to the calling C code. Previously, in this situation, the process could terminate with a GTMASSERT2 fatal error if the call-in was preceded by a ydb_set_s() Simple API call that set a node in the database spanning more than one database block. [#352]

• A Receiver Server on a Supplementary Instance starts replicating after a prior Receiver Server was shutdown after it connected to a Source Server for the first time but before it actually received any data. Previously, a receiver startup in this case could incorrectly terminate with a REPLINSTNOHIST error. [#361]

• The encryption plugin shipped with YottaDB, which uses OpenSSL for TLS for replication and SOCKET devices) does not support SSL 2.0, SSL 3.0 and TLS 1.0 as these are vulnerable to cryptographic attacks (e.g., POODLE). The plugin continues to support the TLS 1.1 and TLS 1.2 protocols. [#375]

• The reference implementation of the encryption plugin supports TLS 1.3 (which in turn is supported by OpenSSL 1.1.1). [#376]

System Administration

• If the environment variable ydb_lct_stdnull environment variable has the value 1, YottaDB uses standard collation for empty string (null) subscripts of local variables, i.e., the empty string collates first, before all numeric values. If set to 0, YottaDB uses the legacy collation, where the empty string collates between numeric and non-empty string subscripts. If the environment variable is not set, YottaDB uses the gtm_lct_stdnull environment variable, and if neither is set, YottaDB uses standard collation. Previously, the ydb_lct_stdnull environment variable was not recognized, and the default of legacy collation (which is not used by any application we know of) was an opportunity for an application to have an obscure, hard to troubleshoot, bug. YottaDB strongly recommends against use of legacy collation. [#113]

• The default value for the GDE region parameter STDNULLCOLL is STDNULLCOLL. Previously it was NOSTDNULLCOLL. As we are aware of no application that uses empty string (null) subscripts and NOSTDNULLCOLL, changing the default is an opportunity to keep an application from having obscure, hard-to-troubleshoot bugs as a consequence of inadvertently using NOSTDNULLCOLL. Note that YottaDB strongly recommends against use of legacy collation. (#114 (closed))

• The ydbinstall script checks for existence and contents of /etc/os-release, and aborts the install of YottaDB if it detects an unsupported platform. Rerunning it with --force-install overrides this behavior and goes ahead with the install even if it is an unsupported platform. [#255]

• MUMPS, DSE, LKE, MUPIP BACKUP etc. honor the ydb_poollimit environment variable and set the poollimit value at process startup. Previously only MUPIP REORG honored this environment variable. In MUMPS, VIEW "POOLLIMIT" continues to be honored and overrides the poollimit setting initially set from the environment variable. [#272]

• The Update Process operates correctly when a trigger executes a SET $ZGBLDIR to switch the current global directory. Previously it switched the global directory correctly the first time the trigger was invoked but later invocations of the trigger did not switch the global directory resulting in any updates in the trigger code unintentionally happening in the primary global directory with which the Update process was started. A workaround in prior versions was to do a NEW $ZGBLDIR before the SET $ZGBLDIR in the trigger code. [#293]

• When sourced, the ydb_env_set file does not clear the value of the environment variable gtm_prompt. Previously it did. The workaround was to use the ydb_prompt environment variable. As the gtm_prompt environment variable is deprecated, YottaDB recommends using ydb_prompt except in cases where compatibility with GT.M must be maintained. This also preserves ANSI escape sequences included in the ydb_prompt/gtm_prompt environment variables (for example one can use this to color the direct mode prompt). Previously such usages could cause a "bad variable name" error (reported as Issue 348 on GitHub). [#309]

• The ydbinstall.sh script now issues an error when an option has been specified with no value following it. For example, ydbinstall.sh --installdir --utf8 default. In this case, --installdir needs to be followed by a directory name but that was missed out in the specification. Previously, --utf8 (because that parameter immediately follows --installdir in the command line) was assumed to be the directory where the install was desired and this led to confusing results. Additionally, if a relative path is specified as the install directory name, the script now creates a subdirectory under the current directory at the time when the script was invoked and installs YottaDB there. Previously, it used to install YottaDB in a subdirectory under /tmp, later remove the entire directory and yet confusingly display a message indicating the install was successful. Finally, the geteuid executable is no longer part of the YottaDB install. Its sole purpose was to get the effective user id at the time of installing YottaDB and this is easily obtained without a standalone executable. If you have any scripts that use getuid, please replace it with id -un. [#338]

• The ydbinstall.sh script checks at the outset for the existence of all utilities that it relies on (e.g. tar, gzip, grep etc.) and exits with a clear error message if any of them do not exist. Previously, it reported hard to diagnose errors at the first point of failure from a missing utility program. [#347]

• YottaDB changes terminal characteristics internally only for the duration of a READ command or while interacting in Direct Mode. Previously, YottaDB changed terminal characteristics at process startup and restored them at process exit, which meant that non-M code reading from a terminal used by M code might not behave as expected by the non-M code. [#350]

• SIGQUIT/SIGINT signals (generated by kill -3 and kill -2 respectively) are correctly handled by YottaDB processes. Previously, these signals were ignored if the target process had spawned another process (for example using the $ydb_procstuckexec) and was waiting for it to finish. [#357]

• Environment variables can be set and unset inside M using VIEW commands. VIEW "SETENV":"envvar":value sets the environment variable named envvar to value and VIEW "UNSETENV":"envvar" unsets the environment variable envvar. [#369]

• YottaDB recompiles a .m source file to generate a new .o object file if a source file and its corresponding object file (as matched by $zroutines) have the same time stamp. Previously, YottaDB recompiled only if the timestamp of the matching object file was older than than of the source file. To avoid unnecessary recompilation, YottaDB ensures that a generated object file has a later timestamp than its source file. Previously, it was possible for both to have the same timestamp. [#385]

• If the environment variables ydb_linktmpdir and gtm_linktmpdir are unspecified, but ydb_tmp or gtm_tmp are, YottaDB uses the latter for the location of the relinkctl control files for auto-relink enabled directories. Previously, owing to a regression effective GT.M V6.3-002 (and in turn YottaDB r1.10), in this case, the relinkctl control files were created in /tmp. [#392]

• YDB_RELEASE is a numeric constant defined in libyottadb.h (e.g., 124 for YottaDB r1.24) that C code accessing YottaDB can use for conditional compilation. [#393]

• The permissions of the temporary directory /tmp/yottadb/$ydb_ver (e.g., /tmp/yottadb/r1.24_x86_64) created by ydb_env_set are read-write-execute for user, group and world if YottaDB is installed to be world-executable, and read-write-execute for the user, and the group permitted to run YottaDB, if YottaDB is installed to allow only members of a group to execute it. Previously it was read-write-execute only for the user, which while appropriate for a single-user environment, is not well suited to multi-user systems. [#395]

• Accessing through SimpleAPI functions issues a ZGBLDIRACC error if the ydb_app_ensures_isolation environment variable is set to a non-null value but the global directory file does not exist. Previously, this abnormally terminated the process with a KILLBYSIGSINFO1 error (SIG-11). [#397]

GT.M V6.3005

The GT.M release notes, edited where appropriate for YottaDB, follow the table. Each of those release notes has a link to the original GT.M release note.

ID	Category	Summary
(GTM-3659)	Admin	Improved MUPIP LOAD error messaging
(GTM-4647)	Admin	The ydb_mstack_size environment variable specifies M stack size in KiB
(GTM-5059)	Admin	ydb_mstack_crit optionaly specifies the percentage of the stack at which YottaDB should produce a STACKCRIT
(GTM-5574)	Other	Percent conversion routines handle larger numbers
(GTM-7960)	Admin	Warnings of approach to maximum database file size
(GTM-8836)	Admin	Account for MUPIP journal rollback multi-threading in hash table condition handler
(GTM-8875)	Other	More complete detection of STACKOFLOW
(GTM-8877)	Admin	Allow user to create M filters for the commands passed to zsystem and PIPE
(GTM-8910)	DB	Prevent disruption of $ORDER(gvn,1) by MUPIP REORG
(GTM-8930)	Language	Improvement to return values from $VIEW("JNLPOOL")
(GTM-8940)	DB	Performance improvement for ftok semaphores
(GTM-8941)	Admin	LKE recognizes the full keyword for the -CRITICAL qualifier
(GTM-8942)	Other	Reduce impact of signal management
(GTM-8943)	Language	ZGOTO 0 in a call-in returns to the invoking C code
(GTM-8945)	Admin	Prevent Receiver hang after killing an Update Process
(GTM-8949)	Other	Prevent recursive calls to system memory allocator
(GTM-8951)	Language	$TEXT() and ZPRINT use auto-relink when their argument includes a routinename
(GTM-8953)	Admin	ROLLBACK FORWARD with -VERIFY properly applies all valid updates
(GTM-8954)	Admin	Source Server sends all errors to the log file
(GTM-8955)	Other	Relink Locking Speedup
(GTM-8956)	Other	-NOWARNING compiler qualifier suppresses more messages
(GTM-8957)	Admin	MUPIP SET -NOREADONLY -ACC=BG works in one command
(GTM-8958)	Admin	TLS reference implementation plug-in disables SSLv3 by default
(GTM-8959)	Language	For 64-bit Linux, fix ZGOTO 0 within a call-in
(GTM-8962)	Language	Clean up ZSHOW "i" output
(GTM-8964)	Admin	MUPIP respects -READONLY (MM) database state
(GTM-8965)	Language	Alternation pattern with a large match produces a PATALTER2LARGE error
(GTM-8967)	DB	Correct reporting of globals reported by TPRESTART
(GTM-8969)	DB	When encrypting for the first time, prevent spurious CRYPTOPFAILED errors
(GTM-8973)	Other	External call table loading more careful about M labelrefs
(GTM-8974)	DB	Improved handling of significant IO pauses in Database or Journal I/O
(GTM-8980)	Language	VIEW and $VIEW() fixes, particularly related to region arguments
(GTM-8981)	Language	Fix to IF @<literal>
(GTM-8985)	Language	Prevent possible incorrect result from a function with all literal arguments in a Boolean expression
(GTM-8988)	Admin	MUPIP RESTORE handling of out-of-space
(GTM-8989)	Admin	Prevent unusual MUPIP RUNDOWN hang
(GTM-8990)	Language	ZRUPDATE treats a cycle of symbolic links like a simple missing routine
(GTM-8992)	DB	YottaDB never decrements database statistics for $DATA(), $GET(), $ORDER(), or $QUERY()
(GTM-8996)	Admin	YottaDB issues an fsync on the database file after a region freeze goes into effect
(GTM-9001)	Other	Improved bounds checking in DSE and MUPIP

Database

• $ORDER(gvn,-1) and $ZPREVIOUS(gvn) appropriately handle the concurrent move of the global variable tree root block of the gvn by MUPIP REORG; previously, this concurrence could occasionally produce a GVORDERFAIL with a status code of tSSS. (GTM-8910)

• YottaDB avoids superfluous public (ftok) semaphore control operations on database startup or rundown. (GTM-8940)

• TPRESTART messages correctly identify the global causing the conflict; previously they could report an incorrect global name. (GTM-8967)

• YottaDB processes work correctly when a concurrent MUPIP REORG -ENCRYPT encrypts a previously unencrypted database file. Previously YottaDB processes could fail with CRYPTOPFAILED errors unaccompanied by other errors explaining the reason for the failure. This issue was only observed in the development environment, and was never reported by a user. (GTM-8969)

• YottaDB processes attempting to acquire a shared database resource and encountering long delays continue to wait if the process holding the resource is still active. Previously they intervened prematurely which could cause DBDANGER errors and accompanying database damage. YottaDB processes attempting to acquire a shared resource required to perform journal writes and encountering an extended delay in acquiring the resource, issue JNLSENDOPER/JNLFLUSH/JNLPROCSTUCK messages to the system log, and, if so configured, freeze the instance as a result, sending REPLINSTFROZEN/REPLINSTFREEZECOMMENT messages to the system log. They continue to attempt the journal writes, removing the instance freeze if they eventually succeed. Previously, such processes received an error, and the instance required a operator action to release the freeze. (GTM-8974)

• YottaDB does not decrement database statistics for $DATA(), $GET(), $ORDER(), or $QUERY(). Note that, to minimize performance impact, YottaDB updates most database statistics without concurrency controls, so the statistics are approximate and may have brief fluctuations. Previously YottaDB made explicit adjustments which reporting was more likely to detect. MERGE operations increment DATA and GET database statistics to more accurately reflect database activity. Previously, Merge suppressed increments to those statistics under certain conditions.(GTM-8992)

Language

• If the process has access to an instance file designation, $VIEW("JNLPOOL") returns its name; if the process has not yet opened the pool, the return contains an asterisk after the name. If the process does not have sufficient information to determine a replication journal instance file, the function returns "No replication instance defined." Previously, the function with this argument returned an empty string if the process had not opened the journal pool. (GTM-8930)

• A ZGOTO 0 in case of call-in, unwinds all the M stack frames and returns to the invoking C routine. In case of a non-CI invocation, this terminates the process. A ydb_ci invocation without calling ydb_init() first, proceeds with the CI invocation. Previously, this produced a segmentation violation (SIG-11) error. This issue was only observed in the development environment, and was never reported by a user. (GTM-8943)

• With auto-relink enabled, $TEXT() and ZPRINT use auto-relink when their argument includes a routinename. Note that ZBREAK does not use auto-relink because its intended purpose is as a debugging facility and if the user intends the ZBREAK for a new version they can explicitly request it with a ZLINK. Previously, $TEXT() and ZPRINT used the version most recently linked by the process even if a new version had become available in an auto-relink enabled environment. (GTM-8951)

• ZGOTO 0 in a call-in returns to the invoking program. Previously on x86-64 Linux systems with glibc 2.24 or later, this generated a segmentation violation (SIG-11).(GTM-8959)

• ZSHOW "I" outputs $ZPIN and $ZPOUT even if they are the same as $PRINCIPAL and no longer displays $ZPROCESS, as it was only meaningful on OpenVMS; also, $ZTNAME appears in proper alphabetic sequence. Previously if $ZPIN or $ZPOUT matched $PRINCIPAL, ZSHOW "I" omitted them, $ZPROCESS appeared as an empty string and $ZTNAME was not ordered appropriately. (GTM-8962)

• Pattern match of a string with an alternation match exceeding what YottaDB can handle produces a PATALTER2LARGE error; previously this condition produced a segmentation violation (SIG-11). (GTM-8965)

• VIEW and $VIEW() with a empty string or inappropriate region-list works appropriately; in r1.22, these could cause inappropriate results, including a segmentation violation (SIG-11). $VIEW("statshare") returns a 0 when the process has sharing disabled, a 1 when it has sharing enabled and a 2 when sharing is selectively enabled. In r1.22, it did not differentiate between the all and selective cases and returned 1 when sharing was disabled and selective disabling was also specified. $VIEW("statshare","<region>") works appropriately even if the region had been selectively disabled when full sharing is disabled and the region had not been opened. In r1.22, this set of conditions produced a segmentation violation (SIG-11). The error messages when invalid parameters are passed to VIEW/$VIEW() print the name of the parameter; previously such error messages did not have the name of the parameter. (GTM-8980)

• IF @<literal> works correctly when the literal evaluates to FALSE; previously, it tended to fail with an inappropriate INDEXTRACHARS error or a segmentation violation (SIG-11). (GTM-8981)

• The compiler appropriately handles possible string returns into Boolean expressions from functions with all literal arguments; previously, it could produce an incorrect result. The workaround was to avoid all literal arguments for $CHAR(), $EXTRACT(), $PIECE(), their $Z*() variants and $SELECT() when they appeared in Boolean expressions. (GTM-8985)

• When ZRUPDATE encounters a cycle of symbolic links without finding a specified routine, it treats the cycle the same as a simple routine not found and ignores the missing routine. Previously, the command issued an error with text about "too many levels of symbolic links". This issue was observed in the development environment after upgrading to newer Linux distributions. (GTM-8990)

System Administration

• MUPIP LOAD reports ranges of records not loaded due to missing database files; previously it reported an error for every such record. (GTM-3659)

• YottaDB supports specifying the M stack size in KiB with the ydb_mstack_size environment variable. No setting or a setting of 0 uses the default (272KiB). The minimum supported size is 25 KiB; YottaDB reverts values smaller than this to 25 KiB. The maximum supported size is 10000 KiB; YottaDB reverts values larger than this to 10000 KiB. (GTM-4647)

• YottaDB recognizes setting the environment variable ydb_mstack_crit_threshold to specify an integer between 15 and 95 defining the percentage of the stack which should be used before YottaDB emits a STACKCRIT warning. If the value is below the minimum or above the maximum YottaDB uses the minimum or maximum respectively. The default is 90. (GTM-5059)

• An automatic database file extension, MUPIP EXTEND, INTEG and SIZE all put a message in the system logs when the database reaches 88% of its maximum size. Beyond this 88% threshold, manual and automatic extends only report at 1% intervals, but INTEG and SIZE report at every subsequent invocation while the condition persists. All but the automatic extension also produce the message for the operator. Previously, YottaDB gave no such warnings, which made it necessary to proactively check on database size. (GTM-7960)

• When a multi-threaded instance of MUPIP journal -rollback runs out of memory during a rehashing operation, child threads transfer error-handling to the parent thread and terminate. Previously, child threads occasionally failed to report the error, causing MUPIP to halt without a descriptive error message. This issue was only observed in the development environment, and was never reported by a user. (GTM-8836)

• The YottaDB restriction mechanism recognizes the following lines:

  ZSYSTEM_FILTER[:M labelref]
  PIPE_FILTER[:M labelref]

  The labelref must include a routine name. If a process is restricted by a ZSYSTEM or PIPE_OPEN line in the restrictions file that restriction takes precedence over the corresponding filter restriction. Otherwise when a process is subject to these restrictions, YottaDB inserts an invocation of the labelref prior to the restricted command, passing a string containing the argument to the ZSYSTEM command or the command deviceparameter of the PIPE OPEN. The path to the filter routine must be included in $zroutines. YottaDB recommends that the filter routine is placed in a location with restricted access such as $ydb_dist. If the filter invocation return is -1, YottaDB produces a RESTRICTEDOP error, otherwise it executes the command using the returned string via output parameters as a, possibly identical, replacement for the original string. Since YottaDB uses the call-ins mechanism to execute the filters, a filter invocation inside a TP transaction in call-ins produces a CITPNESTED error. Note that because ZSYSTEM and OPEN are not Isolated actions YottaDB recommends against their use within a TP transaction. Filters also increment the nested level of call-ins. A recursive filter invocation produces a NOFILTERNEST error. YottaDB reports all filter errors to the operator log accompanied by a COMMFILTERERR.

  The M labelref is mandatory. If it is not specified a syntax error is assumed (which in turn enables all restrictions unconditionally).

  An example restrict file for this:

  cat $ydb_dist/restrict.txt
  
  ZSYSTEM_FILTER:^filterzsy
  PIPE_FILTER:^filterzsy

  The actual filter routine:

  filterzsy(inarg,outarg);
   if ""=inarg set outarg="-1;must provide a command" quit
   for i=1:1 set arg=$piece(inarg,";",i) quit:""=arg  do  quit:$data(outarg)
   . for  quit:$zchar(9,32)'\[$extract(arg)  set arg=$extract(arg,2,9999)
   . set cmd=$piece(arg," ")
   . for restrict="sudo","cd" if cmd=restrict set outarg="-1;command "_restrict_" not permitted" quit
   . quit:$data(outarg)
   . if "echo"=cmd set $piece(arg," ")="echo #",$piece(inarg,";",i)=arg    ;example of modification
   set:'$data(outarg) outarg=inarg
   quit +outarg

  Filter execution starts with $STACK=1 ($ZLEVEL=2).

  Following are the YottaDB commands, Intrinsic Special Variables, and functions whose behavior changes in the context of a filter invocation.

  ZGOTO 0 (zero) returns to the processing of the restricted command as does ZGOTO 1 (one) with no entryref, while ZGOTO 1:entryref replaces the originally invoked filter and continues filter execution.

  $ZTRAP/$ETRAP NEW'd at level 1.

  $ZLEVEL initializes to one (1) in GTM$CI, and increments for every new stack level.

  $STACK initializes to zero (0) in GTM$CI frame, and increments for every new stack level.

  $ESTACK NEW'd at level one (1) in GTM$CI frame.

  $ECODE/$STACK() initialized to the empty string at level one (1) in GTM$CI frame.

  After the filter completes, YottaDB restores the above to their values at the invocation of the filter. (GTM-8877)

• LKE recognizes the full keyword for the -CRITICAL qualifier; previously it only accepted -CRIT. (GTM-8941)

• The Receiver Server process recovers after its Update process was terminated with a signal while idle. Previously, on Linux systems with glibc 2.25 or newer, the Receiver Server process could hang indefinitely, requiring manual cleanup of the process and shared memory. This issue was only observed in the development environment, and was never reported by a user. (GTM-8945)

• MUPIP JOURNAL -ROLLBACK -FORWARD when executed with -VERIFY properly applies all updates. Previously updates that should have been applied to the database were instead sent to the lost transaction file. The workaround was to execute MUPIP JOURNAL -VERIFY independent of the FOWARD ROLLBACK. (GTM-8953)

• The Source Server directs errors to the Source Server log file. A previous fix with GTM-8576 was incomplete and could result in TLS initialization error messages not logged to the server log file. (GTM-8954)

• Executing MUPIP SET -NOREAD_ONLY -ACC=BG on a read-only database sets the access mode to BG and turns off read-only. Previously, this action would result in a READONLYNOBG error message and no changes to the file. (GTM-8957)

• The TLS reference implementation plug-in disables SSLv3 by default. Previously, customers wishing to disable SSLv3 needed to add the configuration option 'ssl-options: "SSL_OP_NO_SSLv3";' to the "tls" namespace in the $ydbcrypt_config configuration file. (GTM-8958)

• MUPIP does not modify -READONLY (MM) database files; previously various MUPIP commands could inappropriately update state information in such database files, causing errors when subsequently using the database. (GTM-8964)

• MUPIP RESTORE exits with an error when it encounters an out-of-space condition. Previously, if MUPIP RESTORE encountered an out-of-space condition, it crashed with a segmentation violation (SIG-11). This issue was only observed in the development environment, and was never reported by a user. (GTM-8988)

• MUPIP RUNDOWN works as expected. In rare situations, processes killed during transaction commits could leave the transaction information in an inconsistent state that caused MUPIP RUNDOWN to hang. Note that YottaDB strongly recommends against kill -9 of a process accessing the database as it can induce database damage. This issue was only observed in the development environment, and was never reported by a user. (GTM-8989)

• YottaDB issues an fsync on the database file after a region freeze goes into effect, which forces the underlying file system to harden changes to secondary storage. (GTM-8996)

Other

• The %DH,%HD,%OH,%HO,%OD, and %DO routines now handle conversions with numbers up to YottaDB's maximum string length in size. Additionally, %DH and %DO routines now handle the conversion of negative decimal numbers properly, even when the specified length is not long enough to represent that fully converted number. Previously, the %DH,%HD,%OH,%HO,%OD, and %DO routines could only deal with numbers up to the equivalent of the maximum 18-digit decimal number. Attempting to convert a number larger than this would cause roundoff in the final result, which still occurs if they are used for arithmetic. Moreover, specifying a length for the %DH and %DO routines that was not long enough to hold the fully converted result would cause the routines to produce incorrect values. The workaround for both these issues was to avoid using these percent routines with numbers greater than the maximum 18-digit decimal number and to always specify an appropriate length for the %DH and %DO routines. (GTM-5574)

• When a stack frame exceeds the M virtual machine stack pointer, YottaDB issues a STACKOFLOW error and produces a YDB_FATAL_* context dump, but no core file, as this an application issue, rather than a YottaDB problem. A module with a very large number of variables and/or dynamic literals can cause this problem. Note that the $ydb_mstack environment variable can, within limits, set the size of the M virtual machine stack size. Previously if the overflow was sufficient to make the stack pointer negative, the process exited with a segmentation violation (SIG-11). (GTM-8875)

• YottaDB avoids superfluous signal management operations. Previously system traces showed more sigprocmask system calls than desirable, particularly on Linux systems with Meltdown/Spectre mitigation in place. (GTM-8942)

• YottaDB prevents recursive calls to the system memory allocator. Previously, the system memory allocator could be called recursively which resulted in a process hanging. This issue was only observed in the development environment, and was never reported by a user. (GTM-8949)

• YottaDB uses a lighter weight locking mechanism to protect the relink control file. Previously, a large number of concurrent relink control operations such as those done on process shutdown could cause the system to use an excessive amount of CPU. (GTM-8955)

• The YottaDB compiler -NOWARNING qualifier for the MUMPS command and $ZCOMPILE suppresses warning messages for BADCHAR, BLKTOODEEP, and LITNONGRAPH; previously it did not. (GTM-8956)

• YottaDB external call loading is more discriminating about M labelrefs; previously it did not detect all invalid labelrefs, which deferred error detection and made diagnosis more challenging. (GTM-8973)

• YottaDB appropriately bounds checks variable length input parameters to MUPIP and DSE. Previously it did not detect an off-by-one buffer overrun. Also, MUPIP EXTRACT issues the ICUNOTENABLED warning message when used with the -OCHSET qualifier while in M mode. Previously, attempting to use -OCHSET while in M mode resulted in a SIG-11. These issues were only observed in the development environment, and was never reported by a user. (GTM-9001)

More Information

(Section Blank for this Release)

Messages

From YottaDB

INVLNPAIRLIST, Invalid lockname/subscript pair list (uneven number of lockname/subscript parameters)

Runtime Error: This message comes only from the Golang wrapper. When specifying the locks to be acquired in the parameter list of LockE(), each lock is represented by a pair of arguments - a lock name and a list of subscripts. If the arguments aren't paired properly such that the last lock name read has no subscripts specified, this error is given.

Action: Determine why the parameter list is not paired up correctly and fix it.

INVSPECREC, pppp Invalid global modifier record

MUPIP Error: This indicates that MUPIP INTEG found a corrupt 4-byte collation record was found for a global variable (that is the 1st of the 4 bytes is not 1). pppp identifies the path in the directory tree (each element of the path consisting of a block#/offset) leading to the error.

Action: Use DSE to examine the corrupt record and fix it. Report the entire incident context to your YottaDB support channel.

INVTPTRANS, Invalid TP transaction - either invalid TP token or transaction not in progress

Runtime Error: This message comes from the threaded Simple API engine (both C and Golang). Each TP callback routine is given a 'tptoken' when it is driven. This token (unmodified!) must be used in all calls to the runtime from THIS TP callback routine at this level or this error message is returned.

Action: Determine why the tptoken value is incorrect and correct it.

NODEEND, End of list of nodes/subscripts

Runtime Error: This 'error' is indicator (like EOF) that the list of nodes or subscripts being fetched via ydb_node_next/ydb_node_previous or ydb_subscript_next/ydb_subscript_previous or their equivalent Golang wrappers, is at an end.

Action: Not an error strictly speaking - Terminate the loop the list is being read in as it is complete.

RECLOAD, Error loading record number: nnnn

MUPIP Error: This message identifies nnnn, a record or a range of records, that MUPIP could not LOAD and follows a message about the cause. If this message is Fatal, which it can be for BIN format, it produces a core file for diagnostic analysis.

Action: Address the cause or, for GO and ZWR format input files, examine the record or range of records with a text editor for possible correction or alternate action and for BIN format if fixing the cause does not resolve the error switch to ZWR format EXTRACT.

REGSSFAIL, Process pppp encountered error eeee contributing to the snapshot for region rrrr - the snapshot is no longer valid.

MUPIP Error: A YottaDB process encountered failure while opening snapshot file or attaching to shared memory or writing a block to the snapshot file, any of which invalidate the snapshot file. The original error eeee that process pppp encountered follows the REGSSFAIL error message and can also be found in the syslog (search for messages from process pppp).

Action: Examine the syslog for messages issued by process pppp to obtain details of the failure and take action, possibly by modifying file access characteristics or user roles, to address the problem.

SETENVFAIL, VIEW "SETENV":"eeee" failed in setenv() system call

Run Time Error: This indicates that a setenv() system call failed for the environment variable named eeee.

Action: Examine the accompanying SYSCALL error message which has more detail on the error returned by the setenv() call.

SIMPLEAPINOTALLOWED, Process cannot switch to using Simple API while already using threaded API.

Run Time Error: This indicates a process has started using the threaded Simple API functions (e.g. ydb_set_st()) and is now trying to use the Simple API functions (e.g. ydb_set_s()).

Action: A process can only use either Simple API functions or threaded Simple API functions, not both.

STAPIFORKEXEC, Calls to YottaDB are not supported after a fork() if threaded Simple API functions were in use in parent. Call exec() first

Run Time Error: This indicates a process that has already used at least one threaded Simple API function did a fork() to create a child process and is trying to use YottaDB (e.g. Simple API functions like ydb_set_s(), or threaded Simple API functions like ydb_set_st()).

Action: Once a process that has used threaded Simple API functions or threaded Simple API functions does a fork(), the child process has to do an exec() before it can call again into YottaDB (using Simple API functions or threaded Simple API functions).

STDERRALREADYOPEN, STDERR deviceparameter specifies an already open device xxxx

Run Time Error: This indicates that the STDERR deviceparameter in the OPEN command of a PIPE device specifies a device name xxxx that is already open in the process.

Action: Specify a device name that is not already an open device in the process.

STRUCTNOTALLOCD, Structure not previously called with Alloc() method

Runtime Error: This message comes only from the Golang wrapper. If a BufferT or BufferTArray structure is used without having been allocated by the Alloc() message, the wrapper can return this error.

Action: Be sure the structure has been allocated before attempting to use it.

TCPCONNTIMEOUT, Connection wait timeout (ssss seconds) has expired

Runtime Error: When MUPIP RESTORE or MUPIP BACKUP are pushing/pulling data over a socket, the attempt to attach to "the other side" can timeout. If it does timeout, this is the error that is returned.

Action: Determine the source of the connection delay, fix and retry.

THREADEDAPINOTALLOWED, Process cannot switch to using threaded Simple API while already using Simple API

Run Time Error: This indicates a process has started using the Simple API functions (e.g. ydb_set_s()) and is now trying to use the threaded Simple API functions (e.g. ydb_set_st()).

Action: A process can only use either Simple API functions or threaded Simple API functions, not both. If the base program corresponding to this process is an M program, it can only use Simple API functions.

UNKNOWNSYSERR, An unknown system error has occurred:

Runtime Error: An unrecognized error code has been passed to ydb_message_t() (C) or MessageT() (Golang)

Action: Determine the source and meaning of the unknown message code such that a proper error message code can be passed in.

UNSETENVFAIL, VIEW "UNSETENV":"eeee" failed in unsetenv() system call

Run Time Error: This indicates that a unsetenv() system call failed for the environment variable named eeee.

Action: Examine the accompanying SYSCALL error message which has more detail on the error returned by the unsetenv() call.

From GT.M

COLLDATAEXISTS, Collation type cannot be changed while xxxx data exists

Run Time Error: This indicates that an attempt was made to change the collation type while xxxx was either a subscripted local for a process collation change or a gvn name for global variable collation.

Action: KILL or NEW the local variables before you change the local collation type, or KILL a gvn before changing its collation.

COMMFILTERERR, Error executing the command filter for FFFF DDDD

Run Time Error: Reports a problem in filter code where FFFF describes the nature of the filter and DDDD some thing about the nature of the issue. There may be associated/related messages. Because filters are a potential security tool, these errors tend are generally reported to the operator log.

Action: Analyze the filter code in light of the messages and revise accordingly.

EXTRINTEGRITY, Database ffff potentially contains spanning nodes or data encrypted with two different keys

MUPIP Error: MUPIP EXTRACT cannot run because the database file ffff might contain spanning nodes or be partially encrypted with a particular key. Proceeding on a live database in such situation could result in data corruption.

Action: If you encounter this error while running MUPIP EXTRACT with -FORMAT="BINARY", re-run the command with the -FREEZE qualifier. MUPIP EXTRACT requires -FREEZE to acquire stand-alone access to produce a consistent copy of the data. However, not using -FREEZE when you request a MUPIP EXTRACT may produce a loadable, if inconsistent output. If you encountered this error while running MUPIP EXTRACT with ZWR or GO format, it is likely that your database is encrypted with more than one key; with BINARY output it may be multiple keys or spanning node data. If the issue is a key change, run MUPIP REORG -ENCRYPT to complete the encryption of the database. As a final resort, you may use an -OVERRIDE qualifier to proceed on a live database that either contains spanning nodes or is undergoing (re)encryption. Although EXTRACT -OVERRIDE may produce text for analysis, the result is not suitable as input for MUPIP LOAD and YottaDB highly discourages using -OVERRIDE.

FAILEDRECCOUNT, LOAD unable to process MMMM records

MUPIP Error: MUPIP LOAD was unable to load MMMM records from the specified input extract.

Action: Examine prior RECLOAD error messages for causes for the failed records and address them.

ICUNOTENABLED, ICU libraries not loaded

Run Time Warning: The operation required the library containing support for International Components for Unicode (ICU) but YottaDB could not find libicu. There may be other messages.

Action: If you require UTF-8 support, install an appropriate ICU library - see the YottaDB Administration and Operations Guide for information on ICU setup.

LOADRECCNT, Last EXTRACT record processed by LOAD: RRRR

MUPIP Information: This message indicates number of records (RRRR) MUPIP LOAD processed. The number of records represents the sum of header records, successfully loaded data records, and failed records. Note: LOAD may have stopped processing due to a record limit in the command or a <CTRL-C>.

Action: Ensure the identified stopping point corresponds with your intentions.

LOWSPC, WARNING: Database DDDD has less than PPPP% of the total block space remaining. Blocks Used: UUUU Total Blocks Available: AAAA

Operator log Information: The database has UUUU block in use and is appoaching its current limit of AAAA blocks. When the database reaches the 88% size threshold, and for every 1% increase in size and beyond, YottaDB reports the blocks used in the LOWSPC warning as the sum of the data blocks and the local bit map blocks.

Action: Purge data if possible. Consider a MUPIP REORG to compact the remaining data. Investigate whether migrating to a database created by a current version has a higher limit. Move some data to another, possibly new, region and delete it from this one.

MSTACKCRIT, User-specified M stack size critical threshold of xxxx not appropriate; must be between mmmm and nnnn; reverting to kkkk

Run Time Error: The environment variable ydb_mstack_crit_threshold was set to an invalid value, either too large, in which case YottaDB uses the largest acceptable value or too low, in which case YottaDB uses the smallest acceptable value.

Action: If the adjusted value is unacceptable, revise or unset the environment variable.

NOFILTERNEST, Filter nesting not allowed

Run Time Error: Filter code must not invoke other code that requires a filter.

Action: Revise the filter code to adhere to the requirement.

PATALTER2LARGE, Pattern match alternation exceeded the LLLL repetition limit on prospective matches

Run Time Error: An alternation pattern applied to a long occurrence of that pattern reached a YottaDB limit (LLLL) on tracking the match.

Action: Revise the logic to reduce the size of the string being matched or to otherwise break up the match into smaller parts.

REGFILENOTFOUND, Database file DDDD corresponding to region RRRR cannot be found

MUPIP Error: This indicates MUPIP cannot locate the database file DDDD mapped to region RRRR.

Action: Ensure that the current global directory is the one intended and that it maps the file intended. If the path is relative or includes environment variables, ensure that the current working directory and any environment variables are appropriate. Also ensure the file exists and has authorizations, including its path, that make it available to the user attempting to access it. If the MUPIP command involves a statsDB (for example MUPIP INTEG -STATS), ensure that the appropriate regions have STATS enabled, that the $ydb_statsdir environment variable has been properly defined, and that other processes are using shared statistics, as MUPIP by itself does not create new statsDB databases. Note that MUPIP INTEG does not create statsDB and reports any that it skips with an informational message, but exits with a normal status after such skips.

VIEWCMD, View parameter pppp is not valid with the VIEW command

Run Time Error: This indicates that the VIEW command has an argument pppp that is only valid with the $VIEW() function.

Action: Modify the argument.

VIEWFN, View parameter pppp is not valid with the VIEW command

Run Time Error: This indicates that the $VIEW() function has an argument pppp that is only valid with the VIEW command.

Action: Modify the argument.

Legal Stuff

Copyright © 2019 YottaDB LLC

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts and no Back-Cover Texts.

YottaDB^® is a registered trademark of YottaDB LLC. GT.M™ is a trademark of Fidelity National Information Services, Inc. Other trademarks belong to their respective owners.

This document contains a description of YottaDB and the operating instructions pertaining to the various functions that comprise the software. This document does not contain any commitment of YottaDB LLC. YottaDB LLC believes the information in this publication is accurate as of its publication date; such information is subject to change without notice. YottaDB LLC is not responsible for any errors or defects.

yottadb_r122_linux_armv7l_pro.tgz

yottadb_r122_linux_armv6l_pro.tgz

yottadb_r122_linux_x8664_pro.tgz

yottadb_r122_rhel7_x8664_pro.tgz

yottadb_r122_src.tgz

YottaDB r1.22

Release Note Revision History

Revision	Date	Summary
1.10	Jan 22, 2019	Added specific notes to the Platforms section for the possible need to recompile YottaDB
1.00	May 15, 2018	r1.22 Initial Release

Contact Information

YottaDB LLC

40 Lloyd Avenue, Suite 104 Malvern, PA 19355, USA [email protected] +1 (610) 644-1898

Support

Customers

Contact your YottaDB support channel.

Others

For free (to you) support from members of communities who run widely available applications on YottaDB, please use an application-specific list where appropriate.

• For issues specific to the use of YottaDB from node.js via nodem, QewdJS or Enterprise Web Developer, post to the Enterprise Web Developer community.

• For issues specific to the use of YottaDB with VistA flavors, post to the Hardhats list.

• For issues specific to the use of YottaDB with M other than for applications above, post to the comp.lang.mumps list.

• If you are not sure where to post, or for requests other than to the above communities, post an issue at https://github.com/YottaDB/YottaDB/issues and include the words "help wanted" in the summary.

r1.22

Overview

YottaDB r1.22 is a minor release, primarily intended to make available in a YottaDB release the enhancements and fixes in the upstream GT.M release.

YottaDB r1.22 is built on (and except where explicitly noted, upward compatible with) both YottaDB r1.20 and GT.M V6.3-004.

As always, there as other enhancements and fixes, as noted below. See our Get Started page to use YottaDB.

Platforms

A platform is a combination of a CPU architecture and an operating system. A platform is Supported, Supportable, or Unsupported. Supported means that we have the platform in our development environment and test each release on that platform. Supportable means that although we do not necessarily have such a platform in our environment, we have no reason to believe that the software will not run on it. All others are Unsupported.

CPU Architecture	Supported OS Version(s)	Notes
64-bit x86	Ubuntu 16.04 LTS; Red Hat Enterprise Linux 7.4	Note that there are separate binary distributions for Ubuntu and Red Hat, owing to differences in library versions of those distributions.
Raspberry Pi 3 Model B; BeagleBone Black Wireless	Raspbian GNU/Linux 9.1; Stretch IoT (non GUI)	While YottaDB r1.22 is Supportable on other ARMv7-A CPUs, owing to variations in the implementations of ARM microarchitectures, we recommend that you ensure the software runs correctly before committing to any specific hardware other than those Supported. Please contact [email protected] if you want a specific combination of OS and CPU microarchitecture to be Supported.
Raspberry Pi Zero	Raspbian GNU/Linux 9.1	While YottaDB r1.22 is Supportable on other ARMv6 CPUs, owing to variations in the implementations of ARM microarchitectures, we recommend that you ensure the software runs correctly before committing to any specific hardware other than those Supported. Please contact [email protected] if you want a specific combination of OS and CPU microarchitecture to be Supported.

Recent releases of major GNU/Linux distributions with contemporary kernels, glibc and ncurses are Supportable. Specific notes:

• On Ubuntu releases after 18.04 LTS, YottaDB needs the libtinfo5 package to be installed.
• On Arch Linux) and possibly other leading edge distributions, YottaDB may need to be recompiled from source code owing to library and tool chain versions significantly more recent than those used in building the distribution.

Installation

See our Get Started page to use YottaDB.

We strongly recommend that you install YottaDB r1.22 in a newly created directory, different from those of YottaDB r1.20 and any GT.M versions you may have installed on the system.

Removing an installed YottaDB release

Assuming $ydb_dist points to the directory where YottaDB is installed:

• Cleanly shut down all application processes using that release.
• Execute mupip rundown && mupip rundown -relinkctl.
• Ensure that there are no gtcm* or gtmsecshr processes active.
• Use sudo lsof | grep $ydb_dist to ensure there are no open files.
• Delete the directory with sudo rm -rf $ydb_dist.

Upgrading to YottaDB r1.22

As YottaDB r1.22 is upward compatible from both YottaDB r1.20 and GT.M V6.3-004, the minimal upgrade steps are:

• Install YottaDB r1.22.
• Recompile any object code, and recreate shared libraries where appropriate.
• If you are using encryption, compile and install the reference implementation plugin or your customized plugin.
• Cleanly shut down the application and ensure that the database files are shut down using mupip rundown from the prior release.
• Switch journal files with the new YottaDB release.
• Start using the new YottaDB release.

To upgrade from older GT.M releases, first upgrade to GT.M V6.0-000 or later and follow the steps above, or contact your YottaDB support channel for assistance and guidance.

A more sophisticated upgrade technique is:

• Install YottaDB r1.22.
• Create a new replicated instance of your application (on the same system or a different system).
• Assuming the existing instance is A, and the new instance is B, start replicating from A to B.
• Once B catches up, switchover so that B is in a primary role replicating to A.
• Once you are satisfied with B, remove (or upgrade) A.

Change History

r1.22

YottaDB r1.22 includes the following changes from YottaDB r1.20.

ID	Category	Summary
(#102)	Other	ydb_coredump_filter to control information in core dumps
(#159)	Language	$TEXT returns source code lines with both carriage return and newline line terminators removed
(#193)	Language	GDE @ command reports success after executing good command files
(#197)	Language	ydb_env_set preserves ydb_gbldir/gtmgbldir and ydb_routines/gtmroutines
(#207)	DB	TPRESTART syslog messages identify correct global name and message type
(#209)	Other	Complete set of ydb_* environment variables corresponding to all gtm*/GTM* environment variables
(#210)	Admin	Source Server errors always appear in the Source Server log file
(#215)	Admin	Ctrl-Z (the suspend signal) is honored in all cases
(#217)	DB	SET and $INCREMENT() work correctly without abnormal process termination in a very rare case
(#218)	DB	ydb_app_ensures_isolation provides initial setting for global variables where application design ensures Isolation
(#220)	Other	mumps -nowarning does not display BLKTOODEEP messages
(#221)	Admin	ydb_* environment variables to drive operation of ydbinstall.sh script
(#224)	Language	Certain edge cases of VIEW and $VIEW() work correctly without terminating the process
(#228)	Language	$ZSEARCH() calls in a loop work correctly in certain edge cases
(#233)	DB	Option to reduce IO impact of MUPIP REORG
(#235)	Other	Source server fails rarely with JNLSWITCHRETRY error when instance is frozen
(#237)	Other	Establish replication connections more efficiently in an edge case
(#247)	DB	Epoch tapering works correctly without process termination from signal 8 in a rare case

Database

• Type 0 and type 3 TPRESTART messages in the syslog (enabled by turning on TP restart logging) correctly report the global variable causing the restart. Furthermore, type 3 messages correctly identify themselves as type 3 messages. Previously, they could report an incorrect global variable reference, and type 3 messages were sometimes incorrectly identified as type 2. (#207)

• SET and $INCREMENT() operations on a global variable work correctly without abnormal termination in a very rare case. Previously, it was possible for a process doing the operation to terminate abnormally with a SIG-11. This was only observed in internal testing, and there was no risk of database damage from this issue. (#217)

• A string value of a comma-separated list of global variable names in the environment variable ydb_app_ensures_isolation informs YottaDB that application design ensures the transaction property of Isolation for global variables in that list and YottaDB need not do so, potentially increasing application throughput by reducing random, accidental TP restarts. The effect is functionally equivalent to executing the M language command view "noisolation":"<list>" where "<list>" is the value of the environment variable. For example, export ydb_app_ensures_isolation="^TMP,^XREF" is equivalent to a process executing view "noisolation":"^TMP,^XREF" before it makes any database accesses. Note that using the environment variable ydb_app_ensures_isolation requires the environment variable ydb_gbldir to be set to a valid global directory. The setting applies to the global variables mapped by that global directory. (#218)

• MUPIP SET -REORG_SLEEP_NSEC= specifies the number of nanoseconds that a MUPIP REORG process operating between blocks that it processes, with default value of 0 and a maximum of 999999999 (i.e. 999,999,999, or 1 nanosecond less than 1 second). Using non-zero values reduces the IO impact of MUPIP REORG, at the cost of increasing the duration of the operation. Note that the existing environment variable ydb_poollimit / gtm_poollimit is the appropriate technique to limit the impact of MUPIP REORG on global buffers; the -reorg_sleep_nsec can be used to limit the impact on the IO subsystem. (#233)

• Epoch tapering works correctly. Previously, on very rare occasions, epoch tapering could result in process termination with a signal 8 resulting from a divide-by-zero error. This issue was only observed in the development environment, and was never reported by a user. (#247)

Language

• $TEXT() returns source code lines with both the carriage return and newline characters removed from routine files that have lines ending with them. Previously, only the newline was removed. (#159)

• The GDE @ command reports success after executing good command files. In r1.20, it would issue a YDB-E-IOEOF error even after sucessfully executing the commands in the file specfied. A workaround was to set the environment variable ydb_msgprefix to "GTM" for the GDE process. (#193)

• Values of environment variables ydb_gbldir/gtmgbldir and ydb_routines/gtmroutines set before sourcing ydb_env_set are preserved. The ydb_* versions of the environment variables is used if both ydb_* and gtm* versions are defined. (#197)

• VIEW commands and $VIEW functions work correctly. Previously, edge cases in YottaDB, and newly introduced in GT.M V6.3-004, in commands like VIEW "NOISOLATION", functions like $VIEW("GVFILE"), etc., could terminate the process abnormally with a SIG-11, SIG-6, GTMASSERT2 error. (#224)

• $ZSEARCH() calls in a loop using the same stream and the same pattern work by ignoring edge cases of changes in the underlying files (such as a permission change of a directory that makes a file inaccessible to the process) and pathological cases such as an infinite symbolic link loop. Previously, ZSEARCH() calls in these edge cases could incorrectly issue an error (SYSTEM-E-ENO40, SYSTEM-E-ENO13, etc.), with some difference in behavior between Ubuntu 18.04 LTS and prior releases resulting from a change in the underlying API. (#228)

System Administration

• Replication Source Server startup errors always appear in the Source Server log file. In previous versions of YottaDB, if the Source Server was started from the terminal, and the Source Server startup command failed or the Source Server successfully started but later failed (e.g. with errors like NULLCOLLDIFF, REPLOFFJNLON, REPLINSTNOHIST, SECNOTSUPPLEMENTARY etc.) this error message did not show up in the Source Server log file or on the terminal where it was started (assuming that terminal still exists at the time of the error). The release notes of GT.M V6.3-004 mention that this issue has been fixed by GTM-8576 but we found the issue exists even in GT.M V6.3-004. (#210)

• Ctrl-Z (the suspend signal) is honored by YottaDB processes in all cases. Previously, if the signal was delivered while the process was executing a section of code where it was not safe to suspend it, the signal was ignored and the user had to retry the Ctrl-Z. Note that as handling the signal is deferred when the process is in sensitive areas of code, it is possible for the response to be momentarily delayed. (#215)

• The ydbinstall.sh script uses environment variables ydb_buildtype, ydb_dryrun, ydb_gtm, ydb_group_restriction, ydb_keep_obj, ydb_lcase_utils, ydb_overwrite_existing, ydb_prompt_for_group, and ydb_verbose to drive its operation, corresponding to the environment variables gtm_buildtype, gtm_dryrun, gtm_gtm, gtm_group_restriction, gtm_keep_obj, gtm_lcase_utils, gtm_overwrite_existing, gtm_prompt_for_group, and gtm_verbose. The latter remain supported, with the proviso that if a ydb_ prefixed environment and the corresponding gtm_ prefixed environment variable are both defined, the former takes precedence. (#221)

Other

• A value of "0xXX" where XX are case-insensitive hexadecimal digits in the environment variable ydb_coredump_filter, sets the corresponding value to /proc//coredump_filter (see man 5 core) at process startup without explicitly setting a value if unspecified. This controls the contents of core dumps generated by the process. A YottaDB process that terminates or is terminated abnormally (terminates with any signal except SIGKILL, SIGQUIT, or SIGTERM) generates a core file, if generating core files is enabled in the system and for that process. As core files can contain confidential / proprietary information, please limit the content of core files to that required for diagnostic purposes, and generate them with appropriate protection. If ydb_coredump_filter is not specified, but gtm_coredump_filter is, the latter environment variable is used. If both are specified, the former takes precedence. Previously, only the environment variable gtm_coredump_filter was recognized, with a default value of "0x73". (#102)

• YottaDB has a complete set of environment variables with "ydb_" prefixes for environment variables with "gtm" and "GTM" prefixes. For example, ydb_routines is the equivalent of gtmroutines, ydb_chset is the equivalent of gtm_chset, ydb_repl_instance is the equivalent of gtm_repl_instance etc. YottaDB continues to support the gtm* environment variable if the corresponding ydb_* environment variable is not defined. If both variables are defined, the ydb_* definition takes precedence. The full list of environment variables is in the Environment Variables section of the Administration and Operations Guide. (#209)

• mumps -nowarning does not display BLKTOODEEP messages. Previously, mumps -nowarning displayed YDB-W-BLKTOODEEP errors even though it did not display other types of errors (e.g. YDB-E-EXPR). (#220)

• Source Server processes continue operation even when the instance is frozen due to an error while switching journal files. In prior versions of YottaDB (and GT.M versions V6.3-001A and above), the Source Server could terminate with a JNLSWITCHRETRY error in this uncommon occurrence. This issue was only observed in the development environment and was never reported by a user. (#235)

• Initiating replication connections between Source and Receiver Servers is more efficient. Previously, in rare cases, the Source Server unnecessarily disconnected the connection and reconnected. This is very similar to (#136) but under slightly different circumstances. (#237)

GT.M V6.3004

ID	Category	Summary
(GTM-1042)	Other	The gtm_mstack environment variable can control the M stack size
(GTM-3146)	Other	MUPIP BACKUP command ignores aliases
(GTM-5730)	Admin	Update Process logs show record type descriptions
(GTM-6747)	Admin	MUPIP SET -JOURNAL does not adjust sequence numbers on replicated regions
(GTM-7483)	Other	Improve message from a MUPIP INTEG when a Directory Tree issue is detected
(GTM-7872)	Admin	Improve message from a MUPIP INTEG directed to an non-existent database file
(GTM-7915)	DB	Trigger key size not limited by database key size
(GTM-8202)	Admin	Journal extract for specific sequence number(s)
(GTM-8576)	Admin	Source Server logs errors to its log file rather than stderr
(GTM-8643)	Language	SOCKET device listen queue depth determined by OS configuration; better retry management for local connects
(GTM-8699)	Language	Optional region argument for $VIEW("STATSHARE")
(GTM-8777)	Other	%GCE, %GSE, %RCE, and %RSE support QUIET entrypoint. %RCE and %RSE support QCALL entrypoint
(GTM-8791)	Other	Prevent certain control character inputs to LKE from causing a SIG-11
(GTM-8859)	Admin	Correct turn-around point calculation for MUPIP JOUNAL -ROLLBACK when dealing with "idle" regions
(GTM-8860)	Admin	Prevent multiple slashes (/) from appearing in MUPIP JOURNAL -EXTRACT output
(GTM-8870)	Other	Clean up some rough edges
(GTM-8874)	Language	VIEW "[NO]STATSHARE" accepts an optional region-list
(GTM-8883)	DB	Online Freeze/Journal switch cleanup
(GTM-8891)	Language	Prevent process failure from a certain pattern of $SELECT() errors
(GTM-8894)	Language	$ZRELDATE ISV provides the UTC data and time of the YottaDB build
(GTM-8895)	Other	%PEEKBYNAME protects the str variable from inappropriate modification
(GTM-8899)	Other	Work-around for bugs in the GnuPG agent
(GTM-8900)	Admin	MUPIP SET -NOENCRYPTABLE works without valid encryption setup
(GTM-8903)	Language	Prevent process failure from a certain pattern of $SELECT() usage
(GTM-8906)	Admin	MUPIP JOURNAL -ROLLBACK and -RECOVER handle a larger number of records
(GTM-8909)	Other	Online help does not report an inappropriate error when exiting after the user typed <CTRL-C>
(GTM-8914)	Language	$VIEW("GVSTATS"), ZSHOW "G" and ZSHOW "T" flag results that are probably not current
(GTM-8919)	DB	MUPIP REORG -ENCRYPT does not induce CRYPTOPFAILED errors in concurrent processes
(GTM-8922)	Language	VIEW region subarguments can be a list of regions
(GTM-8923)	Language	UTF-16 READ * and WRITE * Fixes
(GTM-8924)	Language	Fix for unusual conditions with a $PRINCIPAL SOCKET device for a JOB'd process
(GTM-8926)	Other	Flush Timer Deferred During External Call
(GTM-8927)	DB	Prevent inappropriate JNLCTRL errors

Database

• YottaDB limits the key size of a trigger to 1019 bytes. Previously, it limited the key size to the configured limit of regions that used the trigger, which caused a need to increase the maximum key limit and/or change the name of the trigger. This change also allows YottaDB to store triggers more compactly in cases where regions have a small record size limit. (GTM-7915)

• YottaDB handles journal switch errors encountered during a MUPIP FREEZE -ONLINE -ON correctly. Previously it could leave the specified region with a very large Epoch Interval, which could cause a replication source server to issue SRVLCKWT2LNG errors. In addition, YottaDB only sends a PREVJNLLINKCUT message to the system log when the links have been cut. Previously the message could be reported in rare cases where a journal switch was deferred, then later switched without cutting links. Also, the source server reports the SRVLCKWT2LNG message correctly. Previously the values for PID and the wait time were reversed and the message reported minutes instead of seconds as the unit of time. (GTM-8883)

• YottaDB processes work correctly when a concurrent MUPIP REORG -ENCRYPT changes the encryption key. Previously processes could fail with CRYPTOPFAILED errors unaccompanied by other errors explaining the reason for the failure. This issue was only observed in the development environment, and was never reported by a user. MUPIP RESTORE issues an encryption setup error when attempting to restore an encrypted backup without the correct encryption setup. Previously MUPIP RESTORE issued a CRYPTOPFAILED when the restore instance was not configured for encryption. (GTM-8919)

• YottaDB avoids issuing JNLCNTRL errors inappropriately. This issue was only observed in the development environment, and was never reported by a user. (GTM-8927)

Language

• The listen queue depth specified with WRITE /LISTEN for a listening socket is limited only by the OS; previously, YottaDB enforced an artificial limit of 5. A SOCKET device retries connection attempts on local sockets that fail due to possible transient issues on the other end of the attempted connection (for example an insufficiently large listen queue). Previously such attempts could sometimes appear to succeed, when they actually did not, leading to subsequent errors from READ and WRITE on the socket. (GTM-8643)

• $VIEW("STATSHARE",<region>) returns TRUE (1) if the process is currently sharing database statistics for the region and FALSE (0) if it is not. For the process to be sharing the database must be enabled for sharing and the process must have opted in to share. $VIEW("STATSHARE") with no region argument indicates the process has enabled sharing. (GTM-8699)

• VIEW "[NO]STATSHARE"[:<region-list>] enables or disables database statistics sharing for listed regions which permit such sharing. Without the region-list, the command acts on all regions enabled for sharing. When a targeted region has sharing disabled, STATSHARE has no effect. Note: a VIEW "[NO]STATSHARE" with no region sub-argument opens any unopened mapped regions and any enabled associated statsDB regions; the $gtm_statshare environment variable applies to databases as the application first uses them. When the last VIEW "[NO]STATSHARE" had no region sub-argument, regions implicitly share when the process first references them, but after a VIEW specifies selective sharing, regions don't implicitly share as they open. Previously the VIEW command only supported enabling or disabling sharing for all enabled regions. (GTM-8874)

• A <side-effect-expression><pure-Boolean-operator>$SELECT(0:side-effect-expression)) sequence produces a SELECTFALSE run-time error; a regression introduced with the literal optimizations in V6.3-002 caused this combination to produce a SIG-11 (segmentation violation) at compilation. (GTM-8891)

• The $ZRELDATE Intrinsic Special Variable provides the UTC date/time of the YottaDB build in the form YYYYMMDD 24:60 (using $ZDATE() notation). While $ZVERSION and $ZYRELEASE are probably better identifiers for most uses, $ZRELDATE may be a helpful alternative for those testing pre-release builds. (GTM-8894)

• $SELECT() deals with cases where the first value is evaluated to a literal TRUE (1) and later arguments to the function contain one or more global references. Due to a regression in V6.3-002 associated with compiler optimizations caused this combination to produce a SIG-11 (segmentation violation) at compilation. (GTM-8903)

• $VIEW("GVSTATS",<region>), ZSHOW "G" and ZSHOW "T" return a question-mark (?) at the end of their output strings when the process does not have access to the current shared statistics; they did not do this previously. Please adjust your scripting as needed to allow for this format enhancement. (GTM-8914)

• VIEW commands accepting a region sub-argument also accept a comma (,) delimited string listing of regions. As part of deadlock prevention, YottaDB sorts the regions in an internal order, eliminating any duplicates from the list. Note: a VIEW with no region sub-argument opens any unopened mapped regions in the current global directory, while one with a list only opens the listed regions. If the VIEW argument has a corresponding environment variable to set the default state, the state applies to databases as the application implicitly opens them with references. Previously such commands accepted either one argument or an asterisk ("*") for all regions, but if supplied with a region string tended to do more than the specified region. (GTM-8922)

• READ * and WRITE * operate correctly on files and sockets with a CHSET of UTF-16, UTF-16BE, or UTF-16LE. Previously READ * returned an incorrect codepoint and WRITE * produced an incorrect character or a BADCHAR error. (GTM-8923)

• YottaDB handles socket errors on JOB process startup correctly. Previously if the INPUT and/or OUTPUT for a JOB were in a bad state (e.g. LISTEN) the JOB process could terminate with a KILLBYSIGSINFO1 (signal 11) and a core dump or hang indefinitely. (GTM-8924)

System Administration

• The Update Process logs record types with their corresponding type description; previously it only logged the integer type value. Note that it is possible, the change could disrupt code that parses the modified results. (GTM-5730)

• MUPIP SET -JOURNAL in a replicated environment respects existing region sequence numbers, which aligns with the behavior of MUPIP BACKUP, which can also create new journal files. Previously, MUPIP SET -JOURNAL updated the region sequence numbers to the maximum of the sequence numbers for all regions specified for the command. (GTM-6747)

• MUPIP INTEG produces a MUNOACTION error when the command specifies non-existent database; previously it produced an INTEGERRS message in this situation. (GTM-7872)

• MUPIP JOURNAL -EXTRACT recognizes the -SEQNO=<sequence_number_list> qualifier. <sequence_number_list> is a comma separated list of sequence number(s) in decimal form. It specifies a list of sequence numbers to include or exclude in the journal extract. When a sequence number has a (~) prefix, -SEQNO excludes it from the journal extract. For replicated regions, EXTRACT -SEQNO uses replication sequence numbers, which may select records from multiple regions. For unreplicated regions, EXTRACT uses database transaction numbers, but specifying sequence number selection with more than one region produces a JNLEXTRCTSEQNO error. When the sequence number list contains a sequence number involved in a TP transaction, EXTRACT reports it in a broken transaction file when the result does not contain all regions, which is commonly the case without replication, and may be the case with replication when not all regions are available to the utility. Previously EXTRACT did not support record selection by sequence number. (GTM-8202)

• The Source Server directs errors to the Source Server log file. Be sure to check the Source Server log; previously it directed errors to stderr, which meant that if it was started from a terminal and the terminal session closed, a subsequent error logged an inappropriate NOPRINCIO error and terminated the Server. (GTM-8576)

• MUPIP JOURNAL -ROLLBACK ignores Idle regions (those that received their last updates a long time previously). GT.M V6.3-000 introduced a regression whereby idle regions were not appropriately excluded from determining the ROLLBACK turn-around point. The result was that ROLLBACK could adjust time so far back that the journal files are not present (say due to periodic removal to conserve space) which terminated the ROLLBACK. The operational work-around for this issue was to cut new journal files when deleting old journal files. (GTM-8859)

• YottaDB appropriately removes extra slashes (/) from journal files and output files provided to the MUPIP JOURNAL -EXTRACT command. Previously, when extracting journal files, YottaDB kept any extra slashes in the journal file name or output file name, causing multi-slash paths to appear in the output of the extract. The workaround was to avoid using journal files and output files with successive slashes in their paths. (GTM-8860)

• MUPIP SET -NOENCRYPTABLE can set the database encryptable flag to FALSE. Previously, after invoking MUPIP SET -ENCRYPTABLE attempting to undo that action failed unless the GNUPGHOME pointed to a valid directory and gtm_passwd was defined in the environment. This issue was only observed in the development environment, and was never reported by a user. MUPIP SET -ENCRYPTABLE now performs some basic encryption setup checks. To execute this command, GNUPGHOME must point to a valid directory and gtm_passwd must be defined in the environment. Previously, the command did not require these environment variables. (GTM-8900)

• MUPIP JOURNAL -ROLLBACK and -RECOVER handle large amounts of journal data; while there are still limits, previously when they attempted to handle around 55 million updates or more they inappropriately failed with a GTM-F-MEMORY error. (GTM-8906)

Other

• YottaDB supports setting the M stack size using the gtm_mstack_size environment variable. This specifies the size of the M stack in KiB. No setting or a setting of 0 uses the default (272KiB). The minimum supported size is 25 KiB; YottaDB reverts values smaller than this to 25 KiB. The maximum supported size is 10000 KiB; YottaDB reverts values larger than this to 10000 KiB. (GTM-1042)

• The MUPIP BACKUP command ignores aliases when executing system commands (cp, mv etc.) under the covers. Previously, aliases for cp, mv etc. (in one's .bashrc/.cshrc etc.) could cause the backup to use those alias versions of the cp/mv commands which might not work as desired. (GTM-3146)

• MUPIP INTEG issues a DBKEYMX error in case of a long key name stored in the DT (Directory Tree); previously, it issued a INVSPECREC with no context. This issue was only observed in the development environment, and was never reported by a user. (GTM-7483)

• When invoked at their QUIET or QCALL entrypoints, the %GCE, %GSE, %RCE, and %RSE utility routines report only globals or routines in which they find a match, for example: do QUIET^%GCE or do QCALL^%RSE. The QCALL entryref only exists in %RCE and %RSE. Previously these routines always reported every item they processed regardless of whether it contained a match. (GTM-8777)

• LKE handles certain control characters appropriately; previously, for example, <CTRL-Z> caused a segmentation violation (SIG-11). (GTM-8791)

• YottaDB addresses some issues identified by static analysis tools which appear to be obscure, and, in most cases, harmless. Previously an error with a source file could leak a file descriptor, and, in odd circumstances, trigger management could fail with a segmentation violation. These issues were only observed in the development environment, and were never reported by a user. (GTM-8870)

• %PEEKBYNAME protects the variable str against inappropriate modification in the user symbol space; previously an error in %PEEKBYNAME, possibly caused by user input, would change the value of this variable. (GTM-8895)

• The YottaDB reference encryption plugin library retries a failing decryption request once to mask a bug in GnuPG that caused spurious CRYPTKEYFETCHFAILED errors during process startup or re-encryption. GnuPG version 2.2.4 fixes the underlying bug, so upgrading to GnuPG 2.2.4 and libgcrypt 1.8.2 fixes the underlying issue. These problems were only seen in development and not reported by a customer. (GTM-8899)

• The help facility for the utility programs MUPIP, DSE and LKE ignores control-C; previously if a user pressed <CTRL-C> while using help, the facility could exit with a harmless, but inappropriate, ENO256 error. (GTM-8909)

• MUMPS processes which attempt to process a flush timer while executing an external call defer the timer processing until after the external call is complete. Previously, flush timers could interfere with non-reentrant routines used in the external call with undesired results. In particular, memory allocation operations in the external call can interfere with ASYNCIO setup in the flush timer, resulting in process hangs. (GTM-8926)

More Information

(Section Blank for this Release)

Messages

From GT.M

CRYPTJNLMISMATCH, Encryption settings mismatch between journal file jjjj and corresponding database file dddd

All Components Error: Encryption settings in the header of database file dddd do not match those stored in the header of journal file jjjj. This prevents access to the database. The most likely cause is inappropriate operator action such as replacing the current journal file with an older journal file.

Action: Correct the error that caused the incorrect journal file to be pointed to by the database file. If the correct journal file has been inadvertently deleted, create new journal files with the -noprevjnl switch. Take a backup as soon as possible thereafter. Depending on your situation, you may need to refresh secondary instances.

ENCRYPTCONFLT2, Message: A concurrent MUPIP REORG -ENCRYPT changed the encryption key for RRRR before the process could initialize it

Run Time Warning: Due to a concurrent MUPIP REORG -ENCRYPT, a process was forced to defer encryption key initialization for region RRRR.

Action: None. This information message is only important when followed by other encryption errors.

INVSEQNOQUAL, Invalid SEQNO qualifier value xxxx

MUPIP Error: This error indicates that MUPIP JOURNAL -EXTRACT -SEQNO command could not extract a journal file because an invalid SEQNO format was specified.

Action: Enter a comma separated list of valid sequence numbers ('0' or positive integers) as value for the SEQNO qualifier. The format of the -SEQNO qualifier is -SEQNO=seqno1[,seqno2,seqno3.....] where seqno is the region sequence number in decimal format.

JNLACCESS, Error accessing journal file jjjj

Run Time Error: YottaDB sends this message to the system log followed by other messages detailing the failure. jjjj is the file-specification for the inaccessible journal. In most situations, this error occurs when the journal file storage runs out of disk space.

Action: Review the accompanying message(s) for additional information. This means an error while trying to write to the journal file.

JNLBADRECFMT, Journal Record Format Error encountered for file jjjj at disk address yyyy

MUPIP/Run Time Error: This indicates that an attempt to open a journal file encountered an invalid record.

Action: Report the entire incident context to your YottaDB support channel.

JNLCYCLE, Journal file jjjj causes cycle in the journal file generations of database file dddd

MUPIP Error: This indicates that MUPIP encountered journal file jjjj causing cycle in the journal file generations of database file dddd; that is following the back-pointers in the journal files can wind up repeatedly finding the same journal file.

Action: Contact your YottaDB support channel with appropriate log messages.

JNLDBERR, Journal file jjjj does not correspond to database dddd

Run Time Error: This indicates that YottaDB could not open journal file jjjj for database file dddd because the journal file header identifies itself as belonging to a different database file that does not exist in the system.

Action: Use a MUPIP SET command with the qualifier JOURNAL to create a journal file that matches the database.

JNLDISABLE, Specified journal option(s) cannot take effect as journaling is DISABLED on database file dddd

MUPIP Warning: This indicates that none of the specified journal option(s) in MUPIP SET -JOURNAL or MUPIP BACKUP command took effect, because journaling was found DISABLED on database file dddd.

Action: Revise the selection qualification to exclude the DISABLED region(s) or, if appropriate, enable journaling on those regions.

JNLEXTEND, Journal file extension error for file jjjj.

Run Time/MUPIP Error: Journal file jjjj failed to extend. If the environment is not configured for instance freeze, this causes journaling to be turned off for the region.

Action: Review the accompanying message(s) and take appropriate action. If the environment is not configured for instance freeze, perform a MUPIP BACKUP, that turns journaling on again, to reestablish durability.

JNLEXTR, Error writing journal extract file: xxxx

MUPIP Error: This indicates that an error was encountered while trying to write to either the JNL EXTRACT file or lost-transaction file or broken-transaction file as part of a MUPIP JOURNAL command.

Action: Review the accompanying message(s) for additional information.

JNLEXTRCTSEQNO, Journal Extracts based on sequence numbers are restricted to a single region when replication is OFF

MUPIP Error: When replication is enabled YottaDB applies a uniform set of sequence numbers across regions, but when it is not in use each region has its own set of sequence numbers, and, in that case, MUPIP only works on a region at a time.

Action: If you need cross region sequence numbers, start replication with at least a passive Source Server; otherwise use one MUPIP JOURNAL -EXTRACT command for each region when using the -SEQNO qualifier.

JNLFILOPN, Error opening journal file jjjj for database file dddd

Run Time/MUPIP Error: This indicates that YottaDB was unable to open journal file jjjj for the specified database file dddd. The Source Server exits with a JNLFILOPN message after six failed attempts to open journal files.

Action: Check the authorizations for the user of the process and the health of the file system holding the journal file.

JNLFLUSHNOPROG, No progress while attempting to flush journal file jjjj

Run Time Warning: Indicates processes needing space in the journal buffers were unable to write journal jjjj because even though multiple processes have controlled the resource, this process has not been able to flush records. JNLPROCSTUCK means one process is hogging, while this message means more than one process has tried but none have succeeded. Might indicate a clogged disk subsystem on which journal file JJJJ resides.

Action: Check the log file for other journaling related messages. Consider balancing disk subsystem load.

JNLFSYNCLSTCK, Journaling fsync lock is stuck in journal file jjjj

Run Time Error: A resource controlling journal file actions has remained unavailable for a long period.

Action: Check on the condition of the process identified in the associated messages.

JNLINVALID, jjjj is not a valid journal file Region: rrrr

MUPIP Error: This indicates that YottaDB could not open journal file jjjj, due to an error that is detailed in the accompanying previous message(s). While trying to create a new journal file for the same region it encountered errors. rrrr is the region name associated with the journal.

Action: Review the accompanying error message(s) to determine the cause of the failure of the new journal file creation. After the cause is resolved, to reestablish durability, perform a MUPIP BACKUP that turns journaling back on.

JNLNAMLEN, Journal file jjjj: for database file dddd exceeds maximum of MMMM

MUPIP Error: This indicates that the file-specification jjjj of the journal for database file dddd exceeds the maximum supported length of MMMM.

Action: Modify the journal file-specification to adhere to the file length restrictions.

JNLNOCREATE, Journal file jjjj not created

MUPIP/Run Time Error: This indicates that YottaDB could not create journal file jjjj.

Action: Review the accompanying message(s) for additional information.

JNLORDBFLU, Error flushing database blocks to dddd. See related messages in the operator log

MUPIP Error: This message indicates that hardening journal or database records could not be completed due to an error. The operator log should contain one or more accompanying messages indicating the cause of the error.

Action: Verify the normal state of the file system and appropriate permissions of the database and journal files. Report the entire incident context to your YottaDB support channel along with any operator log messages within the same time frame.

JNLREADEOF, End of journal file encountered for jjjj

MUPIP/Run Time Error: This indicates that MUPIP JOURNAL or a run-time journal operation encountered the end-of-file for the journal file jjjj, before it completed processing.

Action: This error indicates an improperly closed journal file. Restart journaling with a MUPIP BACKUP -NEWJNLFILES or a MUPIP SET -JOURNAL and report all available circumstance to those responsible for supporting your database operations.

JNLRECFMT, Journal file record format error encountered

MUPIP Error: This indicates that MUPIP JOURNAL encountered an invalid record in the journal file.

Action: In the event of YottaDB issuing this error message, use MUPIP BACKUP to ensure durability by creating a fresh set of journals consistent with the database. Else, to resume operation, restore the database from the last backup and play forward the updates using the appropriate MUPIP JOURNAL command. As soon as possible, report the entire incident context with information from the operator log and any other relevant information to your YottaDB support channel.

JNLSPACELOW, Journal file jjjj nearing maximum size, nnnn blocks to go

Run Time Information: This indicates that the journal file jjjj is approaching the maximum size specified for it. The system creates a new journal file when the limit is reached.

Action: None required except as part of monitoring journaling space requirements or when operational practice uses this as a trigger to intervene in journal file management.

JNLSWITCHSZCHG, Journal AUTOSWITCHLIMIT [aaaa blocks] is rounded down to [bbbb blocks] to equal the sum of journal ALLOCATION

MUPIP Information: This indicates that the specified AUTOSWITCHLIMIT value was rounded down as little as possible to make it aligned to the ALLOCATION + a multiple of EXTENSION. Any subsequently created journal file will use this value for AUTOSWITCHLIMIT.

Action: If the rounded value is inappropriate examine the alignsize, allocation and extension values and choose a more suitable value.

JNLSWITCHTOOSM, Journal AUTOSWITCHLIMIT [aaaa blocks] is less than journal ALLOCATION [bbbb blocks] for database file dddd

MUPIP Error: This indicates that the value of AUTOSWITCHLIMIT specified in a MUPIP SET JOURNAL command is less than the default or specified value of ALLOCATION. This error also indicates that the AUTOSWITCHLIMIT value specified was greater or equal to the ALLOCATION but in turn got rounded down, and this rounded down value is lesser than the ALLOCATION.

Action: Specify a higher value of AUTOSWITCHLIMIT.

JNLWRERR, Error writing journal file jjjj. Unable to update header Region: yyyy

Run Time/MUPIP Error: This indicates that YottaDB encountered an error while updating the journal file header as part of trying to open the journal file.

Action: Review the accompanying message(s) for detail on the cause of the error. YottaDB automatically closes the current journal file and creates a new one. To reestablish durability, perform MUPIP BACKUP to create a fresh set of journals consistent with the database.

MSTACKSZNA, User-specified M stack size of SSSS KiB not appropriate; must be between LLLL KiB and MMMM KiB; reverting to VVVV KiB

Run Time Information: The gtm_mstack environment variable species an M stack size outside the range YottaDB supports, where LLLL and MMMM are the lower and upper bounds respectively; VVVV is the value actually used.

Action: None required immediately as the process operates with the reported size M stack, however it would be preferable to eliminate such messages by setting gtm_mstack to a value in the supported range.

PREMATEOF, Premature end of file detected

MUPIP/Run Time Error: A file read or write detected an end-of-file when it was expecting additional records.

Action: Analyze accompanying messages for the type of file on which the operation failed. If the operation was a MUPIP LOAD, refer to the About this Manual section on MUPIP LOAD errors. If the circumstances warrant, contact the group responsible for database integrity at your operation with all the diagnostic context you can gather.

SYSUTILCONF, Error determining the path for system utility. tttt

Run Time Error: tttt represents text describing details of an issue finding a POSIX function that it needed.

Action: Check for aliases or environment variables related to paths that might be interfering with YottaDB's ability to invoke functions.

Tarball Hashes

sha256sum	File
dab80777a1d4d5be52f7c5d0749d8701f383aa8601eb9037884b14512d2b854d	yottadb_r122_linux_armv6l_pro.tgz
62dce638fee3fbac38de2c0f495730ab52abd05bf622d6ab7c89c98718dd5185	yottadb_r122_linux_armv7l_pro.tgz
7c187371924429e30d3254cd17fc1b9d9fb3ac3792254880bec250be728ef056	yottadb_r122_linux_x8664_pro.tgz
ac4cc25b1d1de4e3571aa8a5b448a584931095f9e7f1a9208c034ecc42d6f7f5	yottadb_r122_rhel7_x8664_pro.tgz
44678b658fe6d1396b02250c837826c97aaf2706d9963a70aac6d0ea78085c9b	yottadb_r122_src.tgz

Legal Stuff

Copyright © 2018 YottaDB LLC

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts and no Back-Cover Texts.

YottaDB™ is a trademark of YottaDB LLC. GT.M™ is a trademark of Fidelity National Information Services, Inc. Other trademarks belong to their respective owners.

This document contains a description of YottaDB and the operating instructions pertaining to the various functions that comprise the software. This document does not contain any commitment of YottaDB LLC. YottaDB LLC believes the information in this publication is accurate as of its publication date; such information is subject to change without notice. YottaDB LLC is not responsible for any errors or defects.

yottadb_r120_linux_x8664_pro.tgz

yottadb_r120_rhel7_x8664_pro.tgz

yottadb_r120_linux_armv7l_pro.tgz

yottadb_r120_linux_armv6l_pro.tgz

yottadb_r120_src.tgz

YottaDB r1.20

Release Note Revision History

Revision	Date	Summary
1.00	March 29, 2018	r1.20 Initial Release

Contact Information

YottaDB LLC

40 Lloyd Avenue, Suite 104 Malvern, PA 19355, USA [email protected] +1 (610) 644-1898

Support

Customers

Contact your YottaDB support channel.

Others

For free (to you) support from members of communities who run widely available applications on YottaDB, please use an application-specific list where appropriate.

• For issues specific to the use of YottaDB from node.js via nodem, QewdJSor Enterprise Web Developer, post to the Enterprise Web Developer community.

• For issues specific to the use of YottaDB with VistA flavors, post to the Hardhats list.

• For issues specific to the use of YottaDB with M other than for applications above, post to the comp.lang.mumps list.

• If you are not sure where to post, or for requests other than to the above communities, post an issue at https://github.com/YottaDB/YottaDB/issues and include the words "help wanted" in the summary.

r1.20

Overview

YottaDB r1.20 is the most significant release to date from the YottaDB team. With the ability to call the data management engine directly from C, and eliminating the need for any application code to be written in M (by separating the M language from the database without compromising either in any way), YottaDB r1.20 represents a historic milestone for the YottaDB/GT.M code base.

YottaDB r1.20 is built on (and except where explicitly noted, upward compatible with) both YottaDB r1.10 and GT.M V6.3-003A. Highlights of enhancements and fixes made by the YottaDB team include:

• A C API to call the data management engine directly. As C is the lingua franca of computer languages, this makes the engine accessible from other languages that can call C APIs. In the future, we anticipate creating standard wrappers to the engine from other languages, and we invite members of the community to do so as well.

• An all-new manual, the Multi-Language Programmers Guide to access the C API.

• At US $5 in retail quantities, a new Supported platform, the Raspberry Pi Zero is the lowest cost platform ever for the code base. This and the halving of the storage footprint of the database engine itself (14-15MB), makes YottaDB attractive for Internet of Things applications. Our blog post Edge Computing and YottaDB Everywhere touches on the benefits of YottaDB on low cost computing platforms.

• Docker containers make it easy to get up and running with YottaDB for experimentation, DevOps and Microservices.

• Performance improvements (up to two orders of magnitude in specific test cases) in large local arrays and garbage collection, especially in applications with large numbers of strings that change often.

• The READ command supports a buffer of the same number of prior inputs that direct mode does.

Highlights from the upstream V6.3-003A code base, thanks to the GT.M team include:

• Performance improvements when there are a large number of concurrent locks.

• Introduced as field test grade functionality in a production release, the ability for a process to associated itself with different instances at run-time.

As is the case in every release, YottaDB r1.20 brings other enhancements, e.g., callin when an application is already inside a transaction processing fence, and sending syslog messages to stderr when there is no syslog, as well as fixes to issues as detailed below.

As always, there as other enhancements and fixes, as noted below. See our Get Started page to use YottaDB.

Platforms

A platform is a combination of a CPU architecture and an operating system. A platform is Supported, Supportable, or Unsupported. Supported means that we have the platform in our development environment and test each release on that platform. Supportable means that although we do not necessarily have such a platform in our environment, we have no reason to believe that the software will not run on it. All others are Unsupported.

CPU Architecture	Supported OS Version(s)	Notes
64-bit x86	Ubuntu 16.04 LTS; Red Hat Enterprise Linux 7.4	Note that there are separate binary distributions for Ubuntu and Red Hat, owing to differences in library versions of those distributions.
Raspberry Pi 3 Model B; BeagleBone Black Wireless	Raspbian GNU/Linux 9.1; Stretch IoT (non GUI)	While YottaDB r1.20 is Supportable on other ARMv7-A CPUs, owing to variations in the implementations of ARM microarchitectures, we recommend that you ensure the software runs correctly before committing to any specific hardware other than those Supported. Please contact [email protected] if you want a specific combination of OS and CPU microarchitecture to be Supported.
Raspberry Pi Zero	Raspbian GNU/Linux 9.1	While YottaDB r1.20 is Supportable on other ARMv6 CPUs, owing to variations in the implementations of ARM microarchitectures, we recommend that you ensure the software runs correctly before committing to any specific hardware other than those Supported. Please contact [email protected] if you want a specific combination of OS and CPU microarchitecture to be Supported.

Recent releases of major GNU/Linux distributions with contemporary kernels, glibc and ncurses are Supportable. Running on Arch Linux requires the ncurses5-compat-libs package to be installed.

Getting Started

See our Get Started page to use YottaDB.

We strongly recommend that you install YottaDB r1.20 in a newly created directory, different from those of YottaDB r1.10 and any GT.M versions you may have installed on the system.

Removing an Installed YottaDB Release

Assuming $ydb_dist points to the directory where YottaDB is installed:

• Cleanly shut down all application processes using that release.
• Execute mupip rundown && mupip rundown -relinkctl
• Ensure that there are nogtcm* or gtmsecshr processes active.
• Use sudo lsof | grep $ydb_dist to ensure there are no open files.
• Delete the directory with sudo rm -rf $ydb_dist

Upgrading to YottaDB r1.20

As YottaDB r1.20 is upward compatible from both YottaDB r1.10 and GT.M V6.3-003/-003A, the minimal upgrade steps are:

• Install YottaDB r1.20.
• Recompile any object code, and recreate shared libraries where appropriate.
• If you are using encryption, compile and install the reference implementation plugin or your customized plugin.
• Cleanly shut down the application and ensure that the database files are shut down using mupip rundown from the prior release.
• Switch journal files with the new YottaDB release.
• Start using the new YottaDB release.

To upgrade from older GT.M releases, first upgrade to GT.M V6.0-000 or later and follow the steps above, or contact your YottaDB support channel for assistance and guidance.

A more sophisticated upgrade technique is:

• Install YottaDB r1.20.
• Create a new replicated instance of your application (on the same system or a different system).
• Assuming the existing instance is A, and the new instance is B, start replicating from A to B.
• Once B catches up, switchover so that B is in a primary role replicating to A.
• Once you are satisfied with B, remove (or upgrade) A.

Change History

r1.20

ID	Category	Summary
(https://github.com/YottaDB/YottaDB/issues/23)	Admin	Change references to GT.M into references to YottaDB - shared libraries
(https://github.com/YottaDB/YottaDB/issues/33)	Other	Reduce footprint of engine
(https://github.com/YottaDB/YottaDB/issues/59)	Data	Directly access YottaDB data from C
(https://github.com/YottaDB/YottaDB/issues/77)	Other	Fix reporting glitch in Indirection cache hit ratio when cache hits are greater than 43 million
(https://github.com/YottaDB/YottaDB/issues/80)	Data	Improve performance of large local arrays
(https://github.com/YottaDB/YottaDB/issues/83)	Language	Recall history for READ command to match direct mode
(https://github.com/YottaDB/YottaDB/issues/85)	Data	Stringpool garbage collector performance enhancements
(https://github.com/YottaDB/YottaDB/issues/90)	Language	YottaDB correctly runs M programs which had PATNOTFOUND errors at compile time
(https://github.com/YottaDB/YottaDB/issues/94)	Admin	Ignore gtm_dist environment variable
(https://github.com/YottaDB/YottaDB/issues/97)	Admin	Customize YottaDB message prefix
(https://github.com/YottaDB/YottaDB/issues/99)	Other	ydbinstall.sh correctly reports "YottaDB" or "GT.M" on successful installation
(https://github.com/YottaDB/YottaDB/issues/100)	Language	Timeouts specifying sub-millisecond resolutions time out correctly
(https://github.com/YottaDB/YottaDB/issues/109)	Admin	ydb_repl_filter_timeout environment variable to control replication filter timeout
(https://github.com/YottaDB/YottaDB/issues/115)	Admin	Environment variables ydb_dbglvl, ydb_gbldir, ydb_maxtptime are initialized
(https://github.com/YottaDB/YottaDB/issues/118)	Admin	MUPIP SET JOURNAL issues JNLCRESTATUS error when unable to rename current journal file
(https://github.com/YottaDB/YottaDB/issues/120)	Language	$ZEOF set correctly for files in /proc filesystem
(https://github.com/YottaDB/YottaDB/issues/122)	Admin	Simpler MUPIP JOURNAL ROLLBACK recovery following abnormal termination of process inside transaction commit logic
(https://github.com/YottaDB/YottaDB/issues/126)	Other	ydb script to run YottaDB, and files ydb_env_set & ydb_env_unset to source to set and clear environment
(https://github.com/YottaDB/YottaDB/issues/136)	Admin	Establish replication connections more efficiently in an edge case
(https://github.com/YottaDB/YottaDB/issues/141)	Language	ZSTEP OVER and ZSTEP OUTOF work correctly across extrinsic function returns using QUIT @ syntax
(https://github.com/YottaDB/YottaDB/issues/143)	Other	libyottadbutil.so created when YottaDB for Linux/ARM is installed with UTF-8 support
(https://github.com/YottaDB/YottaDB/issues/145)	Language	DIVZERO error correctly issued in certain edge cases of dividing by zero
(https://github.com/YottaDB/YottaDB/issues/150)	Admin	Fixes to multiple issues affecting databases with READ_ONLY set
(https://github.com/YottaDB/YottaDB/pull/160)	Other	Use ydb and ydb_env_set scripts instead of gtm and gtmprofile scripts, and ydb_prefix environment variables
(https://github.com/YottaDB/YottaDB/pull/161)	Other	Change relative path to ydbmerrors to use src directory
(https://github.com/YottaDB/YottaDB/pull/162)	Other	Create Docker Images
(https://github.com/YottaDB/YottaDB/issues/169)	Data	Certain edge cases of nested triggers work correctly without abnormal process termination
(https://github.com/YottaDB/YottaDB/issues/184)	Language	C program that invokes call-in leaves terminal characteristics in sane state on exit
(https://github.com/YottaDB/YottaDB/issues/186)	Language	EXCEPTION handler correctly executes when Ctrl-C entered
(https://github.com/YottaDB/YottaDB/issues/188)	Language	Call-ins permitted inside TP
(https://github.com/YottaDB/YottaDB/issues/189)	Admin	Send messages to stderr if syslog not present or not functional

Admin

• The file libyottadb.so contains the runtime logic that was previously in libgtmshr.so, which is now a relative symbolic link to libyottadb.so. Similarly, libyottadbutil.so contains the object code for utility routines, and libgtmutil.so is a relative symbolic link to libyottadbutil.so. If UTF-8 support is installed, a similar change also occurs in the utf8 subdirectory. There should be no change to the behavior of any application program or scripting that does not explicitly check the nature of libgtmshr.so and libgtmutil.so. (https://github.com/YottaDB/YottaDB/issues/23)
• YottaDB ignores the environment variable gtm_dist, deriving any needed information from within the running process. Previously, it required $gtm_dist to contain the name of the directory from which it ran. (https://github.com/YottaDB/YottaDB/issues/94)
• The environment variable ydb_msgprefix specifies a prefix for YottaDB messages generated by a process, with the prefix defaulting to "YDB", e.g., YDB-I-DBFILEXT. Previously, the prefix was always "GTM". A value of "GTM" retains the previous format. (https://github.com/YottaDB/YottaDB/issues/97)
• An integer value in seconds for the environment variable ydb_repl_filter_timeout sets a limit for the Source Server to await a response from an external filter program. The default value is 64 seconds; a value less than 32 is treated as 32 seconds; and a value greater than 131072 (2**17) is treated as 131,072 seconds. Set a value for ydb_repl_filter_timeout if, for example, your filter program is functionally correct, but needs more time to respond. Previously, the value was always 64 seconds. (https://github.com/YottaDB/YottaDB/issues/109)
• At process startup, YottaDB initializes the following intrinsic special variables: (1) $zgbldir to the global directory file pointed to by $ydb_gbldir. If ydb_gbldir is not defined, YottaDB uses the gtmgbldir environment variable instead. (2) $zmaxtptime to the number of seconds specified by $ydb_maxtptime. If ydb_maxtptime is not defined, YottaDB uses the gtm_zmaxtptime environment variable instead, defaulting to 0 seconds (infinite timeout).To facilitate debugging application memory allocation bugs, the environment variable ydb_dbglvl optionally provides debugging flags as specified in the file gtmdbglvl.h. If ydb_dbglvl is not defined, YottaDB uses the gtmdbglvl environment variable instead. If neither is defined, no memory allocation debugging is turned on. Previously YottaDB ignored the ydb_dbglvl, ydb_gbldir, and ydb_maxtptime environment variables. (https://github.com/YottaDB/YottaDB/issues/115)
• MUPIP SET JOURNAL issues a JNLCRESTATUS error in case it is not able to rename the current journal file (for example, because of read-only permissions on the directory containing the journal file) before creating the new journal file. In YottaDB r1.10, the MUPIP command used to abnormally terminate with a SIG-11.(https://github.com/YottaDB/YottaDB/issues/118)
• MUPIP JOURNAL ROLLBACK works correctly in case a process updating multiple regions in a TP transaction terminates abnormally (e.g., kill -9). In YottaDB r1.10 (and GT.M V6.3-002), it was possible for the rollback to fail with a DUPTOKEN error in rare cases, depending on where in the transaction commit logic the process was killed. The workaround was to rerun the MUPIP JOURNAL ROLLBACK with a non-zero ERROR_LIMIT qualifier value to allow DUPTOKEN errors. In addition, NULL records are placed in the correct file (journal extract file, broken transaction file, lost transaction file). Previously, they could be incorrectly placed in the lost transaction file instead of the broken transaction file. Finally, the NULL records are extracted in the correct order (journal sequence number) in the extract file (or broken transaction or lost transaction file). Previously, the extract file would be sorted in terms of journal sequence number except for the NULL type of journal records (00 record type in the first column of the extract file) which could be placed in arbitrary order. Note that such a transaction remains Atomic, because the process termination means the transaction was never committed. Note also that YottaDB recommends terminating processes with MUPIP STOP and not kill -9. (https://github.com/YottaDB/YottaDB/issues/122)
• Initiating replication connections between Source and Receiver Servers is more efficient. Previously, in rare cases, the Source Server unnecessarily disconnected the connection and reconnected. (https://github.com/YottaDB/YottaDB/issues/136)
• Multiple issues with READ_ONLY features from GTM-8735 in the upstream code base that generated errors while accessing the help database were addressed and corrected. More details: (https://github.com/YottaDB/YottaDB/issues/150)
• In environments without a syslog facility (such as default Docker containers), messages intended for the syslog go to stderr of the YottaDB process. Previously, they appeared on the user's interactive session. YottaDB uses the existence of /dev/log to decide whether a syslog faclity exists. As syslog messages record important information about the operation of the YottaDB engine, we strongly recommend a syslog facility in all cases except single-user development environments. (https://github.com/YottaDB/YottaDB/issues/189)

Data

• The YottaDB engine is directly accessible using a C-callable API. See https://docs.yottadb.com/MultiLangProgGuide/index.html for user documentation. (https://github.com/YottaDB/YottaDB/issues/59)
• Local arrays with large number of subscripts scale much better. When the number of nodes in a local array is in the millions, node creation time is now noticeably faster (https://github.com/YottaDB/YottaDB/issues/80)
• Applications with large numbers of strings, which can occur with large numbers of local variables, local variables with many nodes, or both, and whose performance is limited by garbage collection, i.e. applications where the strings change frequently, run much faster. In one test case, the improvement was two orders of magnitude. (https://github.com/YottaDB/YottaDB/issues/85)
• Nested database triggers (i.e. database triggers that invoke code to update global variables, which in turn invoke other triggers) work correctly. Previously, it was possible in rare cases involving multiple processes loading triggers at the same time for the processes to abnormally terminate with fatal SIGABRT (SIG-6) errors. (https://github.com/YottaDB/YottaDB/issues/169)

Language

• With the [NO]EDITING deviceparameter set to EDITING, the number of previous inputs to the READ command from a terminal device that can be recalled is a circular buffer of 99 entries, matching the size of the RECALL command buffer for direct mode. READ X#, READ *X, and READ X all share the same history. Empty inputs are excluded from the buffer, and multiple consecutive occurrences of the same input are stored once. The up-arrow key goes back in history, and the down-arrow key goes forward, towards more recent inputs. Previously the READ buffer was limited to just the previous line. (https://github.com/YottaDB/YottaDB/issues/83)
• YottaDB correctly runs M programs which had PATNOTFOUND errors at compile time. Previously, in r1.10 it was possible for mumps processes to terminate abnormally with a SIG-11 as a consequence of a defect in the GT.M V6.3-002 code base.(https://github.com/YottaDB/YottaDB/issues/90)
• Timeouts in JOB, LOCK, OPEN, READ, WRITE /WAIT, WRITE /LISTEN, WRITE /ACCEPT, and WRITE /TLS commands with more than three digits after the decimal point time out shortly after the requested time has elapsed. Previously, in r1.10, timeouts with more than three digits after the decimal point would be treated as 2Gi msec (≃24.8 days), owing to a defective enhancement to allow fractional timeouts introduced in the GT.M V6.3-002 code base GTM-5250 (https://github.com/YottaDB/YottaDB/issues/100)
• $ZEOF is set correctly for files in the /proc file system. Previously, $ZEOF used to incorrectly return 1 after reading one line of such files even though the file had many more lines. (https://github.com/YottaDB/YottaDB/issues/120)
• ZSTEP OVER and ZSTEP OUTOF work correctly across extrinsic function calls which return using the QUIT @ syntax. Previously, the ZSTEP would not pause (and execute the ZSTEP action) after the return from such function calls. (https://github.com/YottaDB/YottaDB/issues/141)
• YottaDB issues a DIVZERO error when the divisor in a division operation is 0. In previous versions (r1.10 and r1.00 as well as all GT.M versions up to V6.3-003A), dividing by 0 in some numeric expressions resulted in a fatal KILLBYSIGSINFO1/SIGINTDIV error (for example set x=2E20 write 1/(x*0)). (https://github.com/YottaDB/YottaDB/issues/145)
• A C program that invokes a call-in leaves the terminal in a sane state when it exits. Previously, some terminal characteristics would be changed (for example, character echo would be disabled) on exit, requiring an stty sane command to restore them. Additionally, YottaDB now changes the terminal characteristics only when necessary (i.e. a READ or WRITE from/to the terminal, or direct mode). Previously, it used to change them unconditionally at process startup. (https://github.com/YottaDB/YottaDB/issues/184)
• A USE statement for a terminal device with parameters (CTRAP=$C(3):EXCEPTION="…") executes the specified M code fragment when a Ctrl-C is entered. Previously, the Ctrl-C was ignored. A workaround was to also specify the NOCENABLE option. (https://github.com/YottaDB/YottaDB/issues/186)
• Call-ins are allowed even if a TP transaction is active (process is within a TSTART/TCOMMIT fence). Previously, a call-in while already in TP was not permitted, resulting in a CITPNESTED error. (https://github.com/YottaDB/YottaDB/issues/188)

Other

• The YottaDB install directory size is 14-15Mb (down from 34Mb in prior versions). (https://github.com/YottaDB/YottaDB/issues/33)
• When run with ydb_dbglvl / gtmdbglvl set as described in (https://github.com/YottaDB/YottaDB/issues/115), YottaDB reports correct ratios when the number of indirection cache hits exceeds 43 million. Previously, it could report a negative number. Note that ydb_dbglvl / gtmdgblvl is not part of the published and supported API whose stability we strive to maintain, and exists to assist YottaDB in supporting customers. (https://github.com/YottaDB/YottaDB/issues/77)
• When asked to install GT.M using the gtm qualifier, the ydbinstall.sh script reports "GT.M" on a successful install. Previously, it reported "YottaDB" unconditionally, whether it installed YottaDB or GT.M. (https://github.com/YottaDB/YottaDB/issues/99)
• For "out of the box" use of YottaDB, ydb_env_set is a file you can source with a POSIX or compatible shell to configure an environment with a default structure and required environment variables, creating a default environment if one does not exist. At this time, support has not been implemented in YottaDB for all ydb_ prefixed environment varables, but each release will increase that set. So as to not require changes as future YottaDB releases add YottaDB counterparts to GT.M environment variables, sourcing ydb_env_set sets both sets of environment variables to appropriate values, which are usually, but not always, the same. Sourcing ydb_env_unset unsets the above environment variables, unsets the aliases, and removes any occurrence of $ydb_dist in $LD_LIBRARY_PATH. For more information, see Issue (https://github.com/YottaDB/YottaDB/issues/126)
• Installing YottaDB with UTF8 support builds $gtm_dist/utf8/libyottadbutil.so (previously named libgtmutil.so) on the Linux/ARM platform. In r1.10, libgtmutil.so was built only in the $gtm_dist (non-utf8) directory. (https://github.com/YottaDB/YottaDB/issues/143)
• When installing YottaDB, gtm is created as a symbolic link to ydb, and gtmprofile is a symbolic link to ydb_env_set. The default names for global directory, database files, and journal files are changed, but if current files exist, their environment setting is done with an M program rather than with shell commands. (https://github.com/YottaDB/YottaDB/pull/160)
• The YDB_SOURCE_DIR cmake variable is used to get the path to ydbmerrors.h instead of a hardcoded relative path (https://github.com/YottaDB/YottaDB/pull/161)
• A Docker image to build and deploy a YottaDB runtime environment is available. (https://github.com/YottaDB/YottaDB/pull/162)

GT.M V6.3-003A

ID	Category	Summary
GTM-8880	Language	Fix issue with (non-default) Standard Boolean evaluation with side-effects
		and certain patterns
GTM-8887	Other	Fix rare timer issue
GTM-8889	Other	Prevent UNDEF error after <CTRL-C> within ZHELP navigation

GT.M V6.3-003

ID	Category	Summary
GTM-4212	Admin	MUPIP better deals with over length file names
GTM-6115	Language	Please see GTM-8792
GTM-7986	Language	Warning on implicit wrapping of source lines exceeding maximum supported length
GTM-8182	DB	Allow updating globals belonging to different instances
GTM-8186	Language	Accept offset alone for an entryref in DO, GOTO and ZGOTO
GTM-8587	Language	Maintain $DEVICE and $KEY for all supported devices
GTM-8617	Admin	MUPIP SET supports N[ULL_SUBSCRIPTS] and STD[NULLCOLL] qualifiers.
GTM-8680	DB	LOCK Improvements
GTM-8732	Admin	Better validation for MUPIP REPLICATE -LOG_INTERVAL and -HELPER, and MUPIP SET -DEFER_TIME
GTM-8735	Admin	READ_ONLY characteristic to prevent state changes to MM databases
GTM-8754	Other	Prevent odd ASYNCIO deadlock
GTM-8767	Admin	MUPIP SET -HARD_SPIN_COUNT and -SPIN_SLEEP_MASK support
GTM-8769	Language	Syntax check $ETRAP, $ZSTEP, $ZTRAP, and EXCEPTION when specified
GTM-8779	Admin	Freeze Notification
GTM-8780	Language	Fix $SELECT() handling of certain syntax errors
GTM-8781	Other	Prevent memory leak in ZSYSTEM
GTM-8786	Language	$NAME() of a naked reference returns any current extended reference
GTM-8787	Admin	MUPIP JOURNAL -EXTRACT='-stdout' doesn't explode at termination if stdout is gone
GTM-8788	Language	The compiler excludes BLKTODEEP lines from the object files
GTM-8789	Language	Prevent NEW $ZGBLDIR from setting up an Update Process failure
GTM-8790	DB	Retain any extended first reference in $REFERENCE when sharing statistics
GTM-8792	Language	Prevent keys that exceed the supported maximum string length
GTM-8794	Admin	MUPIP RUNDOWN -OVERRIDE works on a non-MUPIP backup made during an Instance Freeze
GTM-8795	DB	Journal Updates promptly during MUPIP FREEZE -ONLINE
GTM-9796	DB	Improved error handling during TP and mini transaction commits
GTM-8797	Admin	Installation script fixes
GTM-8798	Admin	MUPIP ENDIANCVT converts Mutex Queue Slots
GTM-8799	Other	Improve performance for a pattern of local variable creation
GTM-8801	Other	cmake build produces appropriate support for the ^%YGBLSTATS utility.
GTM-8804	Language	ZSHOW "T" option to return summary for ZSHOW "GL"
GTM-8805	DB	Fix to havesting of LOCKs abandoned by an abnormally terminated process
GTM-8832	Language	Appropriately report NUMOFLOW for string literal with a huge value when used as a number
GTM-8839	Language	$DEVICE shows the full error message
GTM-8840	Admin	Normalized gtmsecshr message severities
GTM-8842	Admin	ZBREAK and ZSTEP restricted in triggers when TRIGGER_MOD is restricted
GTM-8844	Admin	Restriction available for HALT and ZHALT; ZGOTO 0 can return a non-zero status to the shell
GTM-8846	Admin	GT.M accepts multi-slash journal file names
GTM-8847	Language	Provide a way to detect and limit process private heap storage
GTM-8849	Other	Help databases built with make files have QDBRUNDOWN and NOGVSTATS characteristics
GTM-8850	DB	Allow process exit when MUPIP FREEZE -ONLINE is in place
GTM-8854	Language	Compiler handles a syntax error after a literal postconditional that's FALSE
GTM-8855	Other	Prevent memory leak from an error locating a global directory
GTM-8856	Language	Defer failing evaluations of literal pattern matches to run time
GTM-8857	Language	Improve error detection for certain pattern match cases
GTM-8858	DB	Improve available information in cases of apparent database integrity issues
GTM-8859	Admin	MUPIP ROLLBACK handles idle regions better
GTM-8866	Language	Prevent timeouts with more than three decimal digits from being too long
GTM-8873	DB	Prevent occasional $ORDER(,-1) problem

More Information

(Section Blank for this Release)

Messages

From YottaDB

CALLINTCOMMIT, TCOMMIT at call-in-level=xxxx not allowed as corresponding TSTART was done at lower call-in-level=yyyy.

Run Time Error: This indicates that at least one call-in invocation happened in between when the TP transaction started (either through a ydb_tp_s() call in C or a TSTART command in M) and when the corresponding transaction commit is attempted (through a TCOMMIT command in M).

Action: If a TP transaction is started using SimpleAPI, and the user function driven by ydb_tp_s() does a call-in invocation, care should be taken to ensure the call-in code does not do a TCOMMIT.

CALLINTROLLBACK, TROLLBACK at call-in-level=xxxx not allowed as corresponding TSTART was done at lower call-in-level=yyyy

Run Time Error: This indicates that at least one call-in invocation happened in between when the TP transaction started (either through a ydb_tp_s() call in C or a TSTART command in M) and when the corresponding transaction rollback is attempted (through a TROLLBACK command in M).

Action: If a TP transaction is started using SimpleAPI, and the user function driven by ydb_tp_s() does a call-in invocation, care should be taken to ensure the call-in code does not do a TROLLBACK.

FATALERROR1, Fatal error raised. Generating core and terminating process. Error: .

Run Time Error: This indicates that there was a fatal error in a SimpleAPI call that resulted in the termination of the running process and the generation of a core file. Appears in the system log.

Action: Look up the error indicated in the secondary message text in the documentation to correct the cause of the fatal error.

FATALERROR2, Fatal error raised. Bypassing core generation and terminating process. Error:

Run Time Error: This indicates that there was a fatal error in a SimpleAPI call that resulted in the termination of the running process, and no core file was generated as a result of this. Appears in the system log.

Action: Look up the error indicated in the secondary message text in the documentation to correct the cause of the fatal error.

INSUFFSUBS, Return subscript array for an API call too small.

Run Time Error: This indicates that the return subscript array needs more entries for the ydb_node_next_s() or ydb_node_previous_s() SimpleAPI call than is currently allocated (specified by the input/output parameter *ret_subs_used). In this case *ret_subs_used is set to the needed entries.

Action: Ensure the return subscript array ("ret_subsarray" parameter of ydb_node_next_s() or ydb_node_previous_s()) is allocated with at least *ret_subs_used entries and retry the ydb_node_next_s() or ydb_node_previous_s() call.

INVNAMECOUNT, Number of varnames (namecount parameter in a rrrr call) cannot be less than zero.

Runtime Error: This indicates that the number of variable names specified in a SimpleAPI call (identified in the message text) is less than zero.

Action: Redo the SimpleAPI call with a number of variable names that is greater than or equal to zero.

INVVARNAME, Invalid local/global/ISV variable name supplied to API call.

Run Time Error: This indicates that a SimpleAPI call received an invalid variable name. The invalidity can be one of the following types:

a) The ydb_buffer_t structure corresponding to the variable name has a "len_used" field greater than "alloc_len" OR b) The ydb_buffer_t structure corresponding to the variable name has a zero value of "len_used" OR c) The ydb_buffer_t structure corresponding to the variable name has a non-zero value of "len_used" but a NULL value of "buf_addr" OR d) The variable name starts with a ^ (i.e. is a global variable name), but the second character is not a % or an alpha character (lower or upper case) or at least one of the following characters is not an alphanumeric character (lower or upper case alphabet or a decimal digit) OR e) The variable name starts with a $ (i.e. is an intrinsic special variable name), but is not followed by any other character (i.e. "len_used" has a value of 1) OR f) The variable name starts with a character other than a % or an alpha character (lower or upper case) OR g) The variable name starts with a % or alpha character (lower or upper case) but at least one of the following characters is not an alphanumeric character (lower or upper case alphabet or a decimal digit)

Action: Determine which of the described failures scenarios is the issue and accordingly fix the variable name passed in to the SimpleAPI call

LIBYOTTAMISMTCH, $ydb_dist/libyottadb.so does not match the shared library path.

Runtime Error: This indicates that the full path of the currently running libyottadb.so shared library does not match the path described by $ydb_dist. This is possible for example if a C program tries to directly invoke a base image function (e.g. gtm_main, dse_main, mupip_main etc.) for more than one build/release of YottaDB in the same process.

Action: Make sure a C program invokes a base image function of only one libyottadb.so executable.

MINNRSUBSCRIPTS, Number of subscripts cannot be a negative number.

Run Time Error: This indicates that the number of subscripts in an input array (usually the "subs_used" parameter in various SimpleAPI calls) is a negative number.

Action: Redo the SimpleAPI call with a subscript count that is greater than or equal to zero.

MIXIMAGE, Cannot load more than one base image function on a process.

Run Time Error: This indicates that a C function tries to invoke more than one base image function included in libyottadb.so (e.g. gtm_main, dse_main, mupip_main etc.). Only one base image function can be invoked and only once for the lifetime of the process.

Action: Make sure only one base image function is invoked for the lifetime of one process.

NAMECOUNT2HI, Number of varnames specified (namecount parameter in a rrrr call) exceeds maximum cccc allowed.

Runtime Error: This indicates that the number of variable names specified in a SimpleAPI call (identified in the message text) exceeds the maximum number of allowed variable names (also identified in the message text).

Action: Redo the SimpleAPI call with a fewer number of variable names specified.

PARAMINVALID, Invalid parameter dddd specified in an API (rrrr) call.

Run Time Error: This indicates that a parameter in a SimpleAPI call was not properly specified. The function name (e.g. ydb_set_s()) and the name of the invalid parameter (e.g. subsarray) along with the type of the invalidity is identified in the error message text. If the parameter is an array, the index of the element where the invalidity is detected is also identified. If the parameter is an input parameter of type ydb_buffer_t it is invalid if "len_used" is greater than "alloc_len" OR if it has a "len_used value of 0 but a NULL value of "buf_addr". If the parameter is an output parameter, it is invalid if the ydb_buffer_t pointer is NULL or if the "buf_addr" field in the ydb_buffer_t structure is NULL. Note that no error checks are done if an input ydb_buffer_t typed pointer parameter is NULL (the process would get a SIG-11 and dump core in that case).

Action: Fix the cause of the invalidity and pass in a valid parameter to the SimpleAPI call.

QUERY2, Invalid second argument to $QUERY. Must be -1 or 1.

Run Time Error: This indicates that there is an invalid second argument passed to the function $QUERY. It must be either -1 or 1.

Action: Refer to $QUERY in the Programmer's Guide <https://docs.yottadb.com/ProgrammersGuide/functions.html#query>_ for correct usage.

READONLYLKFAIL, Failed to get a lock on READ_ONLY database file.

Run Time Error: This error is issued by a MUPIP command that requires standalone access (e.g. MUPIP SET -NOREAD_ONLY) to a database file (which has Read-only mode turned on) if other processes are still accessing the database OR by any process that tries to open a database file (which again has Read-only mode turned on) while a MUPIP command that has standalone access on the same database file is concurrently running.

Action: If the error is from the MUPIP command which requires standalone access, ensure all processes which have the database file open are shut down and reattempt the command. If the error is from a process trying to open the database file, wait for the concurrent MUPIP command requiring standalone access to finish and reattempt to open the database.

READONLYNOSTATS, Read-only and Statistics sharing cannot both be enabled on database.

Run Time Error: This error is issued if if one tries to enable the Read-only mode on a database that has Statistics sharing turned on OR if one tries to enable Statistics sharing on a database that has Read-only mode turned on OR if one tries to enable both at the same time.

Action: Make sure at most one of Read-only or Statistics sharing is turned on in the database at any point in time.

SIMPLEAPINEST, Attempt to nest a SimpleAPI call (rrrr) with another SimpleAPI call (RRRR) - - nesting calls is not permitted in simpleAPI.

Run Time Error: This indicates that a SimpleAPI call (function name identified in the message text) was attempted while another SimpleAPI call (whose function name is also identified in the message text) is still running (possible for example through a call-in or trigger invocation). Nesting of such SimpleAPI calls is not currently permitted.

Action: Avoid nesting SimpleAPI calls. Finish one SimpleAPI call before attempting another.

SUBSARRAYNULL, Non-zero number of subscripts xxxx specified but subscript array parameter is NULL in API call.

Run Time Error: This indicates that the value of the subscript array parameter is NULL, meaning there are no subscripts specified, but the parameter specifying the number of subscripts (usually the "subs_used" parameter) has a non-zero value.

Action: Redo the SimpleAPI call with a non-NULL subscript array parameter or with a zero value for the parameter specifying the number of subscripts.

TIME2LONG, Specified time value exceeds supported maximum limit xxxx allowed. Run Time Error: This indicates that a timer value specified in a SimpleAPI call (e.g. ydb_lock_s(), ydb_lock_incr_s() etc.) exceeded the maximum allowed limit. Both the specified time value and the maximum allowed limit are indicated in the message text.

Action: Specify a time value below the maximum limit and retry the SimpleAPI call.

VARNAME2LONG, Variable name length exceeds maximum allowed length xxxx.

Run Time Error: This indicates that the length of a variable name specified in a SimpleAPI call exceeded the maximum limit. The maximum value is identified in the message text.

Action: Specify the variable name within the maximum length limit and retry the SimpleAPI call.

From GT.M

DBFREEZEOFF, Region rrrr is UNFROZEN ([NO]OVERRIDE [NO]AUTOREL)

Operator log/MUPIP Information: The database region rrrr is no longer frozen, most likely due to a MUPIP FREEZE -OFF, with the selected options. [NO]AUTOREL indicates whether an autorelease of the region occurred prior to the MUPIP FREEZE -OFF command.

Action: Confirm that this was the desired action.

DBFREEZEON, Region rrrr is FROZEN ([NO]OVERRIDE [NO]ONLINE [NO]AUTOREL)

Operator log/MUPIP Information: The database region rrrr is frozen, most likely due to a MUPIP FREEZE -ON, with the reported options.

Action: Confirm that this was the desired action.

DBNONUMSUBS, XXXX Key contains a numeric form of subscript in a global defined to collate all subscripts as strings

Run Time/MUPIP Error: The record has a numeric subscript but the collation setting for the global or region indicates all subscripts are filed as strings. The leading context (XXXX) identifies the block and offest of the problematic record. This can arise if an operator uses DSE to force a change to a collation setting or to modify a key when the global already has content.

Action: If you can determine the cause of, and reason for, the change and you may choose to reverse it. If you need to change the collation, the appropriate procedure is to EXTRACT the data, KILL the global, or remove and recreate the database file, and them LOAD the extracted data.

DBNULCOL, XXXX NULL collation representation differs from the database file header setting

DSE/MUPIP/Run Time Error: This indicates the database contains a record with an empty subscript ("Null" subscript) representation that is incompatible with the current setting database file header setting for such a representation. The leading context (XXXX) specifies the block number and offset of the problematic record. This can only arise if someone changes the setting for the database while it contains one or more such subscripts. YottaDB recommends against making such a change. This message can originate from MUPIP INTEG, DSE INTEG or from running with VIEW "GDSCERT"

Action: Use the record and block information to remove the problematic record with DSE and restore the data appropriately, typically with a SET command. Note that the record and block of the record many change due to ongoing updates, so this operation requires great care and familiarity with DSE.

GBLOFLOW, Database segment is full

Run Time/MUPIP Error: This indicates that an error was encountered while extending the database file.

Action: Examine the accompanying message(s) for the cause of the error. If the error is due to insufficient authorization, address that. If the error is due to TOTALBLKMAX (refer to the explaination of that message) or a lack of enough free space on the disk to fit the size of a database file, try performing a KILL of some nodes in the database to get free blocks in the existing allocated space (you may need to KILL several subscripted nodes before you can KILL a name node).

LSINSERTED, Line YYYY, source module XXXX exceeds maximum source line length; line seperator inserted, terminating scope of any prior IF, ELSE, or FOR.

Compile Time Warning: Indicates that source XXXX line YYYY exceeded the maximum line length and YottaDB separated it into multiple lines to allow continued parsing. Internally, YottaDB represents the generated code as N lines for this source line, where N is the number of segments extracted from this source line. Be aware that as a result of this, source lines containing a command whose scope is rest of the line (IF, ELSE, FOR), are now split into multiple lines, each with a separate scope.

Action: Consider refactoring code to avoid source line lengths in excess of 8192 characters.

MUTEXFRCDTERM, Mutual Exclusion subsystem detected forced termination of process pppp. Crit salvaged from database file dddd.

Run Time Warning: This indicates that YottaDB confirmed inappropriate termination of the process pppp, while holding crit on database file dddd.

Action: Determine the cause of the termination and take appropriate action.

NULSUBSC, XXXX Null subscripts are not allowed for current region

Run Time/MUPIP Error: This indicates that a global variable specified a null subscript in a database file which does not accept null subscripts. The leading context (XXXX) specifies more about the event or location of the issue

Action: Look for the source of the null subscript(s) and consider whether they are appropriate or due to a coding error. If they are appropriate, use MUPIP SET -NULL_SUBSCRIPTS, and remember to make the same adjustment with GDE CHANGE REGION -NULL_SUBSCRIPTS to ensure the next time you recreate a database that the characteristic persists.

READONLYNOBG, Read-only cannot be enabled on non-MM databases

MUPIP Error: This indicates an attempt to change a BG database to -READ_ONLY or to change a -READ_ONLY to MM access method; -READ_ONLY only compatible with the MM access mode.

Action: Verify whether the database should not be read only and adjust, if appropriate. Alternatively, set the database to MM access mode then mark it as read-only.

REPLINSTACC, Error accessing replication instance file xxxx

Run Time/MUPIP Error: This indicates that some errors were encountered while accessing the specified replication instance file defined by $gtm_repl_instance or the relevant global directory.

Action: Refer to the accompanying message(s) for additional information.

REPLINSTMISMTCH, Process has replication instance file ffff (jnlpool shmid = ssss) open but database dddd is bound to instance file gggg (jnlpool shmid =tttt)

Run Time Error: The process attempted an update on the replicated database dddd associated with the replication instance file ffff and journal pool shared memory id ssss; however, the process has already associated the database with a different replication instance file gggg or journal pool shmid tttt.

Action: A replicated database can only accept updates by processes that have the same replication instance file (defined by the environment variable gtm_repl_instance or in the global directory) open for that database. Ensure the same replication instance file is used for all processes that update the same replicated database file. This error can also occur if the replication instance file was recreated (while processes were still accessing the replication instance). In this case, the name ffff and gggg would be the same but the corresponding journal pool shared memory ids would be different. To recover from this situation, shut down all processes accessing the instance from before and after the instance file recreate. Run an argumentless MUPIP RUNDOWN to clean up the older journal pool tttt and restart the instance. The Source Server (which is the first process to start on a replicated instance) only binds replicated databases from its global directory to the journal pool that it creates. No other replicated database file can be bound with this journal pool.

REPLMULTINSTUPDATE, Previous updates in the current transaction are to xxxx so updates to yyyy (in rrrr) not allowed

Run Time Error: Previous updates in the current TP transaction mapped to database files associated with replication instance file xxxx, so it cannot make updates to database file yyyy which is associated with replication instance file rrrr.

Action: Modify the application so all updates in a TP transaction to replicated regions are associated with a single replication instance.

STACKCRIT, Stack space critical

Run Time Error: This indicates that the process has consumed almost all of the available stack space.

Action: Look for infinite recursion. If you do not take immediate action to reduce your stack, YottaDB is likely to produce a STACKOFLOW error, which terminates the process. Examine the stack with ZSHOW. Trim the stack using QUIT, ZGOTO, HALT or ZHALT.

STACKOFLOW, Stack overflow

Run Time Fatal: This indicates that the process required more stack space than was available in memory.

Action: Reduce the stack when you get a STACKCRIT error. This error terminates the process.

STPCRIT, String pool space critical

Run Time Error: This indicates that the process has exceeded the heap (string pool) limit specified in the $ZSTRPLLIM ISV. If you do not take prompt action to reduce the process memory requirements, at the next heap expansion, YottaDB produces an STPOFLOW error, which terminates the process.

Action: Investigate whether the process memory usage is appropriate, and if so, increase or remove the limit. Otherwise correct the cause(s) of the excessive memory consumption.

STPOFLOW, String pool space overflow

Run Time Fatal: This indicates that the process has previously exceeded the heap (string pool) limit specified in the $ZSTRPLLIM ISV and still needs more memory, so YottaDB terminates the process.

Action: Investigate whether the process memory usage is appropriate, and if so, increase or remove the limit. Otherwise correct the cause(s) of the excessive memory consumption.

Tarball Hashes

sha256sum	File
cd26897549405b33e63966df52aefb8ad581afd1633db1cb2723ff2c12acce25	yottadb_r120_linux_armv6l_pro.tgz
8993fbb7300cb732da06e90bc7cb1334e9ab5318da7d0b7427900be8919aa640	yottadb_r120_linux_armv7l_pro.tgz
6e7bf4c1fa0b12e29fa2b0e1629bfdaaeebd0541c458eaf561d5676d1f0fc5e6	yottadb_r120_linux_x8664_pro.tgz
e32dc5ffbdd1e8fd17d4ed2f1df97145f05d5748489f2b5d8322ad9ee33008ce	yottadb_r120_rhel7_x8664_pro.tgz
f4310725ff72ff6bd5da41fc0b3eaf5ab918978ce33d08878ed717c1d1cf04c4	yottadb_r120_src.tgz

Legal Stuff

Copyright © 2018 YottaDB LLC

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts and no Back-Cover Texts.

YottaDB™ is a trademark of YottaDB LLC. GT.M™ is a trademark of Fidelity National Information Services, Inc. Other trademarks belong to their respective owners.

yottadb_r110_src.tgz

YottaDB r1.10

Release Note Revision History

Revision	Date	Summary
1.05	October 27, 2017	Updated tarballs with new ydbinstall.sh and configure scripts (address additional fallout because the installation automation could not be fully tested in advance), update sha256sums
1.04	October 26, 2017	Add example of installation with UTF-8 support
1.03	October 26, 2017	Update ydbinstall.sh for correct installation directory default (ydbinstall.sh could not be fully tested without an actual r1.10 release), rebuild tarballs, update sha256sums
1.02	October 26, 2017	Add sha256sum for ydbinstall.sh
1.01	October 26, 2017	Update tarballs, update sha256sums, correct date
1.00	October 26, 2017	r1.10 Initial release

Contact Information

YottaDB LLC

40 Lloyd Avenue, Suite 104 Malvern, PA 19355, USA [email protected] +1 (610) 644-1898

Support

Customers

Contact your YottaDB support channel.

Others

For free (to you) support from members of communities who run widely available applications on YottaDB, please use an application-specific list where appropriate.

• For issues specific to the use of YottaDB from node.js via nodem, QewdJS or Enterprise Web Developer, post to the Enterprise Web Developer community.
• For issues specific to the use of YottaDB with VistA flavors, post to the Hardhats list.
• For issues specific to the use of YottaDB with M other than for applications above, post to the comp.lang.mumps list.
• If you are not sure where to post, or for requests other than to the above communities, post an issue at https://github.com/YottaDB/YottaDB/issues and include the words "help wanted" in the summary.

r1.10

Overview

YottaDB r1.10 is a major release that adds support for Linux on ARM CPUs. Enhancements in this release include reverse $query() - $query(,-1) - as well as optimizations for certain $order() use cases. The ydbinstall.sh script automatically downloads and installs the current YottaDB release. As always, there as other enhancements and fixes, as noted below.

YottaDB r1.10 is built on (and except as noted, upward compatible with) both YottaDB r1.00 and GT.M V6.3-002.

We would like to acknowledge and thank Steve Johnson (@sljohnson1) and Sam Habiel (@shabiel) for making the ARM port possible.

Platforms

A platform is a combination of a CPU architecture and an operating system. A platform is Supported, Supportable, or Unsupported. Supported means that we have the platform in our development environment and test each release on that platform. Supportable means that although we do not necessarily have such a platform in our environment, we have no reason to believe that the software will not run on it. All others are Unsupported.

CPU Architecture	Supported OS Version(s)	Notes
64-bit x86	Ubuntu 16.04 LTS; Red Hat Enterprise Linux 7.4	Note that there are separate binary distributions for Ubuntu and Red Hat, owing to differences in library versions of those distributions.
Raspberry Pi 3 Model B; BeagleBone Black Wireless	Raspbian GNU/Linux 9.1; Stretch IoT (non GUI)	While YottaDB r1.10 is Supportable on other ARMv7-A CPUs, owing to variations in the implementations of ARM microarchitectures, we recommend that you ensure the software runs correctly before committing to any specific hardware other than those Supported. Please contact [email protected] if you want a specific combination of OS and CPU microarchitecture to be Supported.

Recent releases of major GNU/Linux distributions with contemporary kernels, glibc and ncurses are Supportable. Running on Arch Linux requires the ncurses5-compat-libs package to be installed.

Installation

The simplest way to install the current release of YottaDB for your platform is the ydbinstall.sh script. Download it, make the file executable, and execute it:

• sudo ./ydbinstall.sh downloads and installs the current YottaDB release with a reasonable set of defaults.
• sudo ./ydbinstall.sh --utf8 default downloads and installs the current YottaDB release with UTF-8 support, using the default ICU version on the operating system.
• ./ydbinstall.sh --help prints a list of command line options.
• sudo ./ydbinstall.sh --verbose outputs information as it executes, for troubleshooting purposes (for example if Internet access is stalled by a proxy server, the script will print the wget command it is trying to execute and hang).

You can also download the YottaDB binary distribution tarball for your platform from the release notes for the latest release, unpack it, change to the directory with the unpacked files, and (a) run the ydbinstall therein or (b) install the historical way per the instructions for installing GT.M in the Installation Procedure section of Chapter 2 (Installing GT.M) in the GT.M Administration and Operations Guide.

We strongly recommend that you install YottaDB r1.10 in a newly created directory, different from those of YottaDB r1.00 and any GT.M versions you may have installed on the system.

Removing an installed YottaDB release

Assuming $gtm_dist points to the directory where YottaDB is installed:

• Cleanly shut down all application processes using that release.
• Execute mupip rundown && mupip rundown -relinkctl.
• Ensure that there are nogtcm* or gtmsecshr processes active.
• Use sudo lsof | grep $gtm_dist to ensure there are no open files.
• Delete the directory with sudo rm -rf $gtm_dist.

Upgrading to YottaDB r1.10

As YottaDB r1.10 is upward compatible from both YottaDB r1.00 and GT.M V6.3-002, the minimal upgrade steps are:

• Install YottaDB r1.10.
• Recompile any object code, and recreate shared libraries where appropriate.
• If you are using encryption, compile and install the reference implementation plugin or your customized plugin.
• Cleanly shut down the application and ensure that the database files are shut down using mupip rundown from the prior release.
• Switch journal files with the new YottaDB release.
• Start using the new YottaDB release.

To upgrade from older GT.M releases, first upgrade to GT.M V6.0-000 or later and follow the steps above, or contact your YottaDB support channel for assistance and guidance.

A more sophisticated upgrade technique is:

• Install YottaDB r1.10.
• Create a new replicated instance of your application (on the same system or a different system).
• Assuming the existing instance is A, and the new instance is B, start replicating from A to B.
• Once B catches up, switchover so that B is in a primary role replicating to A.
• Once you are satisfied with B, remove (or upgrade) A.

Change History

r1.10

YottaDB r1.10 includes the following changes from YottaDB r1.00.

Id	Category	Summary
#8	Other	gtmgblstat.xc in binary distribution package
#10	Language	Support reverse $query for globals and locals
#11	Database	Speed up $order(gvn,-1) where gvn is of the form ^xxx("")
#12	Database	Repeated calls to $order(^xxx("")) run faster
#19	Other	gtmprofile sets "YDB>" as default prompt
#25	Language	Issue MAXSTRLEN error if length of $query(lvn) exceeds maximum string length
#30	Operations	Source server behaves correctly (no SIG-11) if started with an external filter followed by a deactivate and shutdown
#32	Language	Improve efficiency of C to M calls
#44	Utilities	%HD works on hexadecimal numbers with a 0x or 0X prefix
#47	Database	OFRZAUTOREL message includes full path of database file
#61	Other	Port to Linux on ARM
#66	Other	Make it easier to install YottaDB from a source or binary tarball

In addition to the changes described above, YottaDB r1.10 includes changes released in GT.M V6.3-002 with one exception.

More Information

(Section blank for this release)

Messages

OFRZAUTOREL

This is an existing message with updated text. The Action remains unchanged.

OFRZAUTOREL, Online Freeze automatically released for database file aaaa

Operator log Warning: A process needed to modify the database file aaaa, which had an Online Freeze, but with AutoRelease selected. The process continued normally, modifying the file.

Action: Discard any database copy or snapshot made after the Online Freeze, as its contents are suspect. Perform a MUPIP FREEZE -OFF to clean up the prior Online Freeze. If the AutoRelease behavior is not desired, try again with MUPIP FREEZE -ON -ONLINE -NOAUTORELEASE. If the cause of the AutoRelease is unclear, report this and the accompanying ERRCALL message to your YottaDB support channel.

QUERY2

QUERY2, Invalid second argument to $QUERY. Must be -1 or 1

Run Time Error: This indicates that the second argument to a $QUERY function was not a 1 or -1, which are the only permitted values.

Action: Modify the application code to provide a correct value.

From GT.M

In addition to the error messages added by YottaDB, error messages from GT.M V6.3-002 are also present in YottaDB r1.10.

Tarball hashes

sha256sum	File
ed6b38f96c864a36da0b58e0f991a452e00ffb155c58f50beebaab41e05ef363	ydbinstall.sh
6f6f8eb36a307847e7becb6fd36b2281207352d33f921e16622c0933b7f648ec	yottadb_r110_linux_armv7l_pro.tgz
52e7160dce7fefb1ec91812012084e113c81e0af7e9c4baa9f560bc038ceecc3	yottadb_r110_linux_x8664_pro.tgz
3c4f5ec09f03de42c845c2d0800fb91249be92f8699a10793f230c4d341b7d02	yottadb_r110_rhel7_x8664_pro.tgz
2ee6d1c28b5bbb39dbfe2b52ad2fbef9327edcf462127ce31491d66ef06b7bf4	yottadb_r110_src.tgz

Legal Stuff

Copyright © 2017 YottaDB LLC

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts and no Back-Cover Texts.

YottaDB™ is a trademark of YottaDB LLC. GT.M™ is a trademark of Fidelity National Information Services, Inc. Other trademarks belong to their respective owners.

This document contains a description of YottaDB and the operating instructions pertaining to the various functions that comprise the software. This document does not contain any commitment of YottaDB LLC. YottaDB LLC believes the information in this publication is accurate as of its publication date; such information is subject to change without notice. YottaDB LLC is not responsible for any errors or defects.