Improve efficiency of C to M calls
Created by: estess
Final Release Note
Calling from C to M is more efficient. For an application that uses only M code, all changes for this issue are backward compatible. An application with C code that calls M code will see changes in behavior that we believe are warranted by the improved performance.
- ZSHOW "S" does not display any GTM$CI frame.
- The top level M routine called from C code reports $zlevel=1 and $stack=0, matching the execution of that routine via a mumps -run command. Previously, such a top level M routine reported $zlevel=2 and $stack=1
- ZSHOW "S" reports stack frames up to and including the top level M routine, with the text "(Call-In Level Entry)" for any C code called by M code and in turn calling back into M code. Previously, ZSHOW "S" only displayed stack frames back to the frame corresponding to that intermediate C code.
- Using auto-relink, replacing a routine already on the stack, but above intermediate C code, works correctly. Previously, this resulted in unpredictable behavior when returning to that routine, commonly process termination with a segmentation violation (SIG-11 SEGV) or illegal instruction (SIG-4 ILL).
- When a routine already on the stack, but above intermediate C code, is replaced using auto-relink, M profiling works correctly for that routine. Previously, M profiling reported erroneous information for such a routine.
- zhalt returns to the C caller of M code, returning its argument as the return value to the caller. Previously, zhalt terminated the process, returning its argument as the process status.
- $view("environment") returns a comma separated string with one to three pieces. The pieces are as follows
- The first piece is either "MUMPS" or "MUPIP". The only mupip process for which $view() is meaningful is a replication Update Process that invokes a trigger; all others report "MUMPS".
- A "TRIGGER" second piece indicates that the $view() is in M code being invoked via a trigger.
- A "CALLIN" second or third piece indicates that the $view() is in M code invoked by C code.
Description
The Call-Ins feature of YottaDB that allows other languages to call an M routine has some known inefficiencies in it that cause performance problems. This first initial step in getting better performing Call-Ins addresses some of those inefficiencies in the underlying architecture. The amount of performance improvement one can get depends on what is being called. If the called M routine does a lot of indirection forcing lots of compilation, the overall improvement will be minimal. This change does streamline and optimize the actual Call-In itself though and provides a solid base for further planned improvements in this general area.
Draft Release Note
This change restructures call-ins to provide some call-in overhead reduction. The benefit users can see is restricted to the actual call itself. If what is called are small M stub routines that perform a single action that involves indirects, the benefit seen will be quite small since the bulk of the time of such an operation is the compilation of the indirect references and not the call-in overhead. The change involved a significant rework in how call-ins work so there are some significant user visible changes.
Additionally, there were a few issues found in call-ins that were fixed.
Following are the user-visible effects of these internal changes and fixes:
- The GTM$CI frame no longer exists so won't show up in M stack listings anymore.
- Due to the above, the values of $ZLEVEL and $STACK show up differently than in previous versions. The values for $ZLEVEL and $STACK SVNs has changed such that now, in the routine that is the target of the calling, $ZLEVEL is set to 1 and $STACK is set to 0. Previously the first call-in level showed up as $ZLEVEL=2 and $STACK=1. These values less now matching the values of $ZLEVEL and $STACK when a routine is run with mumps -run.
- When dealing with an M stack where a Call-In is involved, the ZSHOW "S" command shows M stack frames all the way back to the first frame on the M stack. Previously, ZSHOW "S" only displayed frames back to the first Call-In base frame it encountered leaving previous stack frames undisplayed. The new stack display shows the text "(Call-In Level Entry)" where call-in base frames are located.
- When replacing a routine that is active on the stack, we run the stack backwards to verify the routine is not being used. You can still replace an active routine but it is a special case. Unfortunately the loop stopped at the first call-in frame and did not look further back. If a routine being replaced is on the stack further back than a call-in base-frame, it is not found resulting in the routine that is being returned to being released and replaced resulting in unpredictable behavior but likely to fail with a segmentation violation (SIG-11 SEGV) or illegal instruction (SIG-4 ILL) when the stack is unwound back to that earlier frame.
- In similar circulstances, M-Profiling returns the correct information for the routine hidden behind a call-in frame. The M-Profiling code looks up entries on the stack to get information. With previous versions, when the entry is not found, erroneous information is returned for that routine.
- [Z]HALT in a call-in does not halt but returns to the call-in caller as they should. Previously, [Z]HALT halted the process with no return to the caller.
- The $VIEW() function has a new option "ENVIRONMENT". This option returns from one to four tokens. Multiple tokens are separated by commas. The possible tokens are:
- MUMPS - the current process is a mumps process.
- MUPIP - the current process is a mupip process. The only mupip process capable of this currently is the update process so this is coupled with the next token as well.
- TRIGGER - the current M-stack has an active trigger so process is in "trigger-mode".
- CALLIN - the current M-stack has an active call-in so process is in "call-in mode".