Import: Adjustments for the import API
I've been spending the last week and change working on what I want the new import api to look like. I already have a few issues listed that mention import adjustments. The big one is #393 (closed), a call to simplify the interpreter's import hook. The import hook clearly needs to be adjusted. The lingering question has been "how".
Here are the following cases I want to support:
-
The stock interpreter. The stock interpreter searches native and foreign files. It searches the current package and for a package in a packages directory.
-
The sandbox. The sandbox doesn't have libraries available so dlopen/dlsym shouldn't be run. However, it's useful to allow importing
sys
and possibly spoof native modules as well. -
The testing system. I'd like to build a testing system wherein files in a
test
directory can import files from thesrc
directory as if they were in thesrc
directory. That involves some adjustment of where local loading and the package directory are. -
Loading a module and then being interactive. This is a quality of life adjustment that allows people to not worry about having to use scope qualifiers or worry that a symbol isn't exported. Getting this to work for a native preload is easy, but a foreign preload would be more difficult.
-
Creating mock files to help the interpreter's testing suite. In a similar vein, redirecting certain library imports to allow for creating one large testing directory (mentioned by #394 (closed)).
Another topic I want to address is how there are two parse and two render functions. One for parse/render of a file, and one for parse/render of a string. I'd like the import rework to also include having functions to load initial file or string content. This will allow me to simplify the parsing and rendering functions so that they're easier to test.
That's what I want. Here's the plan:
The current lily_load_*
functions will become lily_import_*
functions.
lily_load_file
and lily_load_string
will be created and work on the first (main) module. Whereas now, initialization is a matter of calling lily_parse_file(s, path);
, it will become lily_load_file(s, path); lily_parse(s);
These naming prefixes make the intent clear. Old code that uses lily_load_*
functions as-is will be migrated over and relevant packages bumped.
The renamed importing functions will automatically tack on the .lily
suffix or the .so/.dll/etc
suffix. The goal is to allow embedders to call import functions without needing to use a msgbuf to build path names that have suffixes.
Two directory selection apis will be provided. lily_import_use_local_dir
and lily_import_use_package_dir
. Both will take a prefix to be tacked onto any imports. So, for example, if the testing system wants to allow importing from the source directory, it can use lily_import_use_local_dir(s, "../src/")
. The importing functions will only use one directory at a time.
As an example, the default loading scheme would look similar to:
void lily_import_handler(lily_state *s, const char *target)
{
lily_import_use_local_dir(s, "");
if (lily_import_file(s, target) ||
lily_import_library(s, target))
return;
/* Note: Uses './packages/target/src/target.{suffix}' */
lily_import_use_package_dir(s, "packages");
if (lily_import_file(s, target) ||
lily_import_library(s, target))
return;
}
Speaking of the import handler, it seems reasonable to it to receive only the target that was specified by the import keyword.
Api functions should accept Linux slashes (/
) regardless of platform, transforming them to Windows slashes if necessary.
A function will be added to fetch the path that would be constructed for a given target (lily_import_path_for_target
). That information can be used by overriders to check that they're overriding the correct target.
Since this is going to be a big breaking change, I'm going to do this as one big breaking change instead of several smaller annoying breaks.