Commit 7564cbd3 authored by Phil Hagelberg's avatar Phil Hagelberg

Check in a copy of the orb OS.

parent 40347447
local utils = require "utils"
local orb = require "orb"
local orb = require "os/orb"
local new = function(x, y, dx, dy, mass, image, name, description, star, os)
return { x = x, y = y, dx = dx, dy = dy,
......
This diff is collapsed.
# Orb OS
Orb is an operating system designed for embedding in
[a game](https://github.com/technomancy/calandria) in order to
facilitate learning programming and unix skills.
You can use the OS from the CLI outside the game too:
```
$ lua init.lua
```
However, when run this way it uses blocking input, which will prevent
the scheduler from running more than one process. (This means you
can't pipe from one process to another, as this requires at least some
level of faux-concurrency.) Note the filesystem is purely in-memory
in the Lua process and will not persist when run from the CLI, though
when running in-game it should be persisted in between server
restarts.
## Design
Upon boot, the scripts inside the `resources` directory will be copied
into the in-memory filesystem. Running the `reload` command will
refresh the inner filesystem from the real filesystem.
Most functions take a filesystem table and an environment table. The
environment table is like what you'd expect; it simply maps strings to
strings. The filesystem is a bit more complicated. It's a tree where
directories are just tables, regular files are just strings, and
special nodes are functions. Write to a special node by calling a
function with arguments, and read from it by calling it with no
arguments.
Further, there are two types of filesystems. Raw filesystems are
regular tables, but they are not used very much. Proxied filesystems
are tables wrapped with metatables that enforce read/write permissions
for a given user. In addition, proxied filesystems support lookup
using `fs["/path/to/file"]`, whereas this would fail with a raw
filesystem; it would need to use `fs.path.to.file` instead. Most
functions assume they have a proxied filesystem.
Group membership is implemented by placing a file in
`/etc/group/$GROUP` named after the user in question.
The shell is sandboxed and only has access to the whitelist in
`orb.process.sandbox`, which is currently rather small. Since the
environment is just a table, it can be modified at will by user
code. Sandbox functions which need to trust the `USER` environment
value must be wrapped in order to ensure it hasn't changed.
Spawning processes places entries in the `/proc/$USER/` table. The key
is the process id, and the value is a coroutine for that process. The
scheduler currently runs by looping over all the coroutines in the
`/proc` directory and resuming each of them.
## Executables
* [x] ls
* [x] cat
* [x] mkdir
* [x] env
* [x] cp
* [x] mv
* [x] rm
* [x] echo
* [x] smash (bash-like)
* [x] chmod
* [x] chown
* [x] chgrp
* [x] ps
* [x] grep
* [x] sudo
* [x] passwd
* [x] kill
* [ ] man
* [ ] mail
* [ ] ssh
* [ ] scp
* [ ] more
Other shell features
* [x] sandbox scripts (limited api access)
* [x] enforce access controls in the filesystem
* [x] input/output redirection
* [x] env var interpolation
* [x] user passwords
* [ ] pipes (half-implemented)
* [ ] globs
* [ ] quoting in shell args
* [ ] pre-emptive multitasking (see [this thread](https://forum.minetest.net/viewtopic.php?f=47&t=10185) for implementation ideas)
* [ ] /proc nodes for exposing connected digiline peripherals
* [ ] more of the built-in scripts should take multiple target arguments
## Differences from Unix
The OS is an attempt at being unix-like; however, it varies in several
ways. Some of them are due to conceptual simplification; some are in
order to have an easier time implementing it given the target
platform, and some are due to oversight/mistakes or unfinished features.
The biggest difference is that of permissions. In this system,
permissions only belong to directories, and files are simply subject
to the permissions of the directory containing them. In addition, the
[octal permissions](https://en.wikipedia.org/wiki/File_system_permissions#Notation_of_traditional_Unix_permissions)
of unix are collapsed into a single `group_write` bit. It's assumed
that the directory's owner always has full read-write access and that
members of the group always have read access. The `chown` and `chgrp`
commands work similarly as to unix, but `chmod` simply takes a `+` or
`-` argument to enable or disable group write. Group membership is
indicated simply by an entry in the `/etc/groups/$GROUP` directory
named after the username.
Rather than traditional stdio kept in `/dev/`, here we have `IN` and
`OUT` filenames kept in the environment, and `read` and `write`
default to using these. There is no stderr. Due to limitations
in the engine, there is no character-by-character IO; it is only full
strings (usually a whole line) at a time that are passed to `write` or
returned from `read`. The sandbox in which scripts run have `print`,
`io.write`, and `io.read` redefined to these functions; when a session
is initiated over the terminal it's up to the node definition to set
`IN` and `OUT` in the environment to functions which move the data
to and from the terminal's connection.
Of course, all scripts are written in Lua. Filesystem, the environment
table, and CLI args are exposed as `...`, so scripts typically start
with `local f, env, args = ...`. Filesystem access is simply table
access, though the table you're given is a proxy table that enforces
permissions with Lua metamethods. Regular files in the filesystem are
just strings in a table, and special nodes (like named pipes) are
functions.
Sudo takes the user to switch to as its first argument, and the
following arguments are taken as a command to run as the other
user. There is no password required; if you are in the `sudoers`
group, you can run sudo.
You can refer to environment variables in shell commands, but the
traditional Unix `$VAR` does not work; you must use the less-ambiguous
`${VAR}` instead.
## Portability
Currently when running outside of minetest it needs to shell out to
the `sha1sum` executable because Lua does not have any built-in
checksumming functionality. This could pose a problem when running on
platforms that lack this executable.
## License
Copyright © 2015 Phil Hagelberg and contributors. Licensed under the
GPLv3 or later; see the file COPYING.
This diff is collapsed.
-- a fake lil' OS
assert(setfenv, "Needs lua 5.1; sorry.")
orb = { dir = (minetest and minetest.get_modpath("orb")) or ... }
dofile(orb.dir .. "/utils.lua")
dofile(orb.dir .. "/fs.lua")
dofile(orb.dir .. "/shell.lua")
dofile(orb.dir .. "/process.lua")
-- for interactive use, but also as a sample of how the API works:
if(false) then
-- start with an empty filesystem
f_raw = orb.fs.new_raw()
f0 = orb.fs.seed(orb.fs.proxy(f_raw, "root", f_raw),
{technomancy = "hogarth",
buddyberg = "hello"})
orb.fs.add_user(f0, "zacherson", "robot")
f1 = orb.fs.proxy(f_raw, "technomancy", f_raw)
e0 = orb.shell.new_env("root")
e1 = orb.shell.new_env("technomancy")
-- Open an interactive shell
orb.shell.exec(f1, e1, "smash")
-- co = orb.process.spawn(f1, e1, "smash")
-- -- till we have non-blocking io.read, the scheduler isn't going to do
-- -- jack when run from regular stdin
-- while coroutine.status(co) ~= "dead" do orb.process.scheduler(f0) end
-- tests
t_groups = orb.shell.groups(f0, "technomancy")
assert(orb.utils.includes(t_groups, "technomancy"))
assert(orb.utils.includes(t_groups, "all"))
assert(not orb.utils.includes(t_groups, "zacherson"))
orb.shell.exec(f1, e1, "mkdir mydir")
orb.shell.exec(f1, e1, "mkdir /tmp/hi")
orb.shell.exec(f1, e1, "ls /tmp/hi")
orb.shell.exec(f1, e1, "/bin/ls > /tmp/mydir")
orb.shell.exec(f1, e0, "ls /etc > /tmp/ls-etc")
orb.shell.exec(f1, e1, "cat /bin/cat > /tmp/cat")
f1["/home/technomancy/bin"].bye = "print \"good bye\""
orb.shell.exec(f1, e1, "bye")
assert(orb.fs.readable(f0, f1["/home/technomancy"], "technomancy"))
assert(orb.fs.readable(f0, f1["/bin"], "technomancy"))
assert(orb.fs.readable(f0, f1["/bin"], "zacherson"))
assert(orb.fs.writeable(f0, f1["/home/technomancy"], "technomancy"))
assert(orb.fs.writeable(f0, f1["/tmp"], "technomancy"))
-- assert(not orb.fs.writeable(f0, f1["/etc"], "technomancy"))
-- assert(not orb.fs.writeable(f0, f1["/home/zacherson"], "technomancy"))
-- assert(not orb.fs.readable(f0, f1["/home/zacherson"], "technomancy"))
end
return orb
orb.process = {
-- Create a coroutine for a command to run inside and place it into the
-- process table. The process table is stored in the filesystem under
-- f.proc[user]
spawn = function(f, env, command, extra_sandbox)
local co = coroutine.create(function()
orb.shell.exec(f, env, "smash", extra_sandbox) end)
local id = orb.process.id_for(co)
f.proc[env.USER][id] = { thread = co,
command = command,
id = id,
_user = env.USER,
}
return co, id
end,
-- The process ID is taken from lua's own tostring called on a coroutine.
id_for = function(p)
return tostring(p):match(": 0x(.+)")
end,
-- Loop through all the coroutines in the process table and give them all
-- a chance to run till they yield. No attempt at fairness or time limits
-- yet.
scheduler = function(f)
for _,procs in pairs(f.proc) do
if(type(procs) == "table") then
for k,p in pairs(procs) do
if(type(p) == "table" and p.thread) then
if(coroutine.status(p.thread) == "dead") then
procs[k] = nil
else
coroutine.resume(p.thread)
end
end
end
end
end
end,
}
-- -*- lua -*-
local f, env, args = ...
local group = table.remove(args,1)
if(env.USER == "root" or orb.in_group(f, env.USER, group)) then
for _,user in ipairs(args) do
orb.add_to_group(f, user, group)
end
else
print("Not a member of " .. group)
end
-- -*- lua -*-
local f, env, args = ...
orb.add_user(f, args[1], args[2])
-- -*- lua -*-
local f, env, args = ...
for _,filename in pairs(args) do
print(orb.read(orb.normalize(filename, env.CWD)))
end
-- -*- lua -*-
local f, env, args = ...
local group = args[1]
local dirname = args[2] or env.CWD
local dir = f[orb.normalize(dirname, env.CWD)]
-- TODO: assert arg is an actual group
if(not dir) then
print("Not found: " .. dir)
elseif(not group) then
print("Usage: chgrp GROUP [DIR]")
else
dir._group = group
end
-- -*- lua -*-
local f, env, args = ...
local perms = args[1]
local dirname = args[2] or env.CWD
local dir = f[orb.normalize(dirname, env.CWD)]
if(not dir) then
print("Not found: " .. dir)
elseif(perms == "+") then
dir._group_write = "true"
elseif(perms == "-") then
table.remove(dir, "_group_write")
else
print("Usage: chmod +/- [DIR]")
end
-- -*- lua -*-
local f, env, args = ...
local user = args[1]
local dirname = args[2] or env.CWD
local dir = f[orb.normalize(dirname, env.CWD)]
-- TODO: assert arg is an actual user
if(not dir) then
print("Not found: " .. dir)
elseif(not user) then
print("Usage: chown USER [DIR]")
else
dir._user = user
end
-- -*- lua -*-
local f, env, args = ...
local to_dir, to_base = orb.dirname(orb.normalize(args[2], env.CWD))
f[to_dir][to_base] = f[orb.normalize(args[1], env.CWD)]
-- -*- lua -*-
local f, env, args = ...
if(args[1] == "-n") then
table.remove(args, 1)
io.write(table.concat(args, " "))
else
print(table.concat(args, " "))
end
-- -*- lua -*-
local f, env, args = ...
for k,v in pairs(env) do
if(type(v) == "string") then
io.write(k .. "=" ..v .. "\n")
end
end
-- -*- lua -*-
local f, env, args = ...
local lines = {}
local read_line = function()
if(#lines == 0) then lines = orb.utils.split(io.read(), "\n") end
return table.remove(lines, 1)
end
local line = read_line()
while line do
if(line:match(args[1])) then print(line) end
line = read_line()
end
-- -*- lua -*-
local f, env, args = ...
if(f.proc[env.USER][args[1]]) then
f.proc[env.USER][args[1]] = nil
else
print("Process " .. args[1] .. " not found".)
end
-- -*- lua -*-
local f, env, args = ...
local dirname = args[1] or env.CWD
local dir = f[orb.normalize(dirname, env.CWD)]
if(not dir) then
print("Not found: " .. dir)
elseif(type(dir) == "table") then
for name,contents in pairs(dir) do
if(not name:match("^_")) then
if(type(contents) == "table") then
print(name .. "/")
elseif(type(contents) == "function") then
print("*" .. name)
elseif(type(contents) == "thread") then
print("%" .. name)
else
print(name)
end
end
end
else
print(dirname)
end
-- -*- lua -*-
local f, env, args = ...
orb.mkdir(f, args[1], env)
-- -*- lua -*-
local f, env, args = ...
local dir, base = orb.dirname(orb.normalize(args[1], env.CWD))
local buffer = {}
local max_buffer_size = 1024
f[dir][base] = function(...)
local arg = {...}
if(#arg == 0) then
while #buffer == 0 do coroutine.yield() end
return table.remove(buffer, 1)
elseif(arg[1] == "*buffer") then
return buffer
else -- write
while(#buffer > max_buffer_size) do coroutine.yield() end
for _,output in pairs(arg) do
table.insert(buffer, output)
end
end
end
-- -*- lua -*-
local f, env, args = ...
orb.exec(f, env, "cp " .. args[1] .. " " .. args[2])
orb.exec(f, env, "rm " .. args[1])
-- -*- lua -*-
local f, env, args = ...
orb.change_password(f, env.USER, args[1], args[2])
-- -*- lua -*-
local f, env, args = ...
for k,v in pairs(f.proc[env.USER]) do
if(type(v) == "table") then
print(v.id .. " " .. coroutine.status(v.thread) .. ": " .. v.command)
end
end
-- -*- lua -*-
local f, env, args = ...
-- TODO: allow reloading just a directory
orb.reload()
-- -*- lua -*-
local f, env, args = ...
for _,filename in pairs(args) do
local dir, base = orb.dirname(orb.normalize(filename, env.CWD))
f[orb.normalize(dir, env.CWD)][base] = nil
end
-- -*- lua -*-
local f, env, args = ...
local env = orb.utils.shallow_copy(env)
local smashrc = f[env.HOME .. "/.smashrc"]
if(smashrc) then local f = loadstring(smashrc) assert(f) return f() end
while true do
io.write(orb.utils.interp(env.PROMPT, env))
local input = orb.utils.interp(io.read(), env)
if not input or input == "exit" or input == "logout" then return end
local var, value = input:match("export (.+)=(.*)")
local change_dir = input:match("cd +(.+)")
-- inlining primitives this way is kinda tacky
if(input == "cd") then
env.CWD = env.HOME
elseif(change_dir) then
env.CWD = orb.normalize(change_dir, env.CWD)
elseif(var) then
env[var] = value
elseif(not input:match("^ *$")) then
local success, msg = orb.pexec(f, env, input, orb.extra_sandbox)
if(not success) then
print(msg)
env.LAST_ERROR = msg
else
env.LAST_ERROR = nil
end
end
end
-- -*- lua -*-
local f, env, args = ...
if(orb.utils.includes(args, "--help")) then
print("Usage: sudo USER COMMAND [ARG]...")
print("You may only run sudo if you are a member of the sudoers group.")
if(orb.in_group(f, env.USER, "sudoers")) then
print("You are a member of sudoers.")
else
print("You are not a member of sudoers.")
end
else
local user = table.remove(args, 1)
orb.sudo(f, env, user, args, orb.extra_sandbox)
end
-- shell
orb.shell = {
new_env = function(user)
local home = "/home/" .. user
return { PATH = "/bin:" .. home .. "/bin", PROMPT = "${CWD} $ ",
SHELL = "/bin/smash", CWD = home, HOME = home, USER = user,
}
end,
-- This function does too much: it turns a command string into a tokenized
-- list of arguments, but it also searches the argument list for stdio
-- redirects and sets up the environment's read/write appropriately.
parse = function(f, env, command)
local tokens
if(type(command) == "string") then
tokens = orb.utils.split(command, " +")
elseif(not command) then
return nil, {}
else
tokens = command
end
local args = {}
local executable_name = table.remove(tokens, 1)
local t = table.remove(tokens, 1)
while t do
if(t == "<") then
env.IN = orb.fs.normalize(tokens[1], env.CWD)
break
elseif(t == ">") then
local target = table.remove(tokens, 1)
target = orb.fs.normalize(target, env.CWD)
local dir, base = orb.fs.dirname(target)
if(type(f[dir][base]) == "string") then f[dir][base] = "" end
env.OUT = target
break
elseif(t == ">>") then
local target = table.remove(tokens, 1)
env.OUT = orb.fs.normalize(target, env.CWD)
break
-- elseif(t == "|") then
-- -- TODO: support pipelines of arbitrary length
-- -- TODO: IN and OUT as buffer tables?
-- local env2 = orb.utils.shallow_copy(env)
-- local buffer = {}
-- env2.read = function()
-- while #buffer == 0 do coroutine.yield() end
-- return table.remove(buffer, 1)
-- end
-- env.write = function(output)
-- table.insert(buffer, output)
-- end
-- local co = orb.process.spawn(f, env, table.concat(tokens, " "))
-- break
else
table.insert(args, t)
end
t = table.remove(tokens, 1)
end
return executable_name, args
end,
-- Execute a command directly in the current coroutine. This is a low-level
-- call; usually you want orb.process.spawn which creates it as a proper
-- process.
exec = function(f, orig_env, command, extra_sandbox)
local env = orb.utils.shallow_copy(orig_env)
local executable_name, args = orb.shell.parse(f, env, command)
local try_run = function(executable_path)
if(type(f[executable_path]) == "string") then
local chunk = assert(loadstring(f[executable_path]))
local sandbox = orb.shell.sandbox(f, env, extra_sandbox)
-- getting the filesystem metatable would be a security leak
assert(not sandbox.getmetatable, "Sandbox leak")
setfenv(chunk, sandbox)
chunk(f, env, args)
return true
end
end
if(executable_name:match("^/")) then
if try_run(executable_name) then return end
else
for _, d in pairs(orb.utils.split(env.PATH, ":")) do
local path = orb.fs.normalize(d .."/".. executable_name, env.CWD)
if try_run(path) then return end
end
end
error(executable_name .. " not found.")
end,
-- Like exec, but protected in a pcall.
pexec = function(f, env, command, extra_sandbox)
return pcall(function() orb.shell.exec(f, env, command, extra_sandbox) end)
end,
-- Set up the sandbox in which code runs. Need to avoid exposing anything
-- that could allow security leaks.
sandbox = function(f, env, extra_sandbox)
local read = function() return orb.fs.read(f, env.IN) end
local write = function(...) return orb.fs.write(f, env.OUT, ...) end
-- env is just a table; it can be modified by any user script.
-- Therefore any function exposed in the sandbox which trusts env.USER
-- must be wrapped with this function which asserts that the USER
-- value has not been modified.
local lock_env_user = function(fn, env_arg_position)
return function(...)
local args = {...}
assert(args[env_arg_position].USER == env.USER, "Changed USER!")
return fn(...)
end
end
local box = { orb = { utils = orb.utils,
mkdir = orb.fs.mkdir,
dirname = orb.fs.dirname,
normalize = orb.fs.normalize,
add_user = orb.fs.add_user,
add_to_group = orb.fs.add_to_group,
in_group = orb.shell.in_group,
change_password = orb.shell.change_password,
sudo = lock_env_user(orb.shell.sudo, 2),
exec = lock_env_user(orb.shell.exec, 2),
pexec = lock_env_user(orb.shell.pexec, 2),
read = orb.utils.partial(orb.fs.read, f),
write = orb.utils.partial(orb.fs.write, f),
append = orb.fs.append,
reload = orb.fs.reloaders[f],
extra_sandbox = extra_sandbox, },
pairs = orb.utils.mtpairs,
ipairs = ipairs,
unpack = unpack,
print = function(...)
write(tostring(...)) write("\n") end,
coroutine = { yield = coroutine.yield,
status = coroutine.status },
io = { write = write, read = read },
tonumber = tonumber,
tostring = tostring,
math = math,
type = type,
table = { concat = table.concat,
remove = table.remove,
insert = table.insert,
},
}
for k,v in pairs(extra_sandbox or {}) do box[k] = v end
return box
end,
groups = function(f, user)
local dir = f["/etc/groups"]
local found = {}
for group,members in orb.utils.mtpairs(dir) do
if(type(members) == "table" and orb.utils.includes(members, user)) then
table.insert(found, group)
end
end
return found
end,
in_group = function(f, user, group)
local group_dir = f.etc.groups[group]
return group_dir and group_dir[user]
end,
auth = function(f, user, password)
local raw_fs = (getmetatable(f) and getmetatable(f).raw_root) or f