Commit f72be345 authored by bzt's avatar bzt

tty stuff

parent 4ece8236
--- Error fixes ---
./src/drivers/fs/fsz/main.c:196: // FIXME: block end reached, load next block from sdblk and alter dirent pointer
./src/fs/main.c:340: //FIXME: no sprintf yet
./src/fs/main.c:343: //FIXME: no sprintf yet
./src/fs/taskctx.c:387: // FIXME: read from TTY
./src/core/msg.c:249: /* FIXME: naive implementation, no checks */
./src/ui/compositor.c:97: // FIXME: use compositor pixbuf or focused window's pixbuf if it's fullscreen
./src/core/msg.c:279: /* FIXME: naive implementation, no checks */
./src/libui/pixbuf.c:149: // FIXME: naive implementation, optimize pixbuf offset
--- Features ---
./docs/howto4-rescueshell.md:26:TODO: write howto
./docs/howto4-rescueshell.md:27:TODO: write howto
./src/init/services.c:36: // TODO: load and start user services
./src/drivers/display/fb/main.c:52: // TODO: if second monitor found, register display ":1" too
./src/drivers/display/fb/main.c:113: // TODO: if second monitor found, register display "1" too
./src/drivers/fs/fsz/main.c:142: // TODO: extents
./src/drivers/fs/fsz/main.c:235: // TODO: if loc->creat, then create dir or path
./src/fs/vfs.c:575: // TODO: handle up directory
./src/fs/vfs.c:578: // TODO: detect file system in image
./src/ui/display.c:72: // TODO: if display not found by this name, create a remote display instance if possible
./src/ui/compositor.c:50: // TODO: tell core to swap mapping of pixbuf and pixbuf2
./src/ui/display.c:79: // TODO: if display not found by this name, create a remote display instance
./src/ui/win.c:292: // TODO: send the data packet to the remote terminal server
./src/core/x86_64/disasm.h:1216: //TODO: esc
./src/core/x86_64/libk.S:157:// TODO: use SSE2 copy
./src/core/x86_64/ibmpc/platform.c:118:// TODO: enumerate bus and load drivers
./src/core/msg.c:100: /* TODO: use mq_buffstart properly as a circular buffer */
./src/core/msg.c:123: // TODO: use vmm_map()
./src/core/msg.c:200: // TODO: save memory dump to fs
./src/core/msg.c:112: /* TODO: use mq_buffstart properly as a circular buffer */
./src/core/msg.c:135: // TODO: use vmm_map()
./src/core/msg.c:234: // TODO: save memory dump to fs
./src/core/sched.c:164: /* TODO: memory -> swap (except tcb) */
./src/core/sched.c:180: /* TODO: swap -> memory */
./src/core/elf.c:76: // TODO: this should be able to use FS task too to load elfs
./src/lib/libc/aarch64/stdlib.S:213: /* TODO: getentropy */
./src/lib/libc/x86_64/stdlib.S:354: /* TODO: getentropy */
./src/core/elf.c:77: // TODO: this should be able to use FS task too to load elfs
./src/libui/main.c:126:// TODO: high level api for pixbuf primitives
./src/libui/pixbuf.c:50: // TODO: check if PSF do have unicode translation table? For now I'm sure it doesn't.
./src/libc/aarch64/stdlib.S:213: /* TODO: getentropy */
./src/libc/x86_64/stdlib.S:351: /* TODO: getentropy */
......@@ -22,22 +22,22 @@ Finally jumps to the C startup code `main()` in [src/core/main.c](https://gitlab
To boot up the computer, `core` does the following:
1. as OS/Z is polite, greets you with a "OS/Z starting..." message, then
2. `env_init` parses the [environment variables](https://gitlab.com/bztsrc/osz/blob/master/docs/bootopts.md) in [src/core/env.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/env.c) passed by the loader.
3. `pmm_init()` sets up Physical Memory Manager in [src/core/pmm.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/pmm.c).
4. `sys_init()` initializes [system services](https://gitlab.com/bztsrc/osz/blob/master/docs/services.md), in [src/core/sys.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/sys.c), which loads `IDLE` task and enumerates system buses to locate and load [device drivers](https://gitlab.com/bztsrc/osz/blob/master/docs/drivers.md). It will also initialize the [IRQ Routing Table](https://gitlab.com/bztsrc/osz/blob/master/src/core/x86_64/isr.c#L117) (IRT).
5. second service is the file system service, `service_init(SRV_FS)` in [src/core/service.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/service.c) which is a normal system service, except it has the initrd entirely mapped in in it's bss.
6. user interface is initialized with `service_init(SRV_UI)` in [src/core/service.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/service.c). These first three services (aka `IDLE`, `FS`, `UI`) are mandatory, unlike the rest.
7. loads additional, non-critical tasks by several `service_init()` calls in [src/core/service.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/service.c) like the `syslog`, `inet`, `sound` and `init` system services.
8. drops supervisor privileges by calling `sys_enable()` in [src/core/sys.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/sys.c).
9. that function switches to the FS task, which arranges memory so that it can receive mknod calls from drivers and blocks.
10. then the scheduler, `sched_pick` in [src/core/sched.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/sched.c) chooses driver and service tasks one by one until all of them blocks. Note that pre-emption is not enabled at this point. Driver tasks perform hardware initialization and they fill up the IRT.
11. when the `IDLE` task first scheduled, it will call `sys_ready()` in [src/core/sys.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/sys.c), which
1. as OS/Z is polite, greets you with a "OS/Z starting..." message, then
2. `env_init` parses the [environment variables](https://gitlab.com/bztsrc/osz/blob/master/docs/bootopts.md) in [src/core/env.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/env.c) passed by the loader.
3. `pmm_init()` sets up Physical Memory Manager in [src/core/pmm.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/pmm.c).
4. `sys_init()` in [src/core/sys.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/sys.c), creates `IDLE` task (the CPU "driver") and enumerates system buses to locate and load [device drivers](https://gitlab.com/bztsrc/osz/blob/master/docs/drivers.md). It will also initialize the [IRQ Routing Table](https://gitlab.com/bztsrc/osz/blob/master/src/core/x86_64/isr.c#L117) (IRT).
5. next it loads a [system service](https://gitlab.com/bztsrc/osz/blob/master/docs/services.md), `sys_service(SRV_FS)` which is a normal system service, except it has the initrd entirely mapped in in it's bss.
6. user interface is initialized with `sys_service(SRV_UI)`. These first three services (aka `IDLE`, `FS`, `UI`) are mandatory, unlike the rest (hence the upper-case).
7. loads additional, non-critical tasks by several `sys_service()` calls, like the `syslog`, `inet`, `sound` and `init` system services.
8. drops supervisor privileges by calling `sys_enable()`.
9. that function switches to the FS task, which arranges memory so that it can receive mknod calls from drivers and blocks.
10. then the scheduler, `sched_pick()` in [src/core/sched.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/sched.c) chooses driver and service tasks one by one. Note that pre-emption is not enabled at this point. Driver tasks perform hardware initialization and they fill up the IRT.
11. when all tasks are blocked and the `IDLE` task first scheduled, a call to `sys_ready()` in [src/core/sys.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/sys.c) will be made, which
12. enables IRQs with entries in IRT. It also enables timer IRQ and with that pre-emption may begin.
13. an "OS/Z ready" message is shown. At this point the `core` has done with booting.
14. now that all storage device drivers are finished with their initialization, sys_ready finally sends a SYS_mountfs message to the `FS` task.
15. the control is passed to ["FS" task](https://gitlab.com/bztsrc/osz/blob/master/src/fs/main.c), which in turn parses [fstab](https://gitlab.com/bztsrc/osz/blob/master/etc/sys/etc/fstab). When mounting finished, FS notifies all the other system services (by sending them a message they will no longer be blocked) and normal operation begins.
16. as soon as it's scheduled, one of the system services, the `init` task will load and start user services.
13. optionally the early syslog is dumped at the boot console. At this point the `core` has done with booting.
14. now that all storage device drivers are finished with their initialization, as a last thing `sys_ready()` sends a SYS_mountfs message to the `FS` task.
15. the control is passed to [FS task](https://gitlab.com/bztsrc/osz/blob/master/src/fs/main.c), which in turn parses [fstab](https://gitlab.com/bztsrc/osz/blob/master/etc/sys/etc/fstab). When mounting finished, `FS` notifies all the other system services and normal operation begins.
16. as soon as it's scheduled, one of the system services, the `init` task will load and start user session services.
If rescueshell was requested in [environment](https://gitlab.com/bztsrc/osz/blob/master/etc/sys/config), then [bin/sh](https://gitlab.com/bztsrc/osz/blob/master/src/sh/main.c) is loaded instead of `init`.
......@@ -54,16 +54,17 @@ Finally `init` starts all user services. User services are classic UNIX daemons,
User session
------------
As OS/Z is a multi-user operating system, it needs to authenticate the user. It is done by the "logind" daemon. When
As OS/Z is a multi-user operating system, it needs to authenticate the user. It is done by the `logind` daemon. When
the identity of the user is known, a session can be started by executing a user specific script. As soon as that script
ends the session is over, and execution is returned to "logind", which will display the user login screen again.
ends the session is over, and execution is returned to `logind`, which will display the user login screen again.
This differs from POSIX operating systems, which usually have several ttys, one of which is the graphical session manager
(typically tty8). In OS/Z there's a graphical session first, and you can start Terminal to get a tty (called pseudo-tty in UNIX terminology).
This differs from POSIX operating systems, which usually have several ttys (provided by getty), one of which is the graphical
session (typically tty8, provided by X). In OS/Z there's a graphical session first (provided by the `UI` task), and you can start
a shell to get a tty (called pseudo-tty in UNIX terminology, provided by terminal emulators).
End game
--------
When the `init` system service quits, the execution is [passed back](https://gitlab.com/bztsrc/osz/blob/master/src/core/msg.c) to `core`.
Then OS/Z says 'Good Bye' with `kprintf_reboot()` or with `kprintf_poweroff()` (depending on the exit status), and the platform
Then OS/Z says "Good Bye" with `kprintf_reboot()` or with `kprintf_poweroff()` (depending on the exit status), and the platform
dependent part restarts or turns off the computer.
OS/Z Boot Options
=================
Environment configuration file
Compilation Time Options
------------------------
A few options (mostly for performance reasons) moved to [Config](https://gitlab.com/bztsrc/osz/blob/master/Config). Changing these
will require a recompilation. But most of the features can be configured in boot-time.
Environment Configuration File
------------------------------
The boot options are kept on the first bootable partition on the first bootable disk under `FS0:\BOOTBOOT\CONFIG` or `/sys/config`. When you're
......@@ -23,9 +29,9 @@ Boot Parameters
| debug | 0 | decimal | core | specifies which debug information to show (if [compiled with debugging](https://gitlab.com/bztsrc/osz/blob/master/Config), see below) |
| nrphymax | 2 | number | core | the number of pages to store physical RAM fragments (16 bytes each) |
| nrmqmax | 1 | number | core | the number of pages for Message Queue (64 bytes each) |
| nrlogmax | 8 | number | core | the number of pages for early syslog buffer |
| nrlogmax | 2 | number | core | the number of pages for early syslog buffer |
| quantum | 100 | number | core | scheduler frequency, a task can allocate CPU continously for (quantum) timer interrupts. |
| display | 0[,drv] | number | core | selects output mode, and optinally driver, see below |
| display | 1 | number[,drv] | core | selects visual, and optinally the driver, see below |
| syslog | true | boolean | core | disable syslog [service](https://gitlab.com/bztsrc/osz/blob/master/docs/services.md) |
| networking | true | boolean | core | disable inet service |
| sound | true | boolean | core | disable sound service |
......@@ -33,9 +39,9 @@ Boot Parameters
| pathmax | 512 | number | fs | Maximum length of path, minimum 256 |
| cachelines | 16 | number | fs | Number of block cache lines, minimum 16 |
| cachelimit | 5 | percentage | fs | Flush and free up block cache if free RAM drops below this limit, 1%-50% |
| fps | 10 | number | ui | requested frame rate |
| keymap | en_us | string | ui | keyboard layout, see [etc/kbd](https://gitlab.com/bztsrc/osz/blob/master/etc/sys/etc/kbd) |
| focusnew | true | boolean | ui | automatically focus new windows |
| dblbuff | true | boolean | ui | use double buffering for window composition (memory consuming but fast) |
| lefthanded | false | boolean | ui | swap pointers |
| identity | false | boolean | init | run first time setup before servies to get machine's identity |
| clock | 0 | number | platform | override autodetected clock source |
......@@ -76,19 +82,21 @@ Most of these only available when compiled with [DEBUG = 1](https://gitlab.com/b
Display
-------
A numeric value or exactly one flag, and optionally a driver name after a comma.
A numeric value or exactly one flag, which sets the pixel buffer type (visual), and optionally a driver name after a comma.
| Value | Flag | Define | Description |
| ----: | ---- | ------ | ----------- |
| 0 | mc | DSP_MONO_COLOR | a simple 2D pixelbuffer with 32xRGB0 color pixels |
| 1 | sm,an | DSP_STEREO_MONO | two 2D pixelbuffers*, they are converted grayscale and a red-cyan filtering applied, anaglyph |
| 2 | sc,re | DSP_STEREO_COLOR | two 2D pixelbuffers*, the way of combining left and right eye's view is 100% driver specific, real 3D |
| 0 | mm | PBT_MONO_MONO | a simple 2D pixelbuffer with monochrome pixels |
| 1 | mc | PBT_MONO_COLOR | a simple 2D pixelbuffer with 32xRGB0 color pixels |
| 2 | sm,an | PBT_STEREO_MONO | two 2D pixelbuffers*, they are converted grayscale and a red-cyan filtering applied, anaglyph |
| 3 | sc,re | PBT_STEREO_COLOR | two 2D pixelbuffers*, the way of combining left and right eye's view is 100% driver specific, real 3D |
(* the two buffers are concatenated in a one big double heighted buffer)
Device drivers can be found under "sys/drv/display/", example: `display=mc,nvidia` will look for "sys/drv/display/nvidia.so". If
not given, then the first autodetected display driver will be used. If none was found during boot, then fallbacks to a CPU driven
linear framebuffer driver.
linear framebuffer driver, which is slow, but works. Alternatively you can use a remote driver to force all windows to be sent to
another machine with "remote:(ip)", like `display=mc,remote:192.168.1.1`.
OS/Z supports several displays, the one loaded with the UI task is ":0", which is the first local monitor.
......@@ -107,3 +115,21 @@ Either a numeric value or exactly one flag.
| 1 | ar | aarch64-rpi | Built-in ARM CPU timer (default) |
| 2 | sy | aarch64-rpi | BCM2837 System Timer |
Low Memory Footprint
--------------------
If you want to have an OS/Z for an embedded system where memory is expensive, you can do the following (considering UI task requires the most memory by far):
1. Use ASCII 8x8 system font instead of unicode 8x16 (will shrink core, ui and libui size about 30k each, see [etc/system.8x8.psf](https://gitlab.com/bztsrc/osz/tree/master/etc/)).
2. Turn off double buffering with `dblbuff=false`. This will cut the memory consumption of the compositor in half, but may produce visual artifacts.
3. Use monochrome visual, which is a bit slower, but requies only 8 bits per pixel instead of 32, cutting the memory requirement even further in quater.
4. In extreme cases, replace `logind` with a custom application for the job to avoid unnecessary processes and windows.
5. Alternatively use a remote display driver, so that no pixel buffer will be allocated on the embedded system at all.
Performance Tunning
-------------------
If you aim for a fast system, and you have plenty of RAM and SSSE3 (x86_64) or Neon (AArch64) capable processor, do:
1. Use `OPTIMIZE = 1` in [Config](https://gitlab.com/bztsrc/osz/blob/master/Config). This will utilize SIMD whenever possible (requires recompilation).
2. Use double buffering with true color visual, on a card that natively uses ARGB packed pixel format (and not ABGR). In this case a much faster blit implementation is used (and optimized even further with point 1).
3. Use high precision timer. Delays smaller than the timer interrupt interval are implemented by a userspace side busy loop instead of blocking the task.
......@@ -6,41 +6,36 @@ Preface
In this episode we'll take a look on how to test a live image.
I always push the source to git in a state that's known to [compile](https://gitlab.com/bztsrc/osz/tree/master/docs/compile.md) without errors by a `make clean all` command.
I always push the source to git in a state that's known to [compile](https://gitlab.com/bztsrc/osz/bloc/master/docs/compile.md) without errors by a `make clean all` command.
Although I did my best, it doesn't mean it won't crash under unforseen circumstances :-)
The [latest live dd image](https://gitlab.com/bztsrc/osz/blob/master/bin/) should boot OS/Z in emulators and on real machines. For example type
The [latest live dd image](https://gitlab.com/bztsrc/osz/tree/master/bin/) should boot OS/Z in emulators and on real machines. For example type
```shell
qemu-system-x86_64 -hda bin/osZ-latest-x86_64-ibmpc.dd
```
But for your convience I've added make commands. For example if you clone the repo and [compile](https://gitlab.com/bztsrc/osz/blob/master/docs/compile.md), you can boot OS/Z right from your `bin/ESP` directory
with TianoCore EFI. For that one should type
But for your convience I've added make commands.
```shell
make testesp
```
To run the resulting image in qemu with TianoCore UEFI
To run the dd image in qemu with TianoCore UEFI
```shell
make teste
```
To run the resulting image in qemu with BIOS
To run the dd image in qemu with BIOS
```shell
make testq
```
To run the result with BIOS in bochs
To run the dd image with BIOS in bochs
```shell
make testb
```
To run in VirtualBox (either BIOS or UEFI depending on your configuration):
To run in VirtualBox (either BIOS or UEFI, depending on your configuration):
```shell
make testv
......@@ -49,8 +44,7 @@ make testv
First Break Point - Booting starts
----------------------------------
For this How To we are going to use bochs, as it would stop at predefined
addresses.
For this How-To we are going to use bochs, as it will stop at predefined addresses.
```sh
make testb
......@@ -63,35 +57,34 @@ Next at t=0
<bochs:1> c
```
Just continue as we are not interested in the BIOS.
Just continue as we are not interested in debugging the BIOS.
Second Break Point - OS/Z boot ends
-----------------------------------
Second Break Point - System starting
------------------------------------
<img align="left" style="margin-right:10px;" width="300" src="https://gitlab.com/bztsrc/osz/raw/master/docs/oszdbg0.png" alt="OS/Z Starting">
<img align="left" style="margin-right:10px;" width="300" src="https://gitlab.com/bztsrc/osz/raw/master/docs/oszdbg1.png" alt="OS/Z Starting">
The first intersting point is where the operating system was loaded (arranged it's memory, finished with task setup)
and is about to leave privileged mode by executing the very first `iretq`.
The first interesting point is where the operating system is loaded (has arranged it's memory, finished with task setup)
and is about to leave privileged mode.
You must see white on black "OS/Z starting..." text on the top left corner of your screen,
and something similar to this on debug console:
and something similar to this on bochs debug console:
```
(0) Magic breakpoint
Next at t=45593694
(0) [0x00000019b263] 0008:ffffffffffe15263 (sys_enable+be): iret ; 48cf
<bochs:2>
Next at t=55851587
(0) [0x0000001fadcc] 0008:ffffffffffe08dcc (sys_enable+95): iret ; 48cf
<bochs:2>
```
This `sys_enable()` function is in [src/core/sys.c](https://gitlab.com/bztsrc/osz/blob/master/src/core/sys.c) and it's
job is to map the first task and start executing it by faking a return from ISR.
As the stack stores user mode segment selectors, the CPU will drop
privileged mode, and normal user space (ring 3) execution will began.
As the stack stores user mode segment selectors, the CPU will drop privileged mode, and normal user space (ring 3) execution will began.
```
<bochs:2> print-stack
Stack address size 8
| STACK 0x0000000000000fd8 [0x00000000:0x00200000]
| STACK 0x0000000000000fd8 [0x00000000:0x002000f0]
| STACK 0x0000000000000fe0 [0x00000000:0x00000023]
| STACK 0x0000000000000fe8 [0x00000000:0x00003002]
| STACK 0x0000000000000ff0 [0x00000000:0x001fffa0]
......@@ -99,29 +92,31 @@ Stack address size 8
<bochs:3> s
```
We can see that the user mode code starts at 0x200000, and it's stack is right beneath it (which differs from IRQ handler's safe stack).
We can see that the user mode code starts at 0x2000f0, and it's stack is right beneath it (which differs from ISR's safe stack at 0xfd8).
Third Break Point - OS/Z ready
------------------------------
<img align="left" style="margin-right:10px;" width="300" src="https://gitlab.com/bztsrc/osz/raw/master/docs/oszdbg1.png" alt="OS/Z ready">
Third Break Point - System ready
--------------------------------
When all drivers and system tasks initialized and blocked waiting for messages, a function named `sys_ready()` is called (also in src/core/sys.c).
That will in turn call the platform dependent `isr_fini()` which will finish ISR initialization by enabling IRQs with registered tasks.
This finishes up initialization by enabling IRQs with registered tasks, and sends a message to the FS task to mount file systems.
If you have configured `debug=log` in boot environment, now the syslog is printed on screen, so that you can see all the messages.
```
(0) Magic breakpoint
Next at t=45857039
(0) [0x00000019b616] 0008:ffffffffffe15616 (sys_ready+131): mov eax, 0x00000000 ; b800000000
Next at t=57417249
(0) [0x0000001faf92] 0008:ffffffffffe08f92 (sys_ready+1bc): sub rsp, 0x0000000000000008 ; 4883ec08
<bochs:3>
```
By doing so, it will enable pre-emption and unleash hell, as nobody will know for sure in which order the interrupts fire.
Luckily the message queue is serialized, so there's no need for locking.
By enabling the timer IRQ among others, core enables pre-emption and unleashes hell, as nobody knows for sure in which order
the interrupts will fire. Luckily the message queue is serialized, so there's no need for locking.
<img align="left" style="margin-right:10px;" width="80" src="https://gitlab.com/bztsrc/osz/raw/master/docs/oszdbg2.png" alt="OS/Z Debugger">
Please note that OS/Z has it's own internal debugger, so don't have to use bochs debugger if you don't want to.
Please note that bochs is not the only way to debug. OS/Z has it's own internal debugger, which works on real hardware too.
Here's a screenshot as a teaser. It has a nice interface and available on serial line too (VT100 mode supported).
If you are interested in debugging, read the [next tutorial](https://gitlab.com/bztsrc/osz/blob/master/docs/howto2-debug.md).
To get more info on the whole boot up procedure, and how `sys_enable()` and `sys_ready()` fit into the big picture,
read [OS/Z booting](https://gitlab.com/bztsrc/osz/blob/master/docs/boot.md).
......@@ -34,7 +34,7 @@ to start [GDB](https://www.sourceware.org/gdb/) and connect it to the OS/Z insta
### Get pid
Not possible, at least there's no way I know of.
Not possible, at least there's no way I know of. But you can dump qword in memory at 640.
Debugging with bochs
--------------------
......
......@@ -53,8 +53,8 @@ int keypress(int keycode, int scancode)
int main(int argc, char **argv)
{
/* initialization */
ui_win_t *win;
win = ui_openwindow( 640, 480, 0 );
win_t *win;
win = ui_openwindow( "pixbuf", 10, 10, 640, 480, 0 );
ui_setcursor(UIC_progress);
load_a_lot_of_data();
......@@ -65,12 +65,16 @@ int main(int argc, char **argv)
}
```
The basic UI functionality (like opening a window) is provided by the `libc`. This will give you a raw pixel buffer to draw to, but
nothing more. Functions that deal with the pixel buffer (drawing primitives, buttons, widgets, loading images etc.) are implemented
in `libui`.
Device drivers
--------------
Drivers are shared libraries which use a common [event dispatcher](https://gitlab.com/bztsrc/osz/blob/master/src/lib/libc/dispatch.c) mechanism.
Drivers are shared libraries, and hardware device drivers use a common [event dispatcher](https://gitlab.com/bztsrc/osz/blob/master/src/lib/libc/dispatch.c) mechanism.
Drivers are [categorized](https://gitlab.com/bztsrc/osz/blob/master/src/drivers/README.md), each in it's own sub-directory.
Also they use some privileged `libc` functions, and provide messaging interface therefore a minimal driver looks like
Also they can use some privileged `libc` functions, and a provide messaging interface. Therefore a minimal driver looks like
```c
#include <osZ.h>
......@@ -116,9 +120,10 @@ functions will return the appropriate value for the key, or `default` if key not
hexadecimal (with the '0x' prefix). Boolean understands 0,1,true,false,enabled,disabled,on,off. The `env_str` returns a
malloc'd string, which must be freed later by the caller.
Note that you don't have to do any message receive calls, just create public functions for your message types.
Note that you don't have to do any message receive calls, that's taken care for you by the event dispatcher. Just create public
functions for your message types.
Additionaly to Makefile, the build system looks for [two more files](https://gitlab.com/bztsrc/osz/blob/master/docs/drivers.md#files) along with a device driver source.
Additionaly to Makefile, the build system looks for [two more files](https://gitlab.com/bztsrc/osz/blob/master/docs/drivers.md#files):
* `platforms` - if this file exists, it can limit the compilation of the driver for specific platform(s)
* `devices` - specifies which devices are supported by the driver, used by `core` during system bus enumeration.
......@@ -132,15 +137,17 @@ They start at label `_start` defined in [lib/libc/(platform)/crt0.S](https://git
### Shared libraries
Similarly to executables, but the entry point is called `_init`. If not defined in the library, fallbacks to [lib/libc/(platform)/init.S](https://gitlab.com/bztsrc/osz/blob/master/src/lib/libc/x86_64/init.S).
When an executable loads several shared libraries, their `_init` will be called one by one before `_start` gets called.
Similarly to executables, but the entry point is called `_init`. This function should initialize the library, and return immedately.
When an executable loads several shared libraries, their `_init` will be called one by one before the main executable's `_start` gets
called. Therefore if a library `_init` hangs (for example blocks waiting for a system call), the execution of the main function is
postponed.
### Services and drivers
Service's entry point is also called `_init`, which function must call either `mq_recv()` or `mq_dispatch()`. If not defined otherwise,
fallbacks to the default in [lib/libc/service.c](https://gitlab.com/bztsrc/osz/blob/master/src/lib/libc/service.c) (also
used by device drivers). In that case the service must implement `task_init()` to initialize it's structures. Services can use
shared libraries, in which case the libraries' `_init` will be called one by one before the service's `_init` function.
Service's entry point is also called `_init`, which function must call either `mq_recv()` or `mq_dispatch()`. If not defined
otherwise, fallbacks to the default in [lib/libc/service.c](https://gitlab.com/bztsrc/osz/blob/master/src/lib/libc/service.c)
(also used by device drivers). In that case the service must implement `task_init()` to initialize it's structures. Services
can use shared libraries, in which case the libraries' `_init` will be called one by one before the service's `_init` function.
Application packages
--------------------
......
......@@ -11,8 +11,9 @@ In this episode we'll take a system administator's approach, and we'll see how t
Activating Rescue Shell
-----------------------
You'll have to enable rescue shell in [environment](https://gitlab.com/bztsrc/osz/blob/master/etc/sys/config) by setting `rescueshell=true`. When you boot, `init` won't initialize user services, rather it will drop you
in a root shell.
You'll have to enable rescue shell in [environment](https://gitlab.com/bztsrc/osz/blob/master/etc/sys/config) by setting `rescueshell=true`.
Next time you boot, `sh` will be loaded instead of `init`, meaning OS/Z won't initialize user services, rather it will drop you in
a root shell.
Available Commands
------------------
......
......@@ -13,13 +13,14 @@ System Install
### From Development Environment
It's quite easy to install it on a hard drive or removable media, like an USB stick or SD card.
Just [download live image](https://gitlab.com/bztsrc/osz/blob/master/bin/) and use
Just [download live image](https://gitlab.com/bztsrc/osz/blob/master/bin/) and under Linux use
```sh
dd if=bin/osZ-latest-x86_64-ibmpc.dd of=/dev/sdc
```
Where `/dev/sdc` is the device where you want to install.
Where `/dev/sdc` is the device where you want to install (use `dmesg` to figure out). On MacOSX you can list the available devices
with `diskutil list`.
### Inside OS/Z
......@@ -38,8 +39,8 @@ You can also individually install, upgrade and remove applications.
### Refresh package list
The list of repositories from where packages can be installed are stored in `/sys/etc/repos`.
The meta information for all available packages from all repositories are stored in `/sys/packages`.
The list of repositories from where packages can be installed are stored in `/etc/repos`.
The meta information for all available packages from all repositories are stored in `/sys/var/packages`.
To refresh that list, issue
```sh
......@@ -49,7 +50,7 @@ sys update
### Upgrade the system
Without second argument, all packages with newer versions (so the whole system) will be upgraded.
When a package name is given as second argument, only that package is upgraded.
When a package name is given as second argument, only that package and it's dependencies are upgraded.
```sh
sys upgrade
......
......@@ -9,9 +9,9 @@ Last time we've seen how to [install](https://gitlab.com/bztsrc/osz/blob/master/
Service Management
------------------
In most POSIX like operating systems, services are managed by scripts (SysV init) or by another service (systemd). In both cases
you have commands to start and stop services (or if you like daemons) from the shell. OS/Z is not an exception, it has
`init` system service which responsible for that. Please note that unlike user services, [system services](https://gitlab.com/bztsrc/osz/blob/master/docs/services.md)
In most POSIX like operating systems, services are managed by scripts (SysV init) or by another service (systemd or launchd). In
both cases you have commands to start and stop services (or if you like daemons) from the shell. OS/Z is not an exception, it has
`init` system service which is responsible for that. Please note that unlike user services, [system services](https://gitlab.com/bztsrc/osz/blob/master/docs/services.md)
cannot be controlled from command line.
### Starting a service
......
......@@ -6,3 +6,10 @@ Preface
Last time we talked about [services](https://gitlab.com/bztsrc/osz/blob/master/docs/howto6-services.md) from sysadmin's point of view.
Now we'll talk about using the system as and end user.
Login screen
------------
Because OS/Z is a multi-user operating system, when it's booted up, it will show you a login screen, asking for your username and
password. After successful authentication, `logind` will fork itself and became a session manager. If and when the session manager
exits, the control is passed back to the parent logind instance which will ask for username and password again.
......@@ -6,7 +6,7 @@ OS/Z How a keypress is processed
IRQ message in a task's message queue that's selected by IRQ Routing Table (keyboard driver in our case).
3. task switch to [keyboard driver](https://gitlab.com/bztsrc/osz/blob/master/src/drivers/input/ps2) task, if it was blocked, it's awaken
4. ISR leaves, new IRQs can fire (save keyboard IRQ)
5. ps2 driver handles the message, and dispatches it to [keyboard.S](https://gitlab.com/bztsrc/osz/blob/master/src/drivers/input/ps2/keyboard.S) which in return reads scancode from keyboard
5. ps2 driver handles the message, and dispatches it to `irq1()` in [keyboard.S](https://gitlab.com/bztsrc/osz/blob/master/src/drivers/input/ps2/keyboard.S) which in return reads scancode from keyboard
6. sends a SYS_ack message to core via [syscall](https://gitlab.com/bztsrc/osz/blob/master/src/lib/libc/x86_64/syscall.S), and core re-enables keyboard IRQ
7. sends a scancode message to UI task
8. task switch to UI task (it's also awaken if necessary)
......@@ -17,5 +17,5 @@ OS/Z How a keypress is processed
13. receives key and draws a unicode character for it
14. sends a window flush message to the UI task
15. task switch to UI task
16. composites windows and copies the result to the framebuffer
16. composites windows and the display driver copies the result to the framebuffer
17. finally you see the key that you've typed :-)
docs/oszdbg1.png

2.33 KB | W: | H:

docs/oszdbg1.png

1.97 KB | W: | H:

docs/oszdbg1.png
docs/oszdbg1.png
docs/oszdbg1.png
docs/oszdbg1.png
  • 2-up
  • Swipe
  • Onion skin
......@@ -19,7 +19,7 @@ Porting core
------------
Note there's a distinction between platform and architecture. The latter only includes the CPU with some essentional
features like MMU or FPU (usually built-in CPU, but not necessairly), while the former includes everything shipped on
features like MMU or FPU (usually built-in the CPU, but not necessairly), while the former includes everything shipped on
the motherboard or SoC (usually not replaceable by the end user), like interrupt controllers, nvram, PCI(e) bus and
BIOS or UEFI firmware.
......@@ -42,16 +42,16 @@ rest of the operating system by initializing lowest level parts, enumerating sys
with all of that, it passes control the `init` system service.
A significant part of the `core` is platform independent, and written in C, [sys/core](https://gitlab.com/bztsrc/osz/blob/master/src/core).
Hardware specific part is written in C and Assembly, in [sys/core/(platform)](https://gitlab.com/bztsrc/osz/blob/master/src/core/x86_64). This
is the only part of OS/Z that runs in supervisor mode (ring 0 on x86_64), and it must be kept small.
Hardware specific part is written in C and Assembly, in [sys/core/(platform)](https://gitlab.com/bztsrc/osz/blob/master/src/core/x86_64). The
`core` is the only part of OS/Z that runs in supervisor mode (ring 0 on x86_64, and EL1 on AArch64), and it must be kept small. Currently it's around 100k
(out of which 32k being the UNICODE system font, which can be replaced by a 2k ASCII font), and with debugger compiled in it's still less than 200k.
Under no circumstances should it reach 256k.
Although OS/Z is a micro-kernel, the `core` does a little more than other typical micro-kernels, like Minix. It is so
because the goal was performance and also to provide a minimal, but well-defined support for each platform. So `core`
abstracts the platform in a minimalistic approach: everything that's required to support individual, pre-empted, auto
detected, inter communicating device driver tasks must be in `core`, but nothing more and definitely not the driver's
code itself. Everything else must be pushed out to user space in separate binaries. There's
a simple rule: if the hardware can be physically replaced by the end user (eg.: using PCI card or BIOS setup), it must
be supported by a device driver and not by `core` for sure.
Although OS/Z is a micro-kernel, the `core` does a little bit more than other typical micro-kernels, like Minix. For example
it does not delegate memory memory management for performance reasons. The goal was performance and also to provide a minimal,
but well-defined interface for each platform. So `core` abstracts the platform in a minimalistic approach: everything that's
required to support individual, pre-empted, inter communicating tasks must be in `core`. Everything else (including device drivers)
must be pushed out to user space in separate binaries, as micro-kernel architecture dictates.
Porting device drivers
----------------------
......@@ -78,12 +78,12 @@ Porting libc
This shared library is called `libc` for historic reasons, in reality it's a platform independent interface
to all OS/Z functions, so should be called `libosz`. It provides library functions for user space services, libraries
and applications as well as for device drivers. It's mostly written in C, but some parts had to be written in Assembly
(like memcpy() for performance and calling a supervisor mode service in `core` for compatibility). It hides
all details of low level messaging, but just in case also provides a platform independent,
(like memcpy() for performance and calling a supervisor mode service in `core` to support native instructions). It hides
all the details of low level messaging, but just in case also provides a platform independent,
[user level abstraction for messaging](https://gitlab.com/bztsrc/osz/blob/master/docs/messages.md#low-level-user-library).
All programs compiled for OS/Z must be dynamically linked with `libc`, and must not use other, lower level abstractions
directly. At `libc` level, OS/Z is totally platform independent.
(like syscall/svc interface) directly. At `libc` level, OS/Z is totally platform independent.
Porting libraries and applications
----------------------------------
......@@ -93,14 +93,17 @@ all OS/Z services. This ease the development for new programs, but also makes po
systems a little bit harder. You have to use `#ifdef _OS_Z_` pre-define blocks. There's no way around this as
[OS/Z is not POSIX](https://gitlab.com/bztsrc/osz/blob/master/docs/posix.md) by design. Although I did my best to make
it similar to POSIX, that's still OS/Z's own interface. For one, `errno()` is a function and not a global
variable. On the other hand this `libc` provides everything for interfacing with OS/Z, which guarantees that all OS
specific stuff is limited to this single library.
variable, and there are no `read()`/`fread()` duplications. On the other hand this `libc` provides everything for interfacing
with OS/Z, which guarantees that all OS specific stuff is limited to this single library.
I've tried to do my best to keep system services portable, and used platform independent C code whenever possible. This goal
was in 99% fullfilled, but for performance reasons some little parts had to be written in Assembly (like pixbuf blitter in libui).
OS/Z is designed in a way that at shared library level all applications are binary compatible for a specific
architecture, and source compatible for all architectures and platforms. Therefore libraries should not and applications
must not use any Assembly or platform specific C code. If such a thing is required and cannot be solved with device
drivers, then that code must be placed in a separate library with an interface common on all platforms, and should be
implemented for all the platforms. The pixman library would be a perfect example for that.
implemented for all the platforms. The pixbuf library in `libui` would be a perfect example for that.
Because it is the `libc` level where OS/Z becames platform independent, unit and functionality
[tests](https://gitlab.com/bztsrc/osz/blob/master/src/test) are provided for this level and above, but not for lower
......
......@@ -15,7 +15,17 @@ the system was designed to be limitless as possible, a lot of [errno values](htt
are meaningless and therefore non-existent in OS/Z. Like ENFILE (File table overflow) or EMFILE (Too many open files). Using [FS/Z](https://gitlab.com/bztsrc/osz/blob/master/docs/fs.md) you'll
never see EFBIG (File too large) or EMLINK (Too many links) errors either.
It also has new values, like ENOTUNI (Not an union) or ENOTSHM (Not a shared memory buffer).
It also has new values, like ENOTUNI (Not an union) or ENOTSHM (Not a shared memory buffer) unknown to POSIX.
Straightforwardness
-------------------
The usual POSIX libc provides many functions for exactly the same functionality. OS/Z on the other hand prefers ANSI C's
philosophy, as in provide only one clear way to do a certain job. Therefore there are no open/fopen, write/fwrite, getc/fgetc,
dprintf/fprintf duplications, only one version exists of each function. Where the function deals with a file stream, it's
prefixed with an "f", and expects `fid_t` (and integer index) instead of `FILE *` struct pointer. Also I've standardized the
order of the arguments. Therefore for example writing to a file looks like `size_t fwrite(fid_t f, void *ptr, size_t size)`,
which is definitely not POSIX standard, but makes more sense.
Paths
-----
......@@ -25,7 +35,7 @@ lot of disk access and speed up things. For that it is mandatory to end director
similar how VMS worked. OS/Z also implements per file versioning from FILES11 (the file system the author used and liked a lot
at his university times).
As an addition, OS/Z handles a special `...` directory as a joker. It means all sub-directories at that level, and will be
As an addition, OS/Z also handles a special `...` directory as a joker. It means all sub-directories at that level, and will be
resolved as the first full path that matches. For example, assuming we have '/a/b/c', '/a/d/a' and '/a/e/a' directories, then
'/a/.../a' will return '/a/d/a' only. Unlike in VMS, the three dots only matches one directory level, and does not dive into
their sub-directories recursively.
......
......@@ -3,32 +3,32 @@ OS/Z Services
There are two different kind of services: system services and user services. System services
must reside on the initial ramdisk, as they are the server counterparts of `libc`, and they
cannot be controlled from userspace. User services on the other hand are controlled by the
cannot be controlled from userspace. User services (UNIX daemons) on the other hand are controlled by the
init system service, and they can be loaded from external disks as well.
### Service hierarchy
```
+--------------------------------------------------+
ring 0 | CORE (+IDLE) | (supervisor)
--------+---------+----+----+------------------------------+---------------------
ring 3 | Drivers | FS | UI | syslog | inet | sound | init | (system services)
+-------------------------------------------+ |
| logind | prnd | httpd | ... | (user services controlled by init)
+--------------------------------------------------+
+--------------------------------------------------+
| identity | sh | sys | fsck | test | ... | (normal user applications)
+--------------------------------------------------+
+--------------------------------------------------+
EL1 / ring 0 | CORE (+IDLE) | (supervisor)
--------------+---------+----+----+------------------------------+---------------------
EL0 / ring 3 | Drivers | FS | UI | syslog | inet | sound | init | (system services)
+-------------------------------------------+ |
| logind | prnd | httpd | ... | (user services controlled by init)
+--------------------------------------------------+
+--------------------------------------------------+
| identity | sh | sys | fsck | test | ... | (normal user applications)
+--------------------------------------------------+
```
CORE
----
Special service which runs in privileged supervisor mode. It's always mapped in all memory map.
Special service which runs in privileged supervisor mode. It's always mapped in all address space.
The lowest level code of OS/Z like ISRs, physical memory allocation, task switching, timers and such
are implemented here. Although it looks like a separate task, `IDLE` is part of core as it needs to halt the CPU.
All the other services (including the device driver tasks) are running in non-privileged user mode (ring 3).
All the other services (including the device driver tasks) are running in non-privileged user mode (EL0 / ring 3).
Typical functions: alarm(), setuid(), setsighandler(), yield(), fork(), execve(), mmap(), munmap().
......@@ -36,16 +36,16 @@ Typical functions: alarm(), setuid(), setsighandler(), yield(), fork(), execve()
Drivers
-------
Each device driver has it's own task. The MMIO mapped into their bss segment for architectures that support that, and
they are allowed to access I/O address space. Only "FS" task and CORE allowed to send messages to them. There's one
exception, the video driver, which does not have it's own task, rather it's loaded into "UI" task's address space.
Each device driver has it's own task. The MMIO mapped into their bss segment for architectures that support mapping, and
they are allowed to access I/O address space. Only `FS` task and `CORE` allowed to send messages to them. There's one
exception, the display driver, which does not have it's own task, rather it's loaded into `UI` task's address space.
Typical functions: IRQ(), ioctl(), reset(), read(), write().
FS
--
The file system service. It's a big database server, that emulates all layers as files.
The file system service. It's a big database server, which emulates all layers as files.
OS/Z shares UNIX's and [Plan 9](https://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs)'s famous "everything is a file" approach.
This process has the initrd mapped in it's bss segment, and uses free memory as a disk cache. On boot, the file system
drivers are loaded into it's address space as shared libraries. It has one built-in file system, called "devfs".
......@@ -56,10 +56,11 @@ UI
--
User Interface service. It's job is to manage tty consoles, window surfaces and GL contexts. And
when it's required, composites all of them in a single frame and tells the hardware to flush video memory.
It receives scancode from input devices and sends keycode messages to the focused application in return. It has a
double screen buffer that can hold a stereographic composited image. The video frame buffer is also mapped into its
address space. The video device driver for local ":0" display is loaded into this task instead of it's own driver task.
when required, composites all of them in a single frame and flushes the result to video memory.
It receives scancodes from input devices and sends keycode messages to the focused application in return. It shares
double screen buffers with the applications that can hold a stereographic composited image. The video frame buffer is
also mapped into its address space. The video card driver for local "0" display is loaded into this task (instead of
it's own task) which accesses that frame buffer directly.
Typical functions: ui_opendisplay(), ui_createwindow(), ui_destroywindow().
......
......@@ -15,5 +15,6 @@ Files
- *bochs.rc* [bochs](http://bochs.sourceforge.net/) configuration file
- *logo.png* *logo.tga* OS/Z logo
- *script.gdb* startup script for [GDB](https://www.sourceware.org/gdb/)
- *system.8x16.psf* [PC Screen Font](https://gitlab.com/bztsrc/osz/blob/master/src/core/font.h), unicode font for kprintf
- *system.8x16.txt* font source, use [writepsf](https://gitlab.com/talamus/solarize-12x29-psf/blob/master/writepsf) to compile. Watch out! Big file, will make gitlab sweat!
- *system.8x16.psf* [PC Screen Font](https://gitlab.com/bztsrc/osz/blob/master/src/core/font.h), default unicode font for kprintf and ttys
- *system.8x16.txt* font source, use [txt2psf.sh](https://gitlab.com/bztsrc/osz/blob/master/tools/txt2psf.sh) to compile. Watch out, big file!
- *system.8x8.psf* much smaller alternative font for embedded versions, only 256 glyphs. To use this, you have to edit src/core/Makefile's font.o rule at line 46.
......@@ -61,6 +61,7 @@ extern int errno();
#define ENOTSHM 26 // Not a shared memory buffer
#define EBADF 27 // Bad file number
#define ENOTBIG 28 // Buffer not big enough
#define ENOTIMPL 29 // Not implemented