Fix some errors, update README, try to introduce SHA256 code verification.

parent 8b7bbdf7
Pipeline #65337645 passed with stages
in 10 minutes and 3 seconds
......@@ -19,3 +19,6 @@
[submodule "contrib/single_cream"]
path = contrib/single_cream
url = https://github.com/rain-1/single_cream.git
[submodule "contrib/sha-2"]
path = contrib/sha-2
url = https://github.com/amosnier/sha-2.git
......@@ -2,14 +2,13 @@
`asmc` is an extremely minimal operating system, whose binary seed
(the compiled machine code that is fed to the CPU when the system
boots) is very small (around 15 KiB, maybe could be further shrunk in
the future). Such compiled code is enough to contain a minimal IO
environment and compiler, which is used to compile a more powerful
environment and compiler, further used to compile even more powerful
things, until a fully running system is bootstrapped. In this way
nearly any code running in your computer is compiled during the boot
procedure, except the the initial seed that ideally is kept as small
as possible.
boots) is very small (around 6 KiB). Such compiled code is enough to
contain a minimal IO environment and compiler, which is used to
compile a more powerful environment and compiler, further used to
compile even more powerful things, until a fully running system is
bootstrapped. In this way nearly any code running in your computer is
compiled during the boot procedure, except the the initial seed that
ideally is kept as small as possible.
This at least is the plan; from the moment we are not yet at the point
where we manage to compile a real system environment. However, work is
......@@ -23,8 +22,49 @@ a C compiler. In the end a different path was devised: the initial
seed is written in Assembly and embeds a G compiler (where G is a
custom language, sitting something between Assembly and C, conceived
to be very easy to compile); the G compiler is then use to produce a C
compiler. Assembly is never directly used in this chain, although of
course continuously behind the curtains.
compiler. Assembly is never directly used in this chain, although it
is of course continuously behind the curtains.
`asmc` is currently able to:
* boot from a minimal seed of around 6 KiB;
* compile and execute source code written in the G language (see
below);
* initialize a minimal environment, including a terminal (writing to
serial port and to monitor), dynamic memory allocation, simple data
structures, disk access and a simple virtual file system;
* compile a basic assembler and a basic C compiler, both of which are
written in G;
* use them to compile a [lightly patched
version](https://gitlab.com/giomasce/tinycc/tree/softfloat) of
[tinycc](http://www.tinycc.org/), using a custom basic C standard
library;
* use tinycc to compile another copy of itself (in line of principle
that can be repeated as long as you want);
* use the second tinycc to compile a [patched and customized
version](https://gitlab.com/giomasce/ipxe/) of
[iPXE](http://ipxe.org/);
* use iPXE to initialize the network card and be ready to download
more source code from the network.
Hopefully `asmc` will eventually be able to:
* download the Linux source code, [patch
it](https://gitlab.com/giomasce/linux/tree/tcc2) and compile via a
custom build script;
* compile a minimal userspace environment, which includes at least a
copy of tinycc;
* boot Linux with this userspace, and be ready to compile other
source code to continue bootstrapping.
## Enough talking, show me the deal!
......@@ -42,24 +82,19 @@ the submodules as well:
Then just call `make` in the toplevel directory of the repository. A
subdirectory named `build` will be created, with all compilation
artifacts inside it. In particular `build/boot_asmg.x86.qcow2` is a
bootable disk image, which you can run with QEMU:
artifacts inside it. Before running the virtual machine you have to
start the web server that serves code for the later stages of `asmc`:
open another terminal, enter the `http` directory and run
python3 -m http.server 8080
qemu-system-i386 -m 256M -hda build/boot_asmg.x86.qcow2 -serial stdio -device isa-debug-exit -display none
Then:
qemu-system-i386 -m 256M -hda build/boot_asmg.x86.qcow2 -serial stdio -display none
(if your host system supports it, you can add `-enable-kvm` to benefit
of KVM acceleration)
Unless I have broken something, this should run a little operating
system that compiles a little C compiler, and later uses such compiler
to compile from sources a [patched
version](https://gitlab.com/giomasce/tinycc/tree/softfloat) of
[tinycc](http://www.tinycc.org/), which is then used to compile a
little test C program. In the future, tinycc will be used to continue
the chain and build a Linux kernel and GNU userspace, so that you will
actually have a complete operating system entirely compiled from
scratches at computer boot!
WARNING! ATTENTION! Remember that you are giving full control over you
hardware to an experimental program whose author is not an expert
operating system programmer: it could have bugs and overwrite your
......
......@@ -20,7 +20,7 @@ const TEST_MAP TEST_ALL
const TEST_INT64 TEST_ALL
const TEST_C TEST_ALL
const RUN_MM0 1
const RUN_MM0 0
const RUN_ASM 0
const RUN_FASM 0
const RUN_C 0
......
Subproject commit ff76937294694ec347819d81a1f580187fb34246
......@@ -2,5 +2,9 @@
#include "asmc.h"
#include "ipxe_asmc_structs.h"
downloaded_file *download_file(const char *url, const char *dest_file);
#include "sha-2/sha-256.h"
void download_file(const char *url, const char *dest_file);
void *compile_file(const char *code, const char *symbol, void **buffer);
void *read_whole_file(FILE *fin, size_t *size);
void compute_file_sha256(const char *filename, uint8_t hash[32]);
......@@ -27,17 +27,15 @@ int recursively_compile() {
includes[6] = ASMC_PREFIX "/tinycc/softfloat/8086";
const char *files[1];
printf("Compiling tinycc recursively... (level %d)\n", LEVEL);
#if LEVEL == 1 && defined(RUN_IPXE)
#define ADD_TCC_SYMBOLS
files[0] = ASMC_PREFIX "/run_tcc_ipxe.c";
printf("I will now try to compile and execute tinycc, diverting to iPXE! (level %d)\n", LEVEL);
#elif LEVEL == 1 && defined(RUN_SINGLE_CREAM)
#define ADD_TCC_SYMBOLS
files[0] = ASMC_PREFIX "/run_tcc_sc.c";
printf("I will now try to compile and execute tinycc, diverting to single_cream! (level %d)\n", LEVEL);
#else
files[0] = ASMC_PREFIX "/run_tcc.c";
printf("I will now try to compile and execute tinycc recursively! (level %d)\n", LEVEL);
#endif
TCCState *state = tcc_new();
......@@ -82,7 +80,7 @@ int recursively_compile() {
}
printf("Entering tinycc...\n");
int ret = start();
printf("The program returned %d!\n", ret);
printf("Tinycc returned %d!\n", ret);
tcc_delete(state);
return 0;
......
......@@ -157,6 +157,37 @@ const char *sources[][2] = {
#include "ipxe_handover.h"
#include "asmc_api.h"
#include "sha-2/sha-256.c"
void *read_whole_file(FILE *fin, size_t *size) {
size_t buf_size = 16;
char *buf = malloc(buf_size);
size_t buf_pos = 0;
while (1) {
size_t res = fread(buf + buf_pos, 1, buf_size - buf_pos, fin);
buf_pos += res;
if (buf_pos < buf_size) {
break;
} else {
buf_size *= 2;
buf = realloc(buf, buf_size);
}
}
if (size) {
*size = buf_pos;
}
return buf;
}
void compute_file_sha256(const char *filename, uint8_t hash[32]) {
FILE *fin = fopen(filename, "rb");
size_t size;
void *buf = read_whole_file(fin, &size);
fclose(fin);
calc_sha_256(hash, buf, size);
free(buf);
}
int table_comp(void *t1, void *t2) {
table_sect *s1 = t1;
table_sect *s2 = t2;
......@@ -260,18 +291,36 @@ void *pop_from_ipxe() {
coro_t *coro_ipxe;
downloaded_file *download_file(const char *url, const char *dest_file) {
void download_file(const char *url, const char *dest_file) {
push_to_ipxe(strdup("download"));
push_to_ipxe(strdup(url));
coro_enter(coro_ipxe);
downloaded_file *df = pop_from_ipxe();
assert(df->size > 0);
/*printf("Received document of %d bytes!\n", df->size);
printf("---\n");
for (size_t i = 0; i < df->size; i++) {
putchar(df->data[i]);
FILE *fout = fopen(dest_file, "wb");
fwrite(df->data, 1, df->size, fout);
fclose(fout);
free(df->data);
free(df);
}
void download_check_file(const char *url, const char *dest_file, const uint8_t hash[32]) {
push_to_ipxe(strdup("download"));
push_to_ipxe(strdup(url));
coro_enter(coro_ipxe);
downloaded_file *df = pop_from_ipxe();
assert(df->size > 0);
uint8_t computed_hash[32];
calc_sha_256(computed_hash, df->data, df->size);
printf("Computed hash:");
size_t i;
for (i = 0; i < 32; i++) {
printf(" %02x", computed_hash[i]);
}
printf("\n");
for (i = 0; i < 32; i++) {
assert(computed_hash[i] == hash[i]);
}
printf("---\n");*/
FILE *fout = fopen(dest_file, "wb");
fwrite(df->data, 1, df->size, fout);
fclose(fout);
......@@ -307,29 +356,32 @@ void *compile_file(const char *filename, const char *symbol, void **buffer) {
tcc_add_symbol(state, "tcc_get_symbol", tcc_get_symbol);
tcc_add_symbol(state, "download_file", download_file);
tcc_add_symbol(state, "compile_file", compile_file);
tcc_add_symbol(state, "calc_sha_256", calc_sha_256);
tcc_add_symbol(state, "read_whole_file", read_whole_file);
tcc_add_symbol(state, "compute_file_sha256", compute_file_sha256);
int res;
for (int i = 0; i < sizeof(includes2) / sizeof(includes2[0]); i++) {
res = tcc_add_include_path(state, includes2[i]);
if (res) {
printf("tcc_add_include_path() failed...\n");
return 1;
return 0;
}
}
res = tcc_add_file(state, filename);
if (res) {
printf("tcc_add_file() failed...\n");
return 1;
return 0;
}
int size = tcc_relocate(state, NULL);
if (size < 0) {
printf("tcc_relocate() failed...\n");
return 1;
return 0;
}
*buffer = malloc(size);
size = tcc_relocate(state, *buffer);
if (size < 0) {
printf("second tcc_relocate() failed...\n");
return 1;
return 0;
}
void *symb = tcc_get_symbol(state, symbol);
tcc_delete(state);
......@@ -416,10 +468,17 @@ int main(int argc, char *argv[]) {
// Download the continuation program
const char *url = "http://10.0.2.2:8080/continue.c";
printf("Request to download %s\n", url);
printf("Request to download %s...\n", url);
const uint32_t hash[32] = {0xEB, 0x42, 0xD9, 0x44, 0xC2, 0x2E, 0x46, 0x4F, 0x39, 0xAF, 0x75, 0x31, 0xED, 0x74, 0x99, 0xA1,
0x1D, 0xC5, 0xDC, 0x6E, 0x34, 0x77, 0x3C, 0x35, 0xFE, 0x14, 0x1B, 0xAC, 0x5D, 0x53, 0x3D, 0x5D};
download_file(url, "/ram/continue.c");
//download_check_file(url, "/ram/continue.c", hash);
void *cont_buf;
int (*cont_symb)(int, char*[]) = compile_file("/ram/continue.c", "_start", &cont_buf);
if (!cont_symb) {
printf("Compilation failed...\n");
return 1;
}
char *cont_argv[] = {"_main"};
cont_symb(sizeof(cont_argv) / sizeof(cont_argv[0]), cont_argv);
free(cont_buf);
......@@ -430,7 +489,7 @@ int main(int argc, char *argv[]) {
//res = main_symb(&ih);
printf("Returning from iPXE!\n");
printf("iPXE exited!\n");
coro_destroy(coro_ipxe);
free_handover(&ih);
free(ipxe_buf);
......
../contrib/sha-2
\ No newline at end of file
......@@ -2,8 +2,8 @@
#include <stdio.h>
#include <asmc_api.h>
int main() {
printf("Hello, continuing world!\n");
int main(int argc, char *argv[]) {
printf("Hello world from continue.c!\n");
download_file("http://10.0.2.2:8080/continue2.c", "/ram/continue2.c");
void *cont2_buf;
int (*cont2_symb)(int, char*[]) = compile_file("/ram/continue2.c", "_start", &cont2_buf);
......
......@@ -468,33 +468,13 @@ void prepare_real_mode_gdt() {
gdt[3].base_high = 0x00;
}
void *read_all_file(FILE *fin, size_t **size) {
size_t buf_size = 16;
char *buf = malloc(buf_size);
size_t buf_pos = 0;
while (1) {
size_t res = fread(buf + buf_pos, 1, buf_size - buf_pos, fin);
buf_pos += res;
if (buf_pos < buf_size) {
break;
} else {
buf_size *= 2;
buf = realloc(buf, buf_size);
}
}
if (size) {
*size = buf_pos;
}
return buf;
}
//struct boot_params boot_params __attribute__((aligned(16)));
void prepare_setup_header(struct setup_header *hdr, void *initrd, size_t initrd_size) {
hdr->vid_mode = 0xffff;
hdr->type_of_loader = 0xff;
hdr->loadflags = 0x81;
hdr->ramdisk_image = initrd;
hdr->ramdisk_image = (uint32_t) initrd;
hdr->ramdisk_size = initrd_size;
hdr->heap_end_ptr = 0xde00;
hdr->cmd_line_ptr = 0x1e000;
......@@ -531,12 +511,12 @@ void load_linux32() {
assert(real_mode->hdr.boot_flag == 0xaa55);
assert(real_mode->hdr.header == 0x53726448);
size_t prot_mode_size;
void *prot_mode = read_all_file(fin, &prot_mode_size);
void *prot_mode = read_whole_file(fin, &prot_mode_size);
printf("Protected mode code/data is 0x%x bytes long\n", prot_mode_size);
fclose(fin);
fin = fopen("/ram/cont/initrd.img", "rb");
size_t initrd_size;
void *initrd = read_all_file(fin, &initrd_size);
void *initrd = read_whole_file(fin, &initrd_size);
printf("Initrd is 0x%x bytes long\n", initrd_size);
fclose(fin);
// Align initrd
......@@ -548,7 +528,7 @@ void load_linux32() {
char *cmdline = "verbose";
// Here we really smash our operating system!
memset(0x10000, 0, 0x10000);
memset((void*) 0x10000, 0, 0x10000);
memcpy((void*) 0x10000, real_mode, real_mode_size);
memcpy((void*) 0x1e000, cmdline, strlen(cmdline));
//memcpy((void*) 0x100000, prot_mode, prot_mode_size);
......@@ -591,21 +571,21 @@ void load_linux() {
assert(real_mode->hdr.boot_flag == 0xaa55);
assert(real_mode->hdr.header == 0x53726448);
size_t prot_mode_size;
void *prot_mode = read_all_file(fin, &prot_mode_size);
void *prot_mode = read_whole_file(fin, &prot_mode_size);
printf("Protected mode code/data is 0x%x bytes long\n", prot_mode_size);
fclose(fin);
prepare_setup_header(&real_mode->hdr, 0, 0);
//load_linux_gdt();
fin = fopen("/ram/cont/realmode.bin", "rb");
size_t trampoline_size;
void *trampoline = read_all_file(fin, &trampoline_size);
void *trampoline = read_whole_file(fin, &trampoline_size);
printf("Trampoline is 0x%x bytes long\n", trampoline_size);
fclose(fin);
prepare_real_mode_gdt();
char *cmdline = "verbose";
// Here we really smash our operating system!
memset(0x10000, 0, 0x10000);
memset((void*) 0x10000, 0, 0x10000);
memcpy((void*) 0x500, trampoline, trampoline_size);
memcpy((void*) 0x10000, real_mode, real_mode_size);
memcpy((void*) 0x1e000, cmdline, strlen(cmdline));
......@@ -622,25 +602,25 @@ void load_mlb() {
download_file("http://10.0.2.2:8080/example", "/ram/cont/bootloader");
FILE *fin = fopen("/ram/cont/realmode.bin", "rb");
size_t trampoline_size;
void *trampoline = read_all_file(fin, &trampoline_size);
void *trampoline = read_whole_file(fin, &trampoline_size);
printf("Trampoline is 0x%x bytes long\n", trampoline_size);
fclose(fin);
prepare_real_mode_gdt();
fin = fopen("/ram/cont/bootloader", "rb");
size_t bootloader_size;
void *bootloader = read_all_file(fin, &bootloader_size);
void *bootloader = read_whole_file(fin, &bootloader_size);
printf("Bootloader is 0x%x bytes long\n", bootloader_size);
fclose(fin);
prepare_real_mode_gdt();
memset(0x10000, 0, 0x10000);
memset((void*) 0x10000, 0, 0x10000);
memcpy((void*) 0x500, trampoline, trampoline_size);
memcpy((void*) 0x7c00, bootloader, bootloader_size);
((void (*)()) 0x500)();
}
int main() {
printf("Hello, continuing second world!\n");
int main(int argc, char *argv[]) {
printf("Hello world from continue2.c!\n");
//load_linux32();
//load_linux();
//load_mlb();
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment