This goal of this project is to investigate improving determinism for the Linux Kernel.
For some kinds of software workloads (including AI/ML workloads for autonomous
operation of robots and vehicles) it has been suggested that variability in OS
latency can affect overall performance, throughput and potentially quality.
This is an exploratory project to investigate the latency behaviour of Linux
kernels in a range of scenarios (multiple ISAs, stressed and unstressed, with
and without tweaks to improve latency and determinism for specific loads).
Our outputs are expected to include documentation, free software (ISC License) and results from various experiments which we aim to make reproducible.
The purpose of this wiki is to document the recommended kernel changes found to give a more deterministic system, along with test results.
Throughout the wiki there are many test results performed on various bits of hardware, but this page summarises the latest and best improvements found from the project. Note that this is changing document as more improvements are tested.
In the graphs shown below it's possible to have a first look to the achieved improvements of the latency, to have a deeper look go to the test pages of each hardware.
In the graphs the purple lines are the latency of the baseline kernel and the green lines are the latency of the tuned kernel.
JETSON running memory-bound
RIG running memory-bound
This graph shows a feature that became common throughout our testing; the jumps in latency suggest a CPU frequency change, likely due to thermal variance. It can be seen from our tuned kernel results that those jumps have been smoothed out.
All of the tests that have been done for each hardware, along with analysis and information about each test, can be found here:
A jupyter notebook is a useful, interactive way to present a report from test results. It combines markdown formatted text with live python code, meaning that plotted data can be analysed in more detail and much easier.
To add multiple files as subplots, define an array of axes for the plots to use (each subplot is one array element), then in plot() add an argument ax=axes[n] defining which subplot to use. Note that 2D arrays can be used to plot subplots in grids (i.e. 4 plots in a 2x2 grid).