...
 
Commits (3)
# System
```
[Author]
@SSStormy (GitHub & GitLab)
[Hardware]
CPU: AMD FX6300
Motherboard: ASUS M5A97 EVO R2.0
Motherboard BIOS: 2603
RAM: 16GB
GPU: Gigabyte Nvidia GeForce GTX 970
GPU model: GV-N970G1 GAMING-4GD
GPU BIOS: 84.04.2f.00.80
GPU codename: GM204
[Software]
Linux Distro: Arch Linux
Linux Kernel: 4.18.1-arch1-1-ARCH
Nvidia divers: 396.51
QEMU version: 2.12.1-1
OVMF version: NONE
[Guests]
Windows 10 Pro 1709 x64
```
# The final script
You can find it [here](https://github.com/SSStormy/dotfiles/blob/master/scripts/windows.sh) among my dotfiles.
# How I did it
## vBIOS
I used Method 3.
Dumped it via GPU-Z and then used `bless` in Arch to edit it. The header was there and I removed it with no issues.
## Script will not work if run from within a DM/WM/X11
I could not get the script to work when running it from my WM (i3). lightdm was stopped, X11 and i3 were nowhere to be seen but the GPU was not properly detached. On screen I saw a black terminal with a blinking cursor. I could switch between TTYs.
The solution is to switch to a TTY (ctrl+shift+f2/f3 etc) and run the script from there.
## "module nvidia is in use"
This occured during the initial module unloading stage. The nvidia module wasn't being unloaded.
The solution was to unload the `nvidia_uvm` module: Add
```
modprobe -r nvidia_uvm
```
Next to `modprobe -r nvidia_drm`
From testing it appears that you don't have to load it manually since `modprobe nvidia` does the job, but just in case: To load it back up, add:
```
modprobe nvidia_uvm
```
Next to `modprobe nvidia_drm`
## Cannot boot off of ANY drives with OVMF
During testing where I didn't passthrough any PCI devices, the VM failed to boot off of any drive/iso/CD I gave to it. This only occured when I booted QEMU with the OVMF UEFI. I tested the `ovmf-git` and `edk2-ovmf` (AUR) but nothing had changed.
The solution was to not use OVMF: Remove
```
-drive if=pflash,format=raw,readonly,file=$OVMF_CODE \
```
Doing this still allowed me to passthrough PCI devices.
## Incredibly slow drive speeds
During testing I had experienced some horrible drive speeds. Booting into the OVMF UEFI bios took well over a minute.
The solution was to tell QEMU to run the VM as a q35 machine: Add
```
-machine q35 \
```
Next to `-enable-kvm \`
## QCOW2
I used QCOW2 to create my drive image:
```
qemu-img create -f qcow2 windows.qcow2 200G
```
This is how I passed it onto QEMU:
```
-drive file=/home/$USER/vm/drive/windows.qcow2,if=virtio,format=qcow2 \
```
[Here](https://gist.github.com/shamil/62935d9b456a6f9877b5) is a guide on how to mount QCOW2 as if it were an ISO/block on the host.
## QCOW2 Drive does not show up during windows install
Installing the vioscsi drivers did not help either.
The solution was to install the viostar amd64 driver. Then the drive showed up perfectly fine and worked for all further startups.
## No network
After installing windows and booting into it, windows could not establish a ethernet connection.
The solution was to remove these from the QEMU launch arguments:
```
-device virtio-net-pci,netdev=n1 \
-netdev user,id=n1 \
```
## Passing through USB devices via -object input-linux
For my peripherals (mouse/kb) I passed this to the QEMU arguments:
```
-object input-linux,id=kbd,evdev=/dev/input/by-id/usb-CM_Storm_Keyboard_--_QuickFire_XT-event-if01,grab_all=on,repeat=on \
-object input-linux,id=kbd2,evdev=/dev/input/by-id/usb-CM_Storm_Keyboard_--_QuickFire_XT-event-kbd,grab_all=on,repeat=on \
-object input-linux,id=mouse,evdev=/dev/input/by-id/usb-Logitech_USB_Optical_Mouse-event-mouse
```
## No audio I/O
After install, windows could not detect my sound card.
The solution was to pass it through to the VM:
Config:
```
IOMMU_PCI_AUDIO=00:14.2
pciaudioid="1002 4383"
pciaudiobusid="0000:00:14.2"
```
During detachment, below the GPU audio:
```
echo $pciaudioid > /sys/bus/pci/drivers/vfio-pci/new_id
echo $pciaudiobusid > /sys/bus/pci/devices/$pciaudiobusid/driver/unbind
echo $pciaudiobusid > /sys/bus/pci/drivers/vfio-pci/bind
echo $pciaudioid > /sys/bus/pci/drivers/vfio-pci/remove_id
```
In the QEMU arguments, below IOMMU_GPU:
```
-device vfio-pci,host=$IOMMU_PCI_AUDIO \
```
During reattachment, below nvidia modprobes:
```
echo $pciaudioid > /sys/bus/pci/drivers/snd_hda_intel/new_id
echo $pciaudiobusid > /sys/bus/pci/devices/$pciaudiobusid/driver/unbind
echo $pciaudiobusid > /sys/bus/pci/drivers/snd_hda_intel/bind
```
`$ /dev/input/by-id/usb*` will let you see what peripherals you can passthrough this way.
## The sleep calls
I got rid of most of the `sleep` calls and the script still worked.
## Misc: Monitor stdio
I added `-monitor stdio \` to the QEMU paramaters to be able to launch the script via TMUX, then in windows SSH into the host machine and attach to the TMUX session. This allows me to run QEMU commands from within the VM.
# A compilation of hardware and software combinations working for other people
**Before creating a merge pull, be sure to read this guide**
The files should be named like [{CPU} - {GPU} - {User}.md]; example
```
Ryzen 5 2600 - GTX 770 - @YuriAlek.md
```
## Inside must be something like this:
### System
```
[Author]
@YuriAlek
[Hardware]
CPU: AMD Ryzen 5 2600
Motherboard: Gigabyte AB350M-Gaming 3 rev1.1
Motherboard BIOS: F23d
RAM: 16GB
GPU: Gigabyte Nvidia GeForce GTX 770
GPU model: GV-N770OC-2GD
GPU BIOS: 80.04.C3.00.0F
GPU codename: GK104
[Software]
Linux Distro: ArchLinux
Linux Kernel: 4.17.14 vanilla
Nvidia divers: 396.51-1
QEMU version: 2.12.1-1
OVMF version: r24021
[Guests]
Windows 10 Pro 1709 x64
MacOS High Sierra 10.13.3
```
### How I did it
For extracting the `vBIOS` I used the 1st method in Linux and edited it.
You can add anything that you consider useful like the steps needed in your distro for installing everything or a link to your script. The more information, and solutions, the better.
### Files to modify
#### `scripts/windows.sh`
My GPU uses one more Kernel Module so I have to add it as the first module to unload and the last to load:
```
# Unload the Kernel Modules that use the GPU
modprobe -r nvidia_drm
sleep 1
...
------------------------
# Reload the kernel modules. This loads the drivers for the GPU
...
modprobe nvidia_drm
sleep 1
```
I use PulseAudio so I had to kill it for detaching the GPU.
```
## Kill X and related
pulseaudio -k
---------------------
# Reload the Display Manager to access X
...
pulseaudio --start
```
I don't pass a USB controller so I had to add manually USB devices.
```
# Remove
...
echo $usbid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
sleep 1
echo $usbbusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $usbid > /sys/bus/pci/drivers/vfio-pci/remove_id
#ls -la /sys/bus/pci/devices/$usbbusid/
sleep 1
...
-device vfio-pci,host=$IOMMU_USB \
...
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
echo $usbbusid > /sys/bus/pci/drivers/xhci_hcd/bind
sleep 10
...
----------------------
# Add to the QEMU script
...
-object input-linux,id=kbd,evdev=/dev/input/by-id/usb-HOLDCHIP_USB_Gaming_Keyboard-event-kbd,grab_all=on,repeat=on \
-object input-linux,id=kbd2,evdev=/dev/input/by-id/usb-HOLDCHIP_USB_Gaming_Keyboard-if01-event-kbd,grab_all=on,repeat=on \
-object input-linux,id=mouse-event,evdev=/dev/input/by-id/usb-Logitech_G700_Laser_Mouse_6B5EFC4B0035-event-mouse \
-object input-linux,id=kbd3,evdev=/dev/input/by-id/usb-Logitech_G700_Laser_Mouse_6B5EFC4B0035-if01-event-kbd,grab_all=on,repeat=on \
...
```
I use an image instead of a hard drive.
```
# Remove
-device virtio-scsi-pci,id=scsi0 \
-device scsi-hd,bus=scsi0.0,drive=rootfs \
-drive id=rootfs,file=$HDD,media=disk,format=raw,if=none
-----------------
# Add
-device ide-drive,bus=ide.1,drive=rootfs \
-drive id=rootfs,if=none,file=$IMG,format=raw
```
# System
```
[Author]
@YuriAlek
[Hardware]
CPU: AMD Ryzen 5 2600
Motherboard: Gigabyte AB350M-Gaming 3 rev1.1
Motherboard BIOS: F23d
RAM: 16GB
GPU: Gigabyte Nvidia GeForce GTX 770
GPU model: GV-N770OC-2GD
GPU BIOS: 80.04.C3.00.0F
GPU codename: GK104
[Software]
Linux Distro: ArchLinux
Linux Kernel: 4.17.14 vanilla
Nvidia divers: 396.51-1
QEMU version: 2.12.1-1
OVMF version: r24021
[Guests]
Windows 10 Pro 1709 x64
MacOS High Sierra 10.13.3
```
## How I did it
For extracting the `vBIOS` I used the 2nd method in Linux with nvflash and edited it. The 3rd method also worked for me.
# Install guide for QEMU and VFIO
## ArchLinux
### Install the necessary packages
```
# pacman -S qemu ovmf
```
### Enabling IOMMU support on boot
#### For `systemd-boot` edit `/boot/loader/entries/arch.conf` and add `intel_iommu=on` **OR** `amd_iommu=on` and `iommu=pt`.
```
# AMD
options root=/dev/sda2 amd_iommu=on iommu=pt
---------------------------------------------
# Intel
options root=/dev/sda2 intel_iommu=on iommu=pt
```
Reboot.
#### For `GRUB` edit `/etc/default/grub` and append your kernel options, `intel_iommu=on` **OR** `amd_iommu=on` and `iommu=pt`, to the `GRUB_CMDLINE_LINUX_DEFAULT`.
```
# AMD
GRUB_CMDLINE_LINUX_DEFAULT="amd_iommu=on iommu=pt"
---------------------------------------------------
# Intel
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on iommu=pt"
```
And then automatically re-generate the grub.cfg file with:
```
# grub-mkconfig -o /boot/grub/grub.cfg
```
Reboot.
After reboot IOMMU should be working:
```
$ dmesg | grep -e DMAR -e IOMMU
--------------------------------------------
[ 0.492684] AMD-Vi: IOMMU performance counters supported
[ 0.494370] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40
[ 0.494644] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
```
That's everything; unlike two GPU passthrough you don't need to load kernel modules on boot.
## For more information refer to this guides
[ArchWiki QEMU](https://wiki.archlinux.org/index.php/QEMU)
[ArchWiki PCI passthrough via OVMF](https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF)
[How to setup a gaming virtual machine with gpu passthrough qemu kvm libvirt and vfio](https://www.se7ensins.com/forums/threads/how-to-setup-a-gaming-virtual-machine-with-gpu-passthrough-qemu-kvm-libvirt-and-vfio.1371980/)
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
## User
VM_USER=yu
## For extracting the VBIOS
VBIOS_EXTRACT_PATH=/root/VBIOS.rom
NVFLASH=/root/nvflash_linux
extract_videobusid="0000:"
## IOMMU groups
IOMMU_GPU=06:00.0
IOMMU_GPU_AUDIO=06:00.1
IOMMU_USB=07:00.3
IOMMU_AUDIO=08:00.3
## Virsh devices
VIRSH_GPU=pci_0000_06_00_0
VIRSH_GPU_AUDIO=pci_0000_06_00_1
VIRSH_USB=pci_0000_07_00_3
VIRSH_AUDIO=pci_0000_08_00_3
# PCI BUS ID
videoid="10de 1184"
audioid="10de 0e0a"
videobusid="0000:${IOMMU_GPU}"
audiobusid="0000:${IOMMU_GPU_AUDIO}"
## QEMU options
RAM=12G
CORES=12
MACOS_RAM=12G
MACOS_CORES=4
## Network
TAP_INTERFACE=tap0
TAP_IP="10.10.10.1/24"
DNSMASQ_CONF="${BASH_SOURCE%/*}/dnsmasq.conf"
SMB_CONF="${BASH_SOURCE%/*}/smb.conf"
## Log file
LOG=/home/$VM_USER/qemu_log.log
## Local path for storage
IMAGES=/mnt/wd/vm
## Windows and MacOS
VBIOS=$IMAGES/goodies/GK104_80.04.C3.00.0F-MODED.rom
OVMF=/usr/share/ovmf/x64/OVMF_CODE.fd
OVMF_VARS=/usr/share/ovmf/x64/OVMF_VARS.fd
## Windows
VIRTIO=$IMAGES/goodies/virtio-win.iso
WINDOWS_ISO=$IMAGES/iso/win10.iso
WINDOWS_IMG=$IMAGES/hdd/windows10games.raw
## MacOS
MACOS_ISO=$IMAGES/iso/Mojave-QEMU.iso
MACOS_IMG=$IMAGES/hdd/macos-mojave.raw
MACOS_CLOVER=$IMAGES/goodies/clover-mojave.qcow2
MACOS_OVMF=$IMAGES/OSX-KVM/OVMF_CODE.fd
MACOS_OVMF_VARS=$IMAGES/OSX-KVM/OVMF_VARS.fd
#MACOS_OPTIONS="+pcid,+ssse3,+sse4.2,+popcnt,+aes,+xsave,+avx,+xsaveopt,avx2,+smep,check"
MACOS_OPTIONS="+ssse3,+sse4.2,+popcnt,+aes,+xsave,+avx,+xsaveopt,avx2,+smep,check"
## Ulimit
ULIMIT=$(ulimit -a | grep "max locked memory" | awk '{print $6}')
ULIMIT_TARGET=$(( $(echo $RAM | tr -d 'G')*1048576+100000 ))
#!/bin/bash
## Check if the script is being run as root
[[ "$EUID" -ne 0 ]] && echo "Please run as root" && exit 1
## Tap interface
tap_interface(){
tap_start(){
if [[ ! $(ip tuntap list | grep $1) ]]; then
ip tuntap add mode tap user $VM_USER name $TAP_INTERFACE
ip addr add dev $1 $TAP_IP
ip link set dev $1 up
fi
}
tap_stop(){
if [[ $(ip tuntap list | grep $1) ]]; then
ip link set dev $1 down
ip tuntap del mode tap name $TAP_INTERFACE
fi
}
if [ $1 == start ]; then tap_start $TAP_INTERFACE; elif [ $1 == stop ]; then tap_stop $TAP_INTERFACE; fi
}
## DHCP Server (DNSmasq)
dhcp_server(){
dhcp_start(){
[[ -f /var/run/dnsmasq.pid ]] || dnsmasq --conf-file=$DNSMASQ_CONF
}
dhcp_stop(){
[[ -f /var/run/dnsmasq.pid ]] && sudo kill -15 $(cat /var/run/dnsmasq.pid) && sudo rm /var/run/dnsmasq.pid
}
if [ $1 == start ]; then dhcp_start; elif [ $1 == stop ]; then dhcp_stop; fi
}
## Samba server
samba_server(){
samba_start(){
[[ -f /var/run/smbd.pid ]] || sudo smbd --configfile=$SMB_CONF
}
samba_stop(){
[[ -f /var/run/smbd.pid ]] && sudo kill -15 $(cat /var/run/smbd.pid)
}
if [ $1 == start ]; then samba_start; elif [ $1 == stop ]; then samba_stop; fi
}
## Load the config file
source "${BASH_SOURCE%/*}/config"
## Check libvirtd
[[ $(systemctl status libvirtd | grep running) ]] || systemctl start libvirtd && sleep 2 && LIBVIRTD=STOPPED
## Memory lock limit
[[ $ULIMIT != $ULIMIT_TARGET ]] && ulimit -l $ULIMIT_TARGET
## Kill the Window Manager
killall i3 > /dev/null 2>&1
sleep 1
## Remove the framebuffer and console
echo 0 > /sys/class/vtconsole/vtcon0/bind
echo 0 > /sys/class/vtconsole/vtcon1/bind
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind
## Detach the GPU and USB
virsh nodedev-detach $VIRSH_GPU > /dev/null 2>&1
virsh nodedev-detach $VIRSH_GPU_AUDIO > /dev/null 2>&1
virsh nodedev-detach $VIRSH_USB > /dev/null 2>&1
## Load vfio
modprobe vfio-pci
## Start the network
tap_interface start
dhcp_server start
samba_server start
## QEMU (VM) command
qemu-system-x86_64 -runas $VM_USER -enable-kvm \
-nographic -vga none -parallel none -serial none \
-m $MACOS_RAM \
-cpu Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,$MACOS_OPTIONS\
-machine pc-q35-2.11 \
-smp $MACOS_CORES,sockets=1,cores=$(( $MACOS_CORES / 2 )),threads=2 \
-device vfio-pci,host=$IOMMU_GPU,multifunction=on,x-vga=on,romfile=$VBIOS \
-device vfio-pci,host=$IOMMU_GPU_AUDIO \
-device vfio-pci,host=$IOMMU_USB \
-usb -device usb-kbd -device usb-tablet \
-device nec-usb-xhci,id=xhci \
-netdev tap,id=net0,ifname=$TAP_INTERFACE,script=no,downscript=no \
-device e1000-82545em,netdev=net0,id=net0,mac=52:54:00:c9:18:27 \
-device isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc" \
-drive if=pflash,format=raw,readonly,file=$MACOS_OVMF \
-drive if=pflash,format=raw,file=$MACOS_OVMF_VARS \
-smbios type=2 \
-device ide-drive,bus=ide.2,drive=Clover \
-drive id=Clover,if=none,snapshot=on,format=qcow2,file=$MACOS_CLOVER \
-device ide-drive,bus=ide.0,drive=ISO \
-drive id=ISO,if=none,snapshot=on,media=cdrom,file=$MACOS_ISO \
-device ide-drive,bus=ide.1,drive=HDD \
-drive id=HDD,file=$MACOS_IMG,media=disk,format=raw,if=none >> $LOG 2>&1 &
## Wait for QEMU
wait
## Stop the network
tap_interface stop
dhcp_server stop
samba_server stop
## Unload vfio module
modprobe -r vfio-pci
modprobe -r vfio_iommu_type1
modprobe -r vfio
## Reattach the GPU and USB
virsh nodedev-reattach $VIRSH_USB > /dev/null 2>&1
virsh nodedev-reattach $VIRSH_GPU_AUDIO > /dev/null 2>&1
virsh nodedev-reattach $VIRSH_GPU > /dev/null 2>&1
## Reload the framebuffer and console
echo 1 > /sys/class/vtconsole/vtcon0/bind
nvidia-xconfig --query-gpu-info > /dev/null 2>&1
echo "efi-framebuffer.0" > /sys/bus/platform/drivers/efi-framebuffer/bind
## Reload the Window manager
#sudo -H -u yu bash -c 'DISPLAY=:0 nohup startx &'
## If libvirtd was stopped then stop it
[[ $LIBVIRTD == "STOPPED" ]] && systemctl stop libvirtd
## Restore ulimit
ulimit -l $ULIMIT
# User.
USER=yu
# Path to VBIOS, IMG, Windows ISO, Virtio iso, ...
IMAGES=/home/$USER/vm
# IOMMU groups for passed devices.
IOMMU_GPU=06:00.0
IOMMU_GPU_AUDIO=06:00.1
IOMMU_USB=07:00.3
# Virsh devices, only needed if you use virsh.
VIRSH_GPU=pci_0000_06_00_0
VIRSH_GPU_AUDIO=pci_0000_06_00_1
VIRSH_USB=pci_0000_07_00_3
# PCI BUS ID for binding/unbinding devices.
videoid="10de 1184"
audioid="10de 0e0a"
usbid="1022 145f"
videobusid="0000:06:00.0"
audiobusid="0000:06:00.1"
usbbusid="0000:07:00.3"
# Images needed for QEMU.
VBIOS=$IMAGES/GK104_80.04.C3.00.0F-MODED.rom
IMG=$IMAGES/windows.raw
VIRTIO=$IMAGES/virtio.iso
ISO=$IMAGES/win10.iso
HDD=/dev/sdc
OVMF=/usr/share/ovmf/x64/OVMF_CODE.fd
# QEMU options
RAM=12G
CORES=12
# To run QEMU as user you need to allow more RAM to be locked by an user.
ULIMIT=$(ulimit -a | grep "max locked memory" | awk '{print $6}')
## Network stuff
TAP_INTERFACE=tap0
DNSMASQ_CONF=dnsmasq.conf
SMB_CONF=smb.conf
# User.
USER=yu
# Path to VBIOS, IMG, Mac OS ISO, Virtio iso, Clover, OVMF, ...
IMAGES=/home/$USER/vm
# IOMMU groups for passed devices.
IOMMU_GPU=06:00.0
IOMMU_GPU_AUDIO=06:00.1
IOMMU_USB=07:00.3
# Virsh devices, only needed if you use virsh.
VIRSH_GPU=pci_0000_06_00_0
VIRSH_GPU_AUDIO=pci_0000_06_00_1
VIRSH_USB=pci_0000_07_00_3
# PCI BUS ID for binding/unbinding devices.
videoid="10de 1184"
audioid="10de 0e0a"
usbid="1022 145f"
videobusid="0000:06:00.0"
audiobusid="0000:06:00.1"
usbbusid="0000:07:00.3"
# Images needed for QEMU.
VBIOS=$IMAGES/GK104_80.04.C3.00.0F-MODED.rom
IMG=$IMAGES/mac-hs.raw,id=disk,format=raw,if=none
IMG_MOJAVE=/mnt/wd/vm/hdd/macos-mojave.raw
CLOVER=$IMAGES/Clover-1080.qcow2
CLOVER_MOJAVE=/mnt/wd/vm/macos/clover-mojave.qcow2
ISO=$IMAGES/HighSierra-10.13.6-qemu.iso
OVMF=$IMAGES/OSX-KVM/OVMF_CODE.fd
OVMF_VARS=$IMAGES/OSX-KVM/OVMF_VARS.fd
# QEMU options
MY_OPTIONS="+aes,+xsave,+avx,+xsaveopt,avx2,+smep"
MOJAVE_OPTIONS="+pcid,+ssse3,+sse4.2,+popcnt,+aes,+xsave,+avx,+xsaveopt,avx2,+smep,check"
RAM=12G
CORES=4
# To run QEMU as user you need to allow more RAM to be locked by an user.
ULIMIT=$(ulimit -a | grep "max locked memory" | awk '{print $6}')
## Network stuff
TAP_INTERFACE=tap0
DNSMASQ_CONF=dnsmasq.conf
SMB_CONF=smb.conf
#!/bin/bash
# Check if the script is executed as root
if [ "$EUID" -ne 0 ]
then echo "Please run as root"
exit 1
fi
# END Check if you are sudo
# Variables
VBIOS=/root/vBIOS.rom
videobusid="0000:06:00.0"
# END Variables
_start()
{
# Memory lock limit
## Kill X and related
systemctl stop lightdm > /dev/null 2>&1
killall i3 > /dev/null 2>&1
sleep 2
# Kill the console to free the GPU
echo 0 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 0 > /sys/class/vtconsole/vtcon1/bind
sleep 1
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind
sleep 1
# Unload the Kernel Modules that use the GPU
modprobe -r nvidia_drm
sleep 1
modprobe -r nvidia_modeset
sleep 1
modprobe -r nvidia
sleep 1
modprobe -r snd_hda_intel
sleep 2
}
_stop()
{
# Reload the kernel modules. This loads the drivers for the GPU
modprobe snd_hda_intel
sleep 5
modprobe nvidia_drm
sleep 2
modprobe nvidia_modeset
sleep 2
modprobe nvidia
sleep 5
# Re-Bind EFI-Framebuffer and Re-bind to virtual consoles
# [Source] [https://github.com/joeknock90/Single-GPU-Passthrough/blob/master/README.md#vm-stop-script]
echo 1 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 1 > tee /sys/class/vtconsole/vtcon1/bind
sleep 5
# Reload the Display Manager to access X
systemctl start lightdm
sleep 5
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind
sleep 1
}
extract_vbios()
{
echo 1 > /sys/bus/pci/devices/$videobusid/rom
cat /sys/bus/pci/devices/$videobusid/rom > $VBIOS
echo 0 > /sys/bus/pci/devices/$videobusid/rom
}
_start
extract_vbios
_stop
exit
#!/bin/bash
# Check if the script is executed as root
if [ "$EUID" -ne 0 ]
then echo "Please run as root"
exit 1
fi
# END Check if you are sudo
# Variables
VBIOS=/root/vBIOS.rom
NVFLASH=/root/nvflash_linux
videobusid="0000:06:00.0"
# END Variables
_start()
{
# Memory lock limit
## Kill X and related
systemctl stop lightdm > /dev/null 2>&1
killall i3 > /dev/null 2>&1
sleep 2
# Kill the console to free the GPU
echo 0 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 0 > /sys/class/vtconsole/vtcon1/bind
sleep 1
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind
sleep 1
# Unload the Kernel Modules that use the GPU
modprobe -r nvidia_drm
sleep 1
modprobe -r nvidia_modeset
sleep 1
modprobe -r nvidia
sleep 1
modprobe -r snd_hda_intel
sleep 2
}
_stop()
{
# Reload the kernel modules. This loads the drivers for the GPU
modprobe snd_hda_intel
sleep 5
modprobe nvidia_drm
sleep 2
modprobe nvidia_modeset
sleep 2
modprobe nvidia
sleep 5
# Re-Bind EFI-Framebuffer and Re-bind to virtual consoles
# [Source] [https://github.com/joeknock90/Single-GPU-Passthrough/blob/master/README.md#vm-stop-script]
echo 1 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 1 > tee /sys/class/vtconsole/vtcon1/bind
sleep 5
# Reload the Display Manager to access X
systemctl start lightdm
sleep 5
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind
sleep 1
}
extract_vbios()
{
$NVFLASH --save $VBIOS
}
_start
extract_vbios
_stop
exit
#!/bin/bash
## Outputs the IOMMU groups and the devices inside them
## Source [https://wiki.installgentoo.com/index.php/PCI_passthrough#Step_2:_Find_out_your_IOMMU_groups]
for iommu_group in \
$(find /sys/kernel/iommu_groups/ -maxdepth 1 -mindepth 1 -type d); do \
echo "IOMMU group $(basename "$iommu_group")"; \
for device in $(ls -1 "$iommu_group"/devices/); do \
echo -n $'\t'; lspci -nns "$device"; \
done; \
done
#!/bin/bash
# Check if the script is executed as root
if [ "$EUID" -ne 0 ]
then echo "Please run as root"
exit 1
fi
# END Check if you are sudo
source config-macos
# Memory lock limit
if [ $(ulimit -a | grep "max locked memory" | awk '{print $6}') != 12884900 ]; then
ulimit -l 12884900
fi
## Kill X and related
systemctl stop lightdm > /dev/null 2>&1
killall i3 > /dev/null 2>&1
sleep 2
# Kill the console to free the GPU
echo 0 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 0 > /sys/class/vtconsole/vtcon1/bind
sleep 1
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind
sleep 1
# Unload the Kernel Modules that use the GPU
modprobe -r nvidia_drm
sleep 1
modprobe -r nvidia_modeset
sleep 1
modprobe -r nvidia
sleep 1
modprobe -r snd_hda_intel
sleep 2
# Load the kernel module
modprobe vfio
sleep 1
modprobe vfio_iommu_type1
sleep 1
modprobe vfio-pci
sleep 1
# Detach the GPU from drivers and attach to vfio. Also the usb.
echo $videoid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $videobusid > /sys/bus/pci/devices/$videobusid/driver/unbind
sleep 1
echo $videobusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $videoid > /sys/bus/pci/drivers/vfio-pci/remove_id
sleep 1
echo $audioid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $audiobusid > /sys/bus/pci/devices/$audiobusid/driver/unbind
sleep 1
echo $audiobusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $audioid > /sys/bus/pci/drivers/vfio-pci/remove_id
sleep 1
echo $usbid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
sleep 1
echo $usbbusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $usbid > /sys/bus/pci/drivers/vfio-pci/remove_id
sleep 1
# QEMU (VM) command
qemu-system-x86_64 -runas $USER -enable-kvm \
-nographic -vga none -parallel none -serial none \
-m $RAM \
-cpu Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,$MY_OPTIONS\
-machine pc-q35-2.9 \
-smp $CORES,cores=$CORES \
-device vfio-pci,host=$IOMMU_GPU,multifunction=on,x-vga=on,romfile=$VBIOS \
-device vfio-pci,host=$IOMMU_GPU_AUDIO \
-device vfio-pci,host=$IOMMU_USB \
-usb -device usb-kbd -device usb-tablet \
-device nec-usb-xhci,id=xhci \
-netdev user,id=net0 \
-device e1000-82545em,netdev=net0,id=net0,mac=52:54:00:c9:18:27 \
-device isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc" \
-drive if=pflash,format=raw,readonly,file=$OVMF \
-drive if=pflash,format=raw,file=$OVMF_VARS \
-smbios type=2 \
-device ide-drive,bus=ide.2,drive=Clover \
-drive id=Clover,if=none,snapshot=on,format=qcow2,file=$CLOVER \
-device ide-drive,bus=ide.1,drive=MacHDD \
-drive id=MacHDD,if=none,file=$IMG,format=raw \
-device ide-drive,bus=ide.0,drive=MacDVD \
-drive id=MacDVD,if=none,snapshot=on,media=cdrom,file=$ISO > /dev/null 2>&1 &
# END QEMU (VM) command
# Wait for QEMU to finish before continue
wait
sleep 5
# Unload the vfio module. I am lazy, this leaves the GPU without drivers
modprobe -r vfio-pci
sleep 2
modprobe -r vfio_iommu_type1
sleep 2
modprobe -r vfio
sleep 2
# Reload the kernel modules. This loads the drivers for the GPU
modprobe snd_hda_intel
sleep 5
modprobe nvidia_drm
sleep 2
modprobe nvidia_modeset
sleep 2
modprobe nvidia
sleep 5
# Bind the usb
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
echo $usbbusid > /sys/bus/pci/drivers/xhci_hcd/bind
sleep 10
# Re-Bind EFI-Framebuffer and Re-bind to virtual consoles
echo 1 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 1 > tee /sys/class/vtconsole/vtcon1/bind
sleep 5
# Reload the Display Manager to access X
systemctl start lightdm
sleep 5
# Restore the Frame Buffer
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind
sleep 1
# Restore ulimit
ulimit -l $ULIMIT
#!/bin/bash
# Check if the script is executed as root
if [ "$EUID" -ne 0 ]
then echo "Please run as root"
exit 1
fi
# END Check if you are sudo
source config-macos
# Memory lock limit
if [ $(ulimit -a | grep "max locked memory" | awk '{print $6}') != 12884900 ]; then
ulimit -l 12884900
fi
## Kill X and related
systemctl stop lightdm > /dev/null 2>&1
killall i3 > /dev/null 2>&1
sleep 2
# Kill the console to free the GPU
echo 0 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 0 > /sys/class/vtconsole/vtcon1/bind
sleep 1
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind
sleep 1
# Unload the Kernel Modules that use the GPU
modprobe -r nvidia_drm
sleep 1
modprobe -r nvidia_modeset
sleep 1
modprobe -r nvidia
sleep 1
modprobe -r snd_hda_intel
sleep 2
# Load the kernel module
modprobe vfio
sleep 1
modprobe vfio_iommu_type1
sleep 1
modprobe vfio-pci
sleep 1
# Detach the GPU from drivers and attach to vfio. Also the usb.
echo $videoid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $videobusid > /sys/bus/pci/devices/$videobusid/driver/unbind
sleep 1
echo $videobusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $videoid > /sys/bus/pci/drivers/vfio-pci/remove_id
sleep 1
echo $audioid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $audiobusid > /sys/bus/pci/devices/$audiobusid/driver/unbind
sleep 1
echo $audiobusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $audioid > /sys/bus/pci/drivers/vfio-pci/remove_id
sleep 1
echo $usbid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
sleep 1
echo $usbbusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $usbid > /sys/bus/pci/drivers/vfio-pci/remove_id
sleep 1
qemu-system-x86_64 -runas $USER -enable-kvm \
-nographic -vga none -parallel none -serial none \
-m $RAM \
-cpu Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,$MY_OPTIONS\
-machine pc-q35-2.9 \
-smp $CORES,cores=$CORES \
-device vfio-pci,host=$IOMMU_GPU,multifunction=on,x-vga=on,romfile=$VBIOS \
-device vfio-pci,host=$IOMMU_GPU_AUDIO \
-device vfio-pci,host=$IOMMU_USB \
-usb -device usb-kbd -device usb-tablet \
-device nec-usb-xhci,id=xhci \
-netdev user,id=net0 \
-device e1000-82545em,netdev=net0,id=net0,mac=52:54:00:c9:18:27 \
-device isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc" \
-drive if=pflash,format=raw,readonly,file=$OVMF \
-drive if=pflash,format=raw,file=$OVMF_VARS \
-smbios type=2 \
-device ide-drive,bus=ide.2,drive=Clover \
-drive id=Clover,if=none,snapshot=on,format=qcow2,file=$CLOVER \
-device ide-drive,bus=ide.1,drive=MacHDD \
-drive id=MacHDD,if=none,file=$IMG,format=raw > /dev/null 2>&1 &
# END QEMU (VM) command
# Wait for QEMU to finish before continue
wait
sleep 5
# Unload the vfio module. I am lazy, this leaves the GPU without drivers
modprobe -r vfio-pci
sleep 2
modprobe -r vfio_iommu_type1
sleep 2
modprobe -r vfio
sleep 2
# Reload the kernel modules. This loads the drivers for the GPU
modprobe snd_hda_intel
sleep 5
modprobe nvidia_drm
sleep 2
modprobe nvidia_modeset
sleep 2
modprobe nvidia
sleep 5
# Bind the usb
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
echo $usbbusid > /sys/bus/pci/drivers/xhci_hcd/bind
sleep 10
# Re-Bind EFI-Framebuffer and Re-bind to virtual consoles
echo 1 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 1 > tee /sys/class/vtconsole/vtcon1/bind
sleep 5
# Reload the Display Manager to access X
systemctl start lightdm
sleep 5
# Restore the Frame Buffer
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind
sleep 1
# Restore ulimit
ulimit -l $ULIMIT
#!/bin/bash
## Edit this comand before using it
## Required software
## DNSmasq
## samba
## iproute2
## qemu
## ovmf
## Maybe I am missing something else
## For fucking Windows to be able to install stuff on a network drive
## https://community.spiceworks.com/topic/366976-trying-to-install-a-program-to-a-network-drive
## All you need to do is run Registry Editor (regedit.exe), locate the key
## HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Policies/System, and create a new DWORD entry with the name
## EnableLinkedConnections and value 1:
## Check if the script is being run as root
[[ "$EUID" -ne 0 ]] && echo "Please run as root" && exit 1
## Load the config file.
source config-macos
## Memory lock limit.
## By default, at least in Arch, a user can not lock so much memory, so you have to allow it plus 10 just to be sure.
## The `+100000` is just because it ay use a little shade over 12G and that is 100M
## If you get an error that QEMU can't allocate memory just do `ulimit -l unlimited` as root. Revert it after you shutdown the VM.
if [ $(ulimit -a | grep "max locked memory" | awk '{print $6}') != $(( $(echo $RAM | tr -d 'G')*1048576+100000 )) ]; then
ulimit -l $(( $(echo $RAM | tr -d 'G')*1048576+100000 ))
fi
## Kill X and related. You can change `lightdm` and `i3` for whatever you use.
## Make shure you don't have anything important open because this will kill the X session.
## You can also just do "killall xorg" or "killall xinit"
systemctl stop lightdm > /dev/null 2>&1
killall i3 > /dev/null 2>&1
# killall xorg
# killall xinit
sleep 2
## START ## Network stuff
## THIS REQUIRES A FIREWALL BACKEND LIKE IPTABLES. If there is nothing routing the traffic this will be useless.
## Also open ports 53 tcp/udp 67,68 tcp/udp and icmp in ther firewall rules for the $TAP_INTERFACE
tap_interface(){
tap_stop(){
if [[ $(ip tuntap list | grep $1) ]]
then
sudo ip link set dev $1 down
sudo ip tuntap del mode tap name tap0
fi
}
tap_start(){
if [[ ! $(ip tuntap list | grep $1) ]]
then
#sudo tunctl -u $VM_USER
sudo ip tuntap add mode tap user $USER name tap0
sudo ip addr add dev $1 10.10.10.1/24
sudo ip link set dev $1 up
fi
}
if [ $1 == start ]
then
tap_start $TAP_INTERFACE
elif [ $1 == stop ]
then
tap_stop $TAP_INTERFACE
fi
}
dhcp_server(){
dhcp_stop(){
[[ -f /var/run/dnsmasq.pid ]] && sudo kill -15 $(cat /var/run/dnsmasq.pid) && sudo rm /var/run/dnsmasq.pid
}
dhcp_start(){
[[ -f /var/run/dnsmasq.pid ]] || dnsmasq --conf-file=$DNSMASQ_CONF
}
if [ $1 == start ]
then
dhcp_start
elif [ $1 == stop ]
then
dhcp_stop
fi
}
## Samba server controls
samba_server(){
samba_stop(){
[[ -f /var/run/smbd.pid ]] && echo "Stopping samba" && sudo kill -15 $(cat /var/run/smbd.pid) || echo "Samba was already stopped"
}
samba_start(){
[[ -f /var/run/smbd.pid ]] && echo "Samba was already started" || echo "Starting samba" && sudo smbd --configfile=$SMB_CONF
}
if [ $1 == start ]
then
samba_start
elif [ $1 == stop ]
then
samba_stop
fi
}
## Start the network
tap_interface start
dhcp_server start
samba_server start
## For samba to work you must run as root `smbpasswd -a <USER>`
## END ## Network stuff
## Kill the console to free the GPU.
## The console, by default, is attached to the boot GPU, in this case there is only one and we need to left it unused to remove it from the system.
echo 0 > /sys/class/vtconsole/vtcon0/bind
echo 0 > /sys/class/vtconsole/vtcon1/bind
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind
## Unload the Kernel Modules that use the GPU
## You may need to unload more or other kernel modules (drivers) for your specific GPU.
## This command allows you to see which modules depend uppon others. Change nvidia for noveau if you are using the OSS drivers or for the AMD equivalent.
# lsmod | grep nvidia
## The left column are the modules and the right are the modules that are using that module. The modules should be unloaded only if there is no other modules using it; if there are, then unload them first.
# nvidia_drm 49152 7
# nvidia_modeset 1044480 23 nvidia_drm
# nvidia 16605184 1038 nvidia_modeset
modprobe -r nvidia_drm
modprobe -r nvidia_modeset
modprobe -r nvidia
## Also unload the audio modules, may be more than one, because they use the audio device in the GPU
modprobe -r snd_hda_intel
## Load the kernel module related to vfio
modprobe vfio
modprobe vfio_iommu_type1
modprobe vfio-pci
## Detach the GPU from drivers and attach to vfio. Also the usb.
## This may not be the right way, but it works for me and there is no symptoms of anything not working.
echo $videoid > /sys/bus/pci/drivers/vfio-pci/new_id
echo $videobusid > /sys/bus/pci/devices/$videobusid/driver/unbind
echo $videobusid > /sys/bus/pci/drivers/vfio-pci/bind
echo $videoid > /sys/bus/pci/drivers/vfio-pci/remove_id
echo $audioid > /sys/bus/pci/drivers/vfio-pci/new_id
echo $audiobusid > /sys/bus/pci/devices/$audiobusid/driver/unbind
echo $audiobusid > /sys/bus/pci/drivers/vfio-pci/bind
echo $audioid > /sys/bus/pci/drivers/vfio-pci/remove_id
echo $usbid > /sys/bus/pci/drivers/vfio-pci/new_id
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
echo $usbbusid > /sys/bus/pci/drivers/vfio-pci/bind
echo $usbid > /sys/bus/pci/drivers/vfio-pci/remove_id
## QEMU (VM) command
#qemu-system-x86_64 -runas $USER -enable-kvm \
# -nographic -vga none -parallel none -serial none \
# -m $RAM \
# -cpu Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,$MY_OPTIONS\
# -machine pc-q35-2.9 \
# -smp $CORES,cores=$CORES \
# -device vfio-pci,host=$IOMMU_GPU,multifunction=on,x-vga=on,romfile=$VBIOS \
# -device vfio-pci,host=$IOMMU_GPU_AUDIO \
# -device vfio-pci,host=$IOMMU_USB \
# -usb -device usb-kbd -device usb-tablet \
# -device nec-usb-xhci,id=xhci \
# -device virtio-net-pci,netdev=net0 \
# -netdev tap,id=net0,ifname=$TAP_INTERFACE,script=no,downscript=no,vhost=on \
# -device #isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerI#nc" \
# -drive if=pflash,format=raw,readonly,file=$OVMF \
# -drive if=pflash,format=raw,file=$OVMF_VARS \
# -smbios type=2 \
# -device ide-drive,bus=ide.2,drive=Clover \
# -drive id=Clover,if=none,snapshot=on,format=qcow2,file=$CLOVER \
# -device ide-drive,bus=ide.1,drive=MacHDD \
# -drive id=MacHDD,if=none,file=$IMG,format=raw > /dev/null 2>&1 &
qemu-system-x86_64 -runas $USER -enable-kvm \
-nographic -vga none -parallel none -serial none \
-m $RAM \
-cpu Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,$MOJAVE_OPTIONS\
-machine pc-q35-2.11 \
-smp $CORES,sockets=1,cores=$(($CORES/2)),threads=2 \
-device vfio-pci,host=$IOMMU_GPU,multifunction=on,x-vga=on,romfile=$VBIOS \
-device vfio-pci,host=$IOMMU_GPU_AUDIO \
-device vfio-pci,host=$IOMMU_USB \
-usb -device usb-kbd -device usb-tablet \
-device nec-usb-xhci,id=xhci \
-netdev tap,id=net0,ifname=$TAP_INTERFACE,script=no,downscript=no \
-device e1000-82545em,netdev=net0,id=net0,mac=52:54:00:c9:18:27 \
-device isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc" \
-drive if=pflash,format=raw,readonly,file=$OVMF \
-drive if=pflash,format=raw,file=$OVMF_VARS \
-smbios type=2 \
-device ide-drive,bus=ide.2,drive=Clover \
-drive id=Clover,if=none,snapshot=on,format=qcow2,file=$CLOVER_MOJAVE \
-device ide-drive,bus=ide.1,drive=MacHDD \
-drive id=MacHDD,if=none,file=$IMG_MOJAVE,format=raw,id=disk > /dev/null 2>&1 &
## Network
## virtio-net-pci does not work, obviously
## e1000 does not work
## rtl8139 does not work
## e1000-82545em
## Wait for QEMU to finish before continuing
wait
sleep 1
## Stop the network
tap_interface stop
dhcp_server stop
samba_server stop
## Unload the vfio module. This leaves the GPU without drivers.
modprobe -r vfio-pci
modprobe -r vfio_iommu_type1
modprobe -r vfio
## Reload the kernel modules that previously were unloaded. Load them inversed as you unloaded. The las unloaded should be the first to be loaded and viceversa. THIS MAY NOT BE THE CASE FOR YOU. This SHOULD load the drivers for the GPU.
modprobe snd_hda_intel
modprobe nvidia_drm
modprobe nvidia_modeset
modprobe nvidia
## Bind the USB. If you passed the USB controller to the VM.
#echo $usbid > /sys/bus/pci/drivers/xhci_hcd/new_id
## This does not exist, so...
#echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
echo $usbbusid > /sys/bus/pci/drivers/xhci_hcd/bind
#echo $usbid > /sys/bus/pci/drivers/xhci_hcd/remove_id
#ls -la /sys/bus/pci/devices/$usbbusid/
## Rebind the EFI-Framebuffer and Rebind to virtual consoles
## [Source] [https://github.com/joeknock90/Single-GPU-Passthrough/blob/master/README.md#vm-stop-script]
echo 1 > /sys/class/vtconsole/vtcon0/bind
sleep 1
## This just creates a file called `tee`
#echo 1 > tee /sys/class/vtconsole/vtcon1/bind
sleep 1
## Reload the Display Manager to access X. You can also run startx, though I don't know how to do it and `startx` will probably not work.
systemctl start lightdm
sleep 2
## Restore the Frame Buffer.
## I tried MANY different ways and timings of doing this, and this is the only one that works, FOR ME.
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind
sleep 1
## Restore ulimit to the previous state.
ulimit -l $ULIMIT
#!/bin/bash
# Requires iproute2, dnsmasq, and firewall [iptables, shorewall, ...]
# Check if the script is executed as root
if [ "$EUID" -ne 0 ]
then echo "Please run as root"
exit 1
fi
# END Check if you are sudo
# Variables
NIC=qemubr0
IP=10.10.10.1/24
start_network()
{
ip link add name $NIC type bridge
ip link set $NIC up
ip address add $IP dev $NIC
}
stop_network()
{
ip link set $NIC down
ip link delete $NIC type bridge
}
start_dnsmasq()
{
systemctl start dnsmasq
}
stop_dnsmasq()
{
systemctl stop dnsmasq
}
_help()
{
echo "Usage: network.sh [OPTIONS]"
echo " start - start the network"
echo " stop - stop the network"
}
_start()
{
if [ "$1" = "start" ]; then
start_network
start_dnsmasq
exit
elif [ "$1" = "stop" ]; then
stop_dnsmasq
stop_network
exit
else
_help
exit 1
fi
}
if [[ $1 ]]; then
_start $1
else
_help
exit 1
fi
[Unit]
Description=QEMU virtual machine (%i)
[Service]
#Type=forking
#PIDFile=/run/qemu_%i.pid
EnvironmentFile=/home/yu/scripts/config-macos
ExecStart=/home/yu/scripts/%i.sh
#TimeoutStopSec=1m
[Install]
WantedBy=multi-user.target
[Unit]
Description=QEMU virtual machine (%i)
[Service]
#Type=forking
#PIDFile=/run/qemu_%i.pid
EnvironmentFile=/home/yu/scripts/config
ExecStart=/home/yu/scripts/%i.sh
#TimeoutStopSec=1m
[Install]
WantedBy=multi-user.target
#!/bin/bash
# Check if the script is executed as root
if [ "$EUID" -ne 0 ]
then echo "Please run as root"
exit 1
fi
# END Check if you are sudo
source config.sh
_start()
{
# Memory lock limit
if [ $(ulimit -a | grep "max locked memory" | awk '{print $6}') != 12884900 ]; then
ulimit -l 12884900
fi
## Kill X and related
systemctl stop lightdm > /dev/null 2>&1
killall i3 > /dev/null 2>&1
sleep 2
# Kill the console to free the GPU
echo 0 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 0 > /sys/class/vtconsole/vtcon1/bind
sleep 1
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind
sleep 1
# Unload the Kernel Modules that use the GPU
modprobe -r nvidia_drm
sleep 1
modprobe -r nvidia_modeset
sleep 1
modprobe -r nvidia
sleep 1
modprobe -r snd_hda_intel
sleep 2
# Load the kernel module
modprobe vfio
sleep 1
modprobe vfio_iommu_type1
sleep 1
modprobe vfio-pci
sleep 1
# Detach the GPU from drivers and attach to vfio. Also the usb.
echo $videoid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $videobusid > /sys/bus/pci/devices/$videobusid/driver/unbind
sleep 1
echo $videobusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $videoid > /sys/bus/pci/drivers/vfio-pci/remove_id
sleep 1
echo $audioid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $audiobusid > /sys/bus/pci/devices/$audiobusid/driver/unbind
sleep 1
echo $audiobusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $audioid > /sys/bus/pci/drivers/vfio-pci/remove_id
sleep 1
echo $usbid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
sleep 1
echo $usbbusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $usbid > /sys/bus/pci/drivers/vfio-pci/remove_id
#ls -la /sys/bus/pci/devices/$usbbusid/
sleep 1
}
_stop()
{
# Wait for QEMU to finish before continue
wait
sleep 5
# Unload the vfio module. I am lazy, this leaves the GPU without drivers
modprobe -r vfio-pci
sleep 2
modprobe -r vfio_iommu_type1
sleep 2
modprobe -r vfio
sleep 2
# Reload the kernel modules. This loads the drivers for the GPU
modprobe snd_hda_intel
sleep 5
modprobe nvidia_drm
sleep 2
modprobe nvidia_modeset
sleep 2
modprobe nvidia
sleep 5
# Bind the usb
#echo $usbid > /sys/bus/pci/drivers/xhci_hcd/new_id
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
echo $usbbusid > /sys/bus/pci/drivers/xhci_hcd/bind
sleep 10
#echo $usbid > /sys/bus/pci/drivers/xhci_hcd/remove_id
#ls -la /sys/bus/pci/devices/$usbbusid/
# Re-Bind EFI-Framebuffer and Re-bind to virtual consoles
# [Source] [https://github.com/joeknock90/Single-GPU-Passthrough/blob/master/README.md#vm-stop-script]
echo 1 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 1 > tee /sys/class/vtconsole/vtcon1/bind
sleep 5
#echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind
#sleep 1
# Reload the Display Manager to access X
systemctl start lightdm
sleep 5
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind
sleep 1
# Restore ulimit
ulimit -l $ULIMIT
}
_help()
{
echo "Usage: test-qemu.sh [OPTIONS]"
echo " start"
echo " stop"
}
_do()
{
if [ "$1" = "start" ]; then
_start
exit
elif [ "$1" = "stop" ]; then
_stop
exit
else
_help
exit 1
fi
}
if [[ $1 ]]; then
_do $1
else
_help
exit 1
fi
#!/bin/bash
# Check if the script is executed as root
if [ "$EUID" -ne 0 ]
then echo "Please run as root"
exit 1
fi
# END Check if you are sudo
source config
# Memory lock limit
if [ $(ulimit -a | grep "max locked memory" | awk '{print $6}') != $(( $(echo $RAM | tr -d 'G')*1048576+10 )) ]; then
ulimit -l $(( $(echo $RAM | tr -d 'G')*1048576+10 ))
fi
## Kill X and related
systemctl stop lightdm > /dev/null 2>&1
killall i3 > /dev/null 2>&1
sleep 2
# Kill the console to free the GPU
echo 0 > /sys/class/vtconsole/vtcon0/bind
echo 0 > /sys/class/vtconsole/vtcon1/bind
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind
# Unload the Kernel Modules that use the GPU
modprobe -r nvidia_drm
modprobe -r nvidia_modeset
modprobe -r nvidia
modprobe -r snd_hda_intel
# Load the kernel module
modprobe vfio
modprobe vfio_iommu_type1
modprobe vfio-pci
# Detach the GPU from drivers and attach to vfio. Also the usb.
echo $videoid > /sys/bus/pci/drivers/vfio-pci/new_id
echo $videobusid > /sys/bus/pci/devices/$videobusid/driver/unbind
echo $videobusid > /sys/bus/pci/drivers/vfio-pci/bind
echo $videoid > /sys/bus/pci/drivers/vfio-pci/remove_id
echo $audioid > /sys/bus/pci/drivers/vfio-pci/new_id
echo $audiobusid > /sys/bus/pci/devices/$audiobusid/driver/unbind
echo $audiobusid > /sys/bus/pci/drivers/vfio-pci/bind
echo $audioid > /sys/bus/pci/drivers/vfio-pci/remove_id
echo $usbid > /sys/bus/pci/drivers/vfio-pci/new_id
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
echo $usbbusid > /sys/bus/pci/drivers/vfio-pci/bind
echo $usbid > /sys/bus/pci/drivers/vfio-pci/remove_id
# QEMU (VM) command
qemu-system-x86_64 -runas $USER -enable-kvm \
-nographic -vga none -parallel none -serial none \
-enable-kvm \
-m $RAM \
-cpu host,kvm=off,hv_relaxed,hv_spinlocks=0x1fff,hv_time,hv_vapic,hv_vendor_id=0xDEADBEEFFF \
-rtc clock=host,base=localtime \
-smp $CORES,sockets=1,cores=$CORES,threads=0 \
-device vfio-pci,host=$IOMMU_GPU,multifunction=on,x-vga=on,romfile=$VBIOS \
-device vfio-pci,host=$IOMMU_GPU_AUDIO \
-device vfio-pci,host=$IOMMU_USB \
-device virtio-net-pci,netdev=n1 \
-netdev user,id=n1 \
-drive if=pflash,format=raw,readonly,file=$OVMF \
-drive media=cdrom,file=$ISO,id=virtiocd1,if=none \
-device ide-cd,bus=ide.1,drive=virtiocd1 \
-drive media=cdrom,file=$VIRTIO,id=virtiocd2,if=none \
-device ide-cd,bus=ide.1,drive=virtiocd2 \
-device virtio-scsi-pci,id=scsi0 \
-device scsi-hd,bus=scsi0.0,drive=rootfs \
-drive id=rootfs,file=$HDD,media=disk,format=raw,if=none > /dev/null 2>&1 &
# END QEMU (VM) command
# Wait for QEMU to finish before continue
wait
sleep 1
# Unload the vfio module. I am lazy, this leaves the GPU without drivers
modprobe -r vfio-pci
modprobe -r vfio_iommu_type1
modprobe -r vfio
# Reload the kernel modules. This loads the drivers for the GPU
modprobe snd_hda_intel
modprobe nvidia_drm
modprobe nvidia_modeset
modprobe nvidia
# Bind the usb
#echo $usbid > /sys/bus/pci/drivers/xhci_hcd/new_id
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
echo $usbbusid > /sys/bus/pci/drivers/xhci_hcd/bind
#echo $usbid > /sys/bus/pci/drivers/xhci_hcd/remove_id
#ls -la /sys/bus/pci/devices/$usbbusid/
# Re-Bind EFI-Framebuffer and Re-bind to virtual consoles
# [Source] [https://github.com/joeknock90/Single-GPU-Passthrough/blob/master/README.md#vm-stop-script]
echo 1 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 1 > tee /sys/class/vtconsole/vtcon1/bind
sleep 1
# Reload the Display Manager to access X
systemctl start lightdm
sleep 2