### Working with security related features on macOS
macOS provides a feature know as SIP _System Integrity Protection_ which basically prevents certain files and settings from being modified.
To disable **SIP** on a modern version of macOS
1. Boot into _recovery mode_ <kbd>⌘</kbd> + <kbd>r</kbd>
2. Enter the below command to disable **SIP**
```shell
csrutil disable; reboot
```
To check the status of **SIP**
```shell
csrutil status
```
<aid="useful-links-macos"></a>
### Useful Links macOS [🔝](#contents)
For a comprehensive list of changes of macOS between releases [see](https://developer.apple.com/library/content/releasenotes/MacOSX/WhatsNewInOSX/WhatsNewInOSX.html)
To download macOS High Sierra officially using **App Store.app** via macOS
#### Setting up a NFS Server / Share in macOS High Sierra [🔝](#contents)
By default macOS has the NFS daemon disabled by default, but it can be enabled.
1. Make certain that SIP, ie. System Integrity Protection has been disabled.
2. Edit `/System/Library/LaunchDaemons/com.apple.nfsd.plist` to include the `-N` flag, so that non root users can mount **nfs** shares hosted on macOS.
3. Edit `/etc/exports` and include a path to a directory to be shared.
##### Useful Resources for working with nfsd [🔝](#contents)
```shell
man nfs
man nfs.conf
man nfsd
man exports
```
```fish
ifconfig en0
```
#### macOS > nfs > Useful Links
- [cyberciti.biz > macOS working with nfs server & client](https://www.cyberciti.biz/faq/apple-mac-osx-nfs-mount-command-tutorial/)
<a id="ncurses"></a>
### ncures [🔝](#contents)
To print / show the current version of ncures installed on a system, ie. macOS
```shell
cat /usr/include/ncurses.h | grep -i "version"
```
<a id="working-with-openssh"></a>
### Working with OpenSSH [🔝](#contents)
When troubleshooting OpenSSH, and making changes to `~/.ssh/config` for particular host entries, neither the OpenSSH server or the current shell need to be restarted, ie. reinvoked as changes made to `~/.ssh/config` are reflected in realtime when using `ssh`.
For an incompetent guide comparing LibreSSL to OpenSSL, see [this](https://github.com/ipatch/re/wiki/ssl-battle)
To set up password less login, ie. public key authentication
> If the `id_rsa.pub` file doesn't exists one can be generated using `ssh-keygen`
An alternative way to copy a public key from one host to another is using **ssh-copy-id**
```shell
ssh-copy-id [USER]@[HOSTNAME/IP].local -p[PORT]
```
> [USER] is the name of the $USER that will contain the public key to allow loggin in of the current $USER
<a id="working-with-ssh-agent"></a>
#### Working with ssh-agent [🔝](#contents)
**ssh-agent** is a utility app that allows the OS / user session to store unencrypted SSH keys in memory.
> The safest place to store unencrypted keys for SSH sessions would be in memory and **NOT** on disk.
<a id="working-with-brewed-openssh"></a>
### Working with brewed OpenSSH [🔝](#contents)
> First off, grab a cup of joe or tea if it is that time of the day for you because this isn't something that your gunna why to skim through, that said, if you have your cup of **covfefe** 🙄 and you're ready to roll 🚗
1. Download and install OpenSSH using homebrew
```shell
brew install openssh
```
> If you prefer to use **LibreSSL** over OpenSSL 🙋 then you're gunna wanna check out _no pun intended there_ the version of OpenSSH [I rolled](https://github.com/ipatch/homebrew-core/blob/master/Formula/openssh.rb) 🚬 for homebrew.
1. _1.1_ **optional**
```shell
brew install openssh --with-libressl
```
> **OpenSSH** does not need to be recompiled when upgrading **LibreSSL**, ie. I tested this when upgrading LibreSSL from 2.7.2 to 2.7.3 and running,
> `ssh -V`
>
> The output from the above command is
```shell
OpenSSH_7.7p1, LibreSSL 2.7.3
```
> However, the `sshd` service will need to be stopped and started because launchd doesn't have a "restart" feature per se on macOS in order for OpenSSH to load the new version of LibreSSL.
```shell
> launchctl unload homebrew.mxcl.sshd
> launchctl load homebrew.mxcl.sshd
```
> If using the Apple provided service of OpenSSH _which happens to be compiled with LibreSSL as well ...hmmm 🤔 wonder why that is_ then your first gunna wanna stop that launchd service, and unload it from launchd.
Recent version of macOS provide a nice little feature known as [SIP](https://en.wikipedia.org/wiki/System_Integrity_Protection) that supposedly prevents third parties 🙋 from modfying certain files and services. _thanks Obama_
> Personally, I disabled SIP on my local box a while ago because I like to pretend that I know what I'm doing when using "apps" such as DTrace.
- To stop the Apple provided OpenSSH daemon if it is currently running, and prevent it from launching in the [future](https://www.youtube.com/watch?v=Vm8S331kUPQ)
...and you shouldn't 🙈 see anything printed to STDOUT if `sshd` isn't running.
- Copy or download my provided `sshd_config` along with my `com.chrisrjones.sshd.plist` from [this repo](https://github.com/ipatch/dotfiles/)
> Personally I keep those two files checked into my dotfiles repo, and symlink them to their respected locations on macOS, so I can preserve a copy of the files, and not have to modify / keep track of two separate files. 👍
- **Cross fingers** 🤞 and if everything goes well you should be able to see your newly built and configured **sshd** server running with
```shell
ps aux | grep -i "sshd"
```
Pat yourself on the back if you made it this far and your server is up and running, and make sure to tip your DevOps engineer.
> because developers need heros too.
<a id="gotchas-working-with-brewed-openssh"></a>
#### Gotchas Working with brewed OpenSSH [🔝](#contents)
> Presently the OpenSSH provided by homebrew compiles with OpenSSL by default when installing 🙄 _boo_
The access list for allowing users who can login remotely using SSH is still maintained via **System Preferences...** > **Sharing** > **Remote Login** and toggling the users by adding their name to the **Only theses users:** list.
> In order to get **password** auth working with OpenSSH Server `UsePAM yes` must be set within the respected `sshd_config` file. Most recent build of OpenSSH 7.x > do not allow running `sshd` as a standard user, in short `sshd` needs to be run as the super user on the system, ie. `root` if on a UNIX based system. To aviod permission issues make sure the respected `ssh_host_{dsa_key,ecdsa_key,ed25519_key,rsa_key} files have a *root/wheel* ownership to avoid
```conf
debug1: setgroups() failed: Operationnotpermitted
```
```shell
launchctl list | grep"chrisrjones"
```
presently does not display the sshd service 🤷♂️
<aid="tshooting-working-with-brewed-openssh"></a>
#### Troubleshooting Working with brewed OpenSSH [🔝](#contents)
> I am [avaible](https://www.patreon.com/ipatch/) for children's birthday parties and bar mitzvahs.
If launchd runs into any errors with the provided plist file, when starting `sshd` it should dump the error messages into `/usr/local/var/log/sshd-error.log`.
If there is an issue with the OpenSSH server itself, ie. a bad configuration setting with the `sshd_config` the server should put the error message into `/usr/local/var/log/sshd.log`
Uncomment the `-d` flag in the above mentioned `.plist` file to start **sshd** in debug mode for more versobe troubleshooting.
> `-d` support the use of `-d` `-d` `-d`, ie. 3 levels of d's. When **sshd** is run in **debug** mode the server will stop running after the first disconnection.
<aid="todos-working-with-brewed-openssh"></a>
#### TODOs Working with brewed OpenSSH [🔝](#contents)
> With your newly aquired knowledge you should be able to setup `ssh-agent`.
- [x] ~~Setup ssh-agent to work with newly built OpenSSH server~~
- [ ] See if it's possible if the homebrewed version of OpenSSH server can play nice 👨 👧 👦 with macOS System Preferences. 🤔
<aid="setup-x11-forwarding-through-ssh"></a>
#### Setting Up X11 Forwarding through SSH [🔝](#contents)
> Native macOS apps won't run on \*nix because their not X11 apps, but rather their cocoa apps.
To allow for X11 forwarding through SSH, edit `sshd_config`
```conf
X11Forwardingyes
```
<aid="connecting-to-x11-server"></a>
##### Connecting to a X11 Server with X11 forwarding enabled [🔝](#contents)
```shell
ssh -Y[user]@[host]
```
<aid="useful-links-openssh"></a>
#### Useful Links OpenSSH [🔝](#contents)
-[coderwall.com _using SSH provided by homebrew_](https://coderwall.com/p/qdwcpg/using-the-latest-ssh-from-homebrew-on-osx)
<aid="working-with-locales-on-macOS"></a>
### Working with Locales on macOS [🔝](#contents)
To print the set locales on macOS
```shell
locale
```
<aid="working-with-users-and-groups"></a>
### macOS > Working with Users and Groups [🔝](#contents)
To list all users on a macOS
```shell
dscl . list /Users
```
To list all groups on a macOS
```shell
dscl . list /groups
```
To create, edit, update, or delete a _group_ on macOS from a CLI
> ⚠️ When managing groups from a CLI **System Preferences.app** will not update the **Users & Groups** pane with the updated data until the **System Preferences.app** is _closed_ and then reopened.
#### Working with Xcode from the command line [🔝](#contents)
macOS 10.13 High Sierra, file tagging using color labels appears to be broken. [see](https://discussions.apple.com/thread/8232839)
To compile an [Objective-C](https://en.wikipedia.org/wiki/Objective-C) file from the CLI on macOS, use **xcrun**
```shell
xcrun clang /path/to/mr-fancy-42.m -c
```
> The above will only create an object file ie. **.o** of the hello world source file wrtiten in ojbc
To compile a Swift source file using the Xcode toolchain from the CLI on macOS, use **xcrun**
```shell
xcrun swiftc -c /path/to/mr-fancy-42.swift
```
To compile a binary that can be run from the CLI using **./path/to/hello-world-objc**
```shell
clang -x objective-c -framework Foundation hello-world.m
```
> The above command will create an intermediate object file within the directory that the source file, ie. `hello-world.m` resides. The **-x** flag explicity tells **clang** the source file is an **Objective-C** source file.
To syntax check a source file that can be compiled with **clang** use the **-fsyntax-only** flag
Custom installed **Preference Panes** on macOS 10.13.x are located within `/Library/PrefrencePanes/`
> Try and avoid putting custom **Preference Panes** within `/System/Library/PreferencePanes` because macOS does not like 3rd party files in that location. The above mentioned directory is `755` with `root` owner and `wheel` group perms by default.
**IMHO** I put my daily driver user `echo (whoami)` in the **wheel** group and set the permissions to `775`, so my `$USER` could create custom `PreferencePanes` if needed.
<aid="working-with-files-via-cli"></a>
### Working with files via CLI [🔝](#contents)
To delete all `.DS_Store` files in the current working directory & and all sub directories recursively
```shell
find .-name'.DS_Store'-delete
```
<aid="tweaking-macos-settings"></a>
### Tweaking macOS settings [🔝](#contents)
> ❗️ **tags** macOS osx os x settings setting customize
To disable app icons from bouncing in the **dock**
> When editing system settings using **defaults write** settings for $USER are stored in /Users/$USER/Library/Preferences as a Apple binary Plist file.
<aid="macos-automating-cli-tasks"></a>
### macOS Automating CLI tasks [🔝](#contents)
The below directories run commands automatically
```conf
/etc/periodic/daily
/etc/periodic/weekly
/etc/periodic/monthly
```
To run the **periodic** scripts in macOS
```shell
periodic daily weekly monthly
```
<aid="macos-automating-tasks-on-sierra"></a>
#### macOS Automating tasks on macOS >= Sierra [🔝](#contents)
To create an ISO to be burnt as a data DVD for PC and MAC
```shell
mkisofs -J-r-o myimage.iso data_dir
```
> `mkisofs` is not native on macOS 10.12.x
To copy the contents of an ISO to a USB drive
```shell
dd if=/path/to/myISO.iso of=/dev/myUSB bs=1m
```
#### Creating a Bootable Windows 10 USB Install Drive for Intel Macs [🔝](#contents)
-**Rufus** can be used to create a bootable USB Windows 10 install disk. Rufus provides options to create a bootable USB for both BIOS and UEFI systems. 👍
The quickest and most painless way of creating a bootable Windows 10 USB drive on macOS is to use **Boot Camp Assistant** provided by Apple.
> 🚨 A bootable Windows 10 USB Install drive created with **Boot Camp Assistant** will only work with a Intel Mac, ie. modern MacBook Pro, and **will not** work with other UEFI laptops! **Etcher.app** will NOT be able to create a bootable Windows 10 USB drive. 😩
To check and see if an Intel Mac is using a 32 or 64 bit UEFI system
```shell
ioreg -l-p IODeviceTree | grep firmware-abi
```
The above command 👆 should either print **EFI64** or **EFI32**
To boot an Intel Mac into Internet Recovery Mode
<kbd>option</kbd>+<kbd>⌘</kbd>+<kbd>R</kbd> or <kbd>shift</kbd>+<kbd>option</kbd>+<kbd>⌘</kbd>+<kbd>R</kbd>
To boot an Intel Mac into **Apple Diagnostic/Hardware Test** mode
1. Turn the computer completely off
2. Hold <kbd>option</kbd>+<kbd>D</kbd> after the startup chime, select the appropiate WIFI network, and then the Mac should download the appropiate **Apple Diagnosit/Hardware Test** for the Mac.
> Apple changed the **Apple Diagnostic/Hardware Test** program some time around 2013, and Macs prior to 2013 user an older blue GUI version of the test.
<aid="working-with-filesystems"></a>
### Working with file systems and disks, ie. storage devices [🔝](#contents)
To securely wipe a partition on a disk, ie. storage medium
A quick way to list the file system, ie. **HFS+** for a particular mount point
```shell
diskutil info -all
```
or
```shell
diskutil list
```
To display / show all file systems accessible to a macOS system
```shell
mount
```
> No flags or arguments are required to display the currently mounted file systems on macOS, and the above command should the mounted disks, ie. a removable USB drive and their respected file systems accessible to macOS.
To unmount a file system on macOS
```shell
diskutil unmountDisk /path/to/disk
```
> The above command will unmount all volumes associated with a disk.
<strong>Ex</strong>
```shell
diskutil unmountDisk /dev/disk2
```
To quick and dirty way to see the type of partition table a disk using on macOS
```shell
diskutil list
```
> The above command should 99% of the time list volume 0 as either **GUID_partition_scheme** or **FDISK_partition_scheme**
<strong>Ex</strong>
```conf
/dev/disk3 (external, physical):
0: GUID_partition_shceme
```
To format a file system on macOS with a specific format
```shell
newfs_*-F /path/to/device
```
To format a USB disk with a MSDOS FAT 16 file system on macOS
<strong>Ex</strong>
```shell
newfs_msdos -v"MR_FANCY_VOLUME"-F 16 /dev/disk2
```
> BSD derivatives of UNIX use **newfs_[FS_TYPE]** to where as GNU derivatives use **mkfs.[FS_TYPE]**
On macOS an alternative way to format a disk is to use **diskutil**
#### Working with FAT-32 file systems [🔝](#contents)
To the best of my knowledge the largest file a FAT-32 file system can store is 4GB.
To mount a FAT[16-32] file system on macOS using **diskutility**
```shell
diskutil mount /path/to/blockDevice
```
<strong>Ex</strong>
```shell
diskutil mount /dev/disk3s2
```
> The above command will mount the file system within the **/Volumes** directory on the local macOS system with the respected volume label as the mount point.
To unmount a volume / partition on macOS with FAT[16-32] file system.
```shell
diskutil umount /path/to/mount/point
```
> The path to the block device can also be specified.
##### Working with large files on a FAT-32 file system [🔝](#contents)
To split a **really** large file into chunks 🤮 that can be copied to a **FAT-32** file system
```shell
rar a -v4G /path/to/mr-fancy-42-universe.rar /path/to/mr-fancy-42-universe.qcow2
```
An alternative to using **rar** is using **GNU Core Utilites** to split the file using the aptly named **split** command.
> When using the **split** command, specifying **-bytes=4G** will create files too large for a FAT32 file system.
###### Transferring large files to a FAT-32 file system [🔝](#contents)
**cp** is great for copying files, however it lacks any sort of progress bar, but there are various bolt-on 🔩 solutions, ie. piping through **pv** or using some form of python, but from my personal expierence **rsync** is the best solution for transferring large files.
> **make** should be run the **grub-build** dir and **NOT** the grub git source project root.
-**Optional** Specify a font to use with GRUB, on macOS most system fonts can be found in **/Library/Fonts**, that said, choose a **.ttf** font for the below command.
> The above command will not succeed on a removable USB drive if **--removable** is omitted from the command. If the above command succeeds the following message will output to STDOUT
```conf
Installationfinished. Noerrorreported
```
##### grub-install alternative [🔝](#contents)
To install grub on a GPT formatted removable USB disk using macOS a couple of steps will need to be performed
- Verify the disk has been formatted with a GPT parititon table, and **not** an MBR partition table.
```shell
diskutil list
```
- set a path to where the EFI partition can be mounted, and you have read/write access to the path.
<strong>Ex</strong>
```shell
mkdir /mnt/usb-gpt-efi
sudo chown-R$USER:wheel /mnt/usb-gpt-efi
```
> If the above directory is created within the `/Volumes` directory then the mount point, ie. directory that holds the mounted file system will be deleted.
- mount the **EFI** partition of the USB disk
```shell
sudo mount -t msdos /dev/disk3s1 /Volumes/usb-gpt-efi
```
- Verify the partition has properly been mounted.
```shell
mount
```
- Install GRUB boot loader to the USB drive to work with a **UEFI** system / boot rom.
#### Working with the bless command on macOS [🔝](#contents)
In order to boot from a removable media, ie. a USB flash drive on macOS a drive or Volume needs to blessed, using the native **bless** command from macOS or using the **hfs-bless** provided by mactel linux.
To see / print the currently **blessed** disk that will boot macOS
```shell
bless -getBoot
```
<aid="working-with-automountd"></a>
#### Working with automountd [🔝](#contents)
**automountd** is a service provided by macOS that can automount file systems, disks, local & remote, ie. across the network.
The configuration files for **automountd**
```conf
/etc/autofs.conf
/etc/auto_master
/etc/auto_resources
```
To show the file systems / partitions that have been automatically mounted
```shell
automount -v
```
If a change is made to one of the above files, the cache needs to be flushed
```shell
automount -cv
```
Some useful man pages include
```shell
man automount
man automountd
man auto_master
man autofs.conf
```
<aid="benchmarking-hdd-performance"></a>
#### Benchmarking HDD performance [🔝](#contents)
To test HDD performance on a macOS / Linux system, [see](https://www.cyberciti.biz/faq/howto-linux-unix-test-disk-performance-with-dd-command/)
A general universal command to test read / write performance to a storage media
```shell
time sh -c"dd if=/dev/zero of=/tmp/testfile bs=100k count=1k && sync"
```
> Results running macOS 10.12.6 with a HFS+ filesystem on a 1TB Samsung SSD
> The above command may not be applicable for macOS High Sierra as **Disk Utility** by default can show all partitions.
To list the supported file systems a disk can formatted with
```shell
diskutil listFilesystems
```
To list the available disks attached to the system
```shell
diskutil list
```
To erase a **Volume** using `diskutil` and not allocate the _free space_ with a partition.
```shell
diskutil eraseVolume free %noformat% [IDENTIFIER]
```
> [IDENTIFIER] can be printed using `diskutil list`
To erase an entire disk and not just a partition / volume on a disk and allocate with _free space_
```shell
diskutil eraseDisk free %noformat% [IDENTIFIER]
```
> The [IDENTIFIER] can be obtained with `diskutil list`, and verified with the **/dev/disk[NUMBER]**, also using the above command will format the disk with a GUID partition map, ie. known as a **Scheme** in the macOS world.
<strong>Ex</strong>
```shell
diskutil list
```
<strong>Output</strong>
```conf
/dev/disk3 (external, physical):
...
```
To resize a partition for a particular disk, ie. a USB flash drive using **diskutil** on macOS, use the **eraseDisk** in conjunction with the **resizeVolume** command.
> From my empirical evidence (could be wrong on this assessment) using diskUtil on macOS it's not possible to create a partition with a specific size from a **free space** drive using **diskUtil** on macOS. The reason for this is because there are **no** logical volumes, ie. partitions on the disk, that said, that's why the simplest solution I found was to create a **JHFS+** partition that can be resized at later date and time.
1. Create a JHFS+ partition that occupies all free / empty space for a disk.
```shell
diskutil eraseDisk JHFS+ grub-for-mac /dev/disk3
```
> About the above command, **eraseDisk** will nuke 🤯 the entire disk ie. removing all data from it, **grub-for-mac** will be the **volume label** for the disk, and **/dev/disk3** specifies the path to the disk to put the **JHFS** file system and partition on.
- Validate the disk is properly partitioned and formatted.
```shell
diskutil list
```
> Should see ~ 3 partitions indexed at 0
1. GUID_partition_scheme
2. EFI
3. Apple_HFS
- Next resize the JHFS+ partition to a smaller size
```shell
diskutil resizeVolume /dev/disk3s2 512M
```
> To get an idea of what size a file system on a partition / logical volume can be
```shell
diskutil resizeVolume /dev/disk3s2 limits
```
There now should be a partition on the removable USB disk that is ~ 512MB in size and formatted with a JHFS+ file system.
To create an additional partition on the above mentioned disk with a 512MB JHFS+ partition use an external program named **gdisk**.
1.**gdisk** can be easily obtained if [Homebrew](brew.sh) is installed.
```shell
brew cask install gdisk
```
> ⚠ To modify any attributes associated with a disk **gdisk** needs be launched with super user privileges.
- Launch **gdisk**
```shell
sudo gdisk
```
- Specify path to disk
```conf
/dev/disk[NUMBER]
```
- Create a partition with they type hex code of **0700** so a FAT file system can be formatted to the partition.
> To create a partition with a specific size using **gdisk** ie. create a 5GB on a 32GB disk use `+5G`
- Format the newly created partition with a **FAT-32** file system
To create Linux compatible file system for a particular partition on a disk
```conf
/opt/gnu/sbin/mkfs.ext2 /dev/disk2s2
```
> 🚨 The above mentioned command will create a **ext2** formatted filesystem. If you'd prefer to use an **ext4** filesystem, use **mkfs.ext4** instead. Also **mkfs.ext2** requires tools to be properly installed and setup on macOS in order to format a disk with a **ext{2,3,4}** file system.
To disable / turn off journaling for a **hfs+** disk
To verify if the file system successfully mounted using **FUSE for macOS**
```shell
mount
```
> The above command will display all locally mounted file systems.
To unmount a fuse-ext2 file system
```shell
sudo-E unmount /full/path/to/mount/point
```
#### fuse-ext2 Gotchas [🔝](#contents)
When working with fuse-ext2 to mount ext{2,3,4} file systems, presently `sudo` is required to mount a file system _boo_. In order to mount a file system and to allow access for a particular user on the system
...and **NOT** the **<key>Label</key>** in the **.plist** file.
- when testing a launchd service refer to the value in **<string>** defined below the **<key>Label</key>** within the **.plist**, ie.
```shell
launchctl list com.example.mr-fancy-42-service
```
- if **StandardErrorPath** & **StandardOutPath** or defined in the **.plist** file / service, launchd should create the files if not created yet, _NOTE_ double check permissions of directory where files will be written.
- launchd should append to the files defined by **StandardErrorPath** & **StandardOutPath** and not overwrite them each time the service is started. 👌
To get a frame of reference on how launchd service files, ie. **.plist** files can be written, see **/System/Library/Launch{Angents,Daemons}**
> From all my empirical evidence, environment variables can not be set within the **StandardOutPath** or the **StandardErrorPath**
<aid="launchd-general-notes"></a>
#### launchd General Notes [🔝](#contents)
A **daemon** is a program that runs in the background as part of the overall system (that is, it is not tied to a particular user). A daemon cannot display any GUI; more specifically, it is not allowed to connect to the window server.
An **agent** is a process that runs in the background on behalf of a particular user. Agents are useful because they can do things that daemons can't, like reliably access the user's home directory or connect to the window server.
> If a launchd service / `.plist` file calls / runs a shell script one can verify the the shell script contains no errors by passing the `-n` flag as an arugument to the shell script, ie.
```shell
sh -n mr_fancy_shell_script.sh
```
If the above command does not print any output in the terminal then their aren't any syntax errors using that particular shell interpreter.
### Working with Graphics Programming on macOS [🔝](#contents)
<aid="working-with-opengl-on-macos"></a>
#### Working with OpenGL on macOS [🔝](#contents)
macOS does not ship with OpenGL header files for building programs against OpenGL, however Xcode can be installed to build against some of the OpenGL header files.
To the best of my knowledge Apple stopped pulling in upstream features of OpenGL as of v3.2.
-[**StackExchange** setting `asciiquarium` as a screen saver](https://unix.stackexchange.com/questions/395151/how-can-i-install-asciiquarium-as-a-screensaver)
#### TODO [🔝](#contents)
- [ ] Mess around with Screen Saver project template in Xcode
<aid="working-with-applescript"></a>
### Working with AppleScript [🔝](#contents)
<aid="useful-links-applescript"></a>
#### Useful Links AppleScript [🔝](#contents)
-[The language of automation](https://macosxautomation.com/applescript/firsttutorial/11.html)
If a virtual hard disk for a QEMU VM needs to be increased then **fdisk** can be used to increase the partition after **qemu-img** has been used to increase the size of the virtual disk.
To increase the size a virtual **qcow** disk
```shell
qemu-img resize /path/to/virtual-disk.qcow2 10G # resize disk to 10G
```
To add 10 Gigabytes to a virtual disk
```shell
qemu-img resize /path/to/virtual-disk.qcow2 +10G # append 10G to virtual disk
```
<aid="linux-working-with-users-and-groups"></a>
### Working with Users and Groups [🔝](#contents)
To change the login password for a user on a GNU+Linux system
```shell
sudo passwd $USER
sudo passwd mr-fancy
```
To change the UID for a user on a GNU+Linux system
```shell
usermod -u[NEW_UID] mr-fancy
```
> ❗️mr-fancy can not change his own UID and no processes can be associated with mr-fancy. If mr-fancy has associated processes run the below commands
```shell
pkill -u mr-fancy pid
pkill -9-u mr-fancy
```
The same procedure is applicable for changing the GID of _mr-fancy_ as well
```shell
groupmod -g[NEW_GID] mr-fancy
```
❗️ Be certain to update old files and dirs that were associated with the **GID**[see](https://www.cyberciti.biz/faq/linux-change-user-group-uid-gid-for-all-owned-files/)
To print the **GID** for a particular group name ie. **staff**
To set an empty / blank password for the user **pi** on Raspbian
1. Make certain the **pi** user has sudo permissions.
❗️ The **pi** may require setting the default editor to run `visudo` otherwise the default editor will be set to **nano**
To set the default editor for **visudo**
```shell
sudo update-alternatives --config editor
```
2. Add the below line to **/etc/sudoers**
```conf
piALL=(ALL) NOPASSWD:ALL
```
3. Delete the current password for the **pi** user
```shell
sudo passwd -d`whoami`
```
After the above settings have been made the **pi** user should be able to login without being prompted for a password.
<aid="working-with-tty"></a>
#### Working with tty / pts from a CLI
A quick way to list **tty**'s / **pts**'s on a GNU+Linux distro
```shell
w
```
- ❗️ On **macOS** listing all active terminal sessions is a little different, and the **w** command only shows the initial console session when running `w`. However, a crude count can be obtained by listing all **/dev/ttysXXX*** devices within the **dev** directory.
- [To print print the active pseudo terminal / tty for a shell
```shell
tty
```
- To kill a different tty from an existing tty session
```shell
pkill -9-t pts/[NUMBER]
```
<strong>TODO</strong>
```shell
ps -ft pts/[NUMBER] pts/42 pts/24
kill[PID] [PID]
```
> ❗️ BSD and macOS are different beasts from GNU+Linux
To list the default options that will be applied when creating a new user on the system
```shell
useradd -D
```
To delete a user from the system, for more info [see](https://www.cyberciti.biz/faq/linux-remove-user-command/)
```shell
userdel mr_fancy_user
```
To create a new user with a specified shell, home directory, and user name
#### Setting new files in a directory to belong to a particular group [🔝](#contents)
1. Set the directory to have a **umask** of 007
```shell
umask 007
```
> The above command will make all newly created files to
```shell
u=rwx,g=rwx,o=
```
To verify the above setting
```shell
umask-S
```
To remove the **setuid** ie. _sticky_ bits from a directory
```shell
chmod ug-s /path/to/dir
echo"or try the below"
chmod 0755 /path/to/dir
```
To print **all** users on a Linux box
```shell
cut-d: -f1 /etc/passwd
```
To print **all** groups on a Linux system
```shell
cut-d: -f1 /etc/group
```
To print user and groups values of a particular user
```shell
id[user_name]
```
To print the group memberships for a particular user
```shell
groups[mr_fancy_user_name]
```
To create a new group on a Linux system
```shell
groupadd [mr-fancy-42-group]
```
To add an existing Linux system user to an existing group
```shell
usermod -a-G[name_of_group] [name_of_user]
```
> If adding a $USER to a group on a remote system through ssh more than likely the $USER will have to logout and then log back in after adding the $USER to a group.
To create **FAT-32** file system on a removable USB device on the **3rd** partition / volume of the device
```shell
mkdosfs -F 32 -I /dev/sda3
```
> /dev/sda is dependent on the disk path!
To remove a removable USB disk from a Linux system, make sure all partitions have been unmounted, then use **udisksctl**
-**keywords** power, off, turn, poweroff, turnoff, USB, powerdown, down
```shell
udiskctl power-off -b /dev/sd[A,B,C]
```
> **[A,B,C]** refers to the disk that one would like to unmount.
To safely remove a USB mass storage, 1) unmount the disks, 2) then power down the drive
```shell
sudo udisks --unmount /dev/sd[???]
echo"the below command will power down the drive
sudo udisks --detach /dev/[sd[???]
```
#### Working with autofs
To unmount all filesystems that have been mounted with **autofs**
```shell
umount -a-t autofs
```
<aid="working-with-udev"></a>
### Working with udev [🔝](#contents)
For an exhaustive guide to working with udev, [see](http://www.reactivated.net/writing_udev_rules.html)
To print the currently installed version of udev
```shell
udevadm version
```
To setup a custom network interface name using **udev** for a standard interface name, ie. **eth0** or **wlan0** create a **70-persistent-net.rules** within `/etc/udev/rules.d`
#### Setting up backports on Debian Stretch [🔝](#contents)
To setup Debian Stretch 9.x to work with a **backports-repo**
1. Edit `/etc/apt/sources.list` file or add a `.list` file, ie. a `debian-strech-backports.list` file within the `/etc/apt/sources.list.d/` directory and add the below entry.
```shell
deb http://ftp.debian.org/debian stretch-backports main
To print a list of files associated with a package
```shell
dpkg -L[mr-fancy-42-package]
```
To show package records for a particular package
```shell
apt-cache show postgresql
```
To remove a specific version from the list
```shell
apt-get remove postgresql=9.6+181+deb9u1
```
To list all packages related to particular string, ie. [postgres]
```shell
dpkg --get-selections | grep postgres
```
To display the package version number of all packages currently installed on the system
```shell
dpkg-query -l
```
To find the package that provides a particular ".h" file
```shell
dpkg -S <name_of_h_file.h>
```
Example,
```shell
dpkg -S stdio.h
```
Output
> libc6-dev: /usr/include/stdio.h<br />
> libc6-dev: /usr/include/bits/stdio.h<br />
> perl: /usr/lib/perl/5.6.0/CORE/nostdio.h<br />
Adding a key signature to apt-get, [see](https://github.com/yarnpkg/yarn/issues/4505)
To remove artifacts from `apt-get` installed packages
```shell
rm-rf /var/lib/apt/lists/*
```
then rerun the below command.
```shell
apt-get update
```
To see what packages can be upgraded
```shell
apt-get upgrade --dry-run
```
To **search** for a particular package from the `apt-get` sources
```shell
apt-cache search [mr-fancy-apt-package]
```
To **show** or list the dependencies a package requires using `apt`
```shell
apt-cache depends [mr-fancy-package]
```
ie.
```shell
apt-cache depends xsel
```
or
```shell
apt-cache depends xclip
```
> Running `apt-get update` on Debian 9.x was giving an expired an key error.
See the following [issue](https://github.com/fish-shell/fish-shell/issues/4753) about getting the updated key in order to resolve the expired key issue for OpenSuse server, ie. it is related to fish shell.
To reinstall an _app_ on Debian using `apt-get`
```shell
apt-get install--reinstall postgresql
```
To download the source for a particular package that would installed using **apt-get** command
```shell
apt-get source[packagename]
```
To pin / hold back a package from being upgraded via **apt** or **apt-get**
```shell
sudo apt-mark hold [PACKAGE_NAME]
```
To unpin / unhold a package
```shell
sudo apt-mark unhold [PACKAGE_NAME]
```
To add a **path** for every user on the system edit `/etc/login.defs` add the respected **path** entries for normal users, and the root user.
To list the partially installed or removed packages
```shell
dpkg -C
```
Then remove or reinstall the offending apt packages
```shell
apt install[OFFENDING_PKG]
apt install--force-yes[OFFENDING_PKG]
apt remove [OFFENDING_PKG]
apt remove --force-yes[OFFENDING_PKG]
```
From my empirical evidence **apt** performs without errors if there is only a single version of python in the **PATH**, ie. do not have **pyenv** to use both a 2.7.x and a 3.{6,7}.x variant of python with the below command
```shell
pyenv global system 3.6.5 2.7.15
```
Instead **use**
```shell
pyenv global system 3.6.5
```
<aid="debian-date-time-timezone"></a>
#### Debian date, time, timezone [🔝](#contents)
The quickest way to setup the proper time + date, and correct timezone is to install **ntp** if a network connection is present.
```shell
apt-get install ntp
```
To configure the Debian system for a specific timezone, ie. US Centeral TZ
```shell
sudo dpkg-reconfigure tzdata
```
> Follow the curses based menus for setting up tz data.
To remove / purge old packages `.deb` files on a Debian system
```shell
sudo aptitude clean
```
<aid="linux-apt-reseach-missing-packages"></a>
##### Research missing packages with apt [🔝](#contents)
To get a list of files a package can provide install **apt-file**, generate a DB for **apt-file** and then query **apt-file** with search strings. Also, **apt-file** can take a `list` argument to see what files a package provides, ie.
##### Working with source code using apt-get [🔝](#contents)
> Source code for Debian packages installed via `apt-get` is currently being maintained on [salsa.debian.org](https://salsa.debian.org)
An account will need me made on the _GitLab_ clone thingy, ie. server specifically designed for Debian packages, and then the apropriate package can be cloned to a local box.
If `apt-get` is having difficulty finding scripts to run, ie. `/usr/sbin/update-info-dir` make certain that `/usr/sbin` is in the current `$USER`'s `$PATH`
<aid="upgrading-from-jessie-to-stretch"></a>
#### Upgrading from Jessie to Stretch [🔝](#contents)
> Make sure all Jessie updates have been applied to the system first. To make sure no updates need to be applied
```shell
apt-get update upgrade --dry-run
```
Then run the two below commands to make sure there are no issues pending
```shell
dpkg --audit
dpkg --get-selection | grep hold
```
> Services such as postgres will need to be halted when performing a system upgrade because tools / libs ie, lib-c require them to be halted during the upgrade process.
To stop the postgres service on Debian jessie
```shell
/etc/init.d/postgresql-[version] stop
```
If postgres has been setup to work with systemd then the postgres server can be halted with the below command.
```shell
systemctl stop postgres
```
Then edit `sources.list`
```shell
nvim /etc/apt/sources.list
```
Remove or comment out source entries related to jessie and add the below entries related for stretch
<!-- NOTE: make sure to add a newline before and after fenced code blocks -->
```shell
deb http://mirrors.digitalocean.com/debian stretch main
deb-src http://mirrors.digitalocean.com/debian stretch main
deb http://security.debian.org/ stretch/updates main
deb-src http://security.debian.org/ stretch/updates main
deb http://mirrors.digitalocean.com/debian stretch-updates main
deb-src http://mirrors.digitalocean.com/debian stretch-updates main
```
Then run the below commands again
```shell
apt-get update
apt-get upgrade
```
Then run the below command
```shell
apt-get dist-upgrade
```
If the upgrade process halts for whatever reason it can be resumed using the below command
```shell
apt-get -f dist-upgrade
```
After everything has been installed, and the system has been rebooted, the old kernels can be purged.
To display the particular cron jobs for a user on a Debian box.
```shell
crontab -l
```
> The file printed from the above command is stored in the following in the following location on the system.
```shell
/var/spool/cron/cron/crontabs/
```
> When running a cron job / shell script make sure the script does not contain any commands being run with `sudo`. Long story short, a password will have to be stored in plain text somewhere on the system. 😬 When using env vars in a shell script run by cron make sure the env vars are set within the script itself.
See [this](https://askubuntu.com/questions/173924) for more details.
<aid="working-with-pamd"></a>
### Working with pam.d [🔝](#contents)
To restart the **pam.d** service
```shell
/usr/sbin/pam-auth-update
```
<aid="working-with-locales"></a>
### Working with locales [🔝](#contents)
**Update** February 17 2019
The below command resolved locale issues I was having after upgrading locales on Debian Stretch [see](https://github.com/Linuxbrew/brew/issues/568#issuecomment-367417842) for more info.
```shell
sudo sh -c'echo en_US.UTF-8 UTF-8 >>/etc/locale.gen'&&sudo /usr/sbin/locale-gen
```
To list the locales on a system
```shell
locale -a
```
To configure locales on a Debian based distro, ie. Raspbian
```shell
dpkg-reconfigure locales
```
> Follow the instructions in the curses based menu program. ⚠ `dpkg-reconfigure locales` must be run as **root** and not used via **sudo**
🚨 When generating locales using `locale-gen` run as root do not use `sudo` if you value your sanity 👩⚕️
To remove erronous locales from a system edit `/etc/locale.gen` and comment out the unnecessary locales.
The [following](https://askubuntu.com/a/144448/134740) link appears to have resolved the locale / perl warnings
To fix a broken locale system on a Debian box, [see](https://unix.stackexchange.com/questions/110757)
To remove unused / extra locales on a Debian box
```shell
sudo apt-get install localepurge
```
Unselect all the locales that aren't being used.
> When changing *locale* for a particular user, sometimes reloading the shell, ie. `exec bash` will not pick up on the *locale* change, so it's important to "logout" and "log" back in.
> The above command should be run as the `postgres` user on the system, DO NOT USE `sudo` or `root` to start the daemon unless you like headaches 🤕 If the [postgresql.service](https://github.com/ipatch/dotfiles/blob/master/jobs/Linux/systemd/postgresql.service) is being used in conjuction with PostgreSQL then the postgres can be controlled via Systemd
```shell
systemctl start postgresql.service
```
To check and see if postgresql server is running on the system
```shell
ps -ef | grep postgre
```
To reload postgresql
```shell
/etc/init.d/postgresql reload
```
To create a user for Postgres
```shell
createdb [mr-fancy-42-user]
```
> The above command helps alleviate the common below error message
```shell
psql: FATAL: database [user] does not exist
```
<aid="postgres-backing-up-databases"></a>
#### Postgres Backing up databases [🔝](#contents)
To implement a backup solution for postgres, [see](https://wiki.postgresql.org/wiki/Automated_Backup_on_Linux)
To setup a `cron` job to run the backup scripts daily copy or symlink `pg_back*` in the `Linux` dir within the `jobs` dir.
To test a daily cron job on Debain Linux
```shell
run-parts -v /etc/cron.daily -v
```
To back a Postgresql database, [see](https://www.digitalocean.com/community/tutorials/how-to-backup-postgresql-databases-on-an-ubuntu-vps) for more details.
> As of June 30 2018 I have a couple of shell scripts running to automate the process of taking daily snapshots of my Postgres databases. That said the scripts produce two back up files, ie. a **.gz** and a **.custom**
To restore a Postgres database from a snapshot using the `.custom` file, use the following,
> When restoring a DB that works with a rails web application, make sure to restore the DB, before running the migrations, and then run the migrations after restoring the database. 👌
If the `schema.rb` file is affected from importing data, and migrations are rerun, more than likely puma will need to be restarted after intitial deploy with new data in the database.
<aid="working-with-postgres-on-debian"></a>
#### Working with Postgres on Debian [🔝](#contents)
> `pg_ctlcluster` supersedes `pg_ctl` and in practice `pg_ctlcluster` should be used in favor of using `pg_ctl` when possible. That said, `pg_ctl` can be found at the below location
```shell
/usr/lib/postgresql/[MAJOR.MINOR]/bin/pg_ctl
```
The PostgreSQL configuration files, ie. `pg_hba.con` are located in the below location
```shell
/var/lib/postgresql/10_3_1/
```
The postgres binaries, ie. commands are located in the below location
```shell
/usr/lib/postgresql/10/bin
```
> `psql` is located in both `/usr/bin/psql` and `/usr/lib/postgresql/10/bin`
<aid="getting-postgres-to-work-with-systemd"></a>
##### Getting postgres to work with systemd
> `postgresql@.service` acts a template for setting up postgres to work with systemd.
The units file that interacts with Postgres server is,
```shell
/lib/systemd/system/postgresql.service
```
As of March 21, 2018, all database records / data are stored in the below location
```shell
/var/lib/postgresql/10_3_1
```
<aid="upgrading-postgres-from-10-3-to-10-4"></a>
#### Upgrading Postgres from 10.3.x to 10.4.x [🔝](#contents)
1. Installed Postgresql 10.4.x using **apt-get**
2. Dumped all existing data from 10.3.x
```shell
pg_dumpall > backup-may-30-2018
```
3. Stop the current database server, ie. Postgresql 10.3.x
```shell
pg_ctl stop
```
if using **systemd**
```shell
systemctl stop postgresql
```
4. Create a new **database cluster**
```shell
initdb -D /var/lib/postgresql/10_4_1
```
5. Start the new database server
```shell
postgres -D /var/lib/postgresql/10_4_1
```
> systemd
```shell
systemctl start postgresql
```
6. Restore the data from 10_3_1
```shell
psql -d postgres -f m backup-may-30-2018
```
> For more detailed instructions [see](https://www.postgresql.org/docs/10/static/upgrading.html)
To suppress the **last login from X.X.X.X IP address**
1. Edit /path/to/sshd_config
Add the below line to the file
```conf
PrintLastLogno
```
To suppress the **connection from IP closed**
1. Edit `$HOME/.ssh/config`
Add the below setting
```conf
LogLevelQUIET
```
<aid="working-with-systemd"></a>
### Working with systemd [🔝](#contents)
To restart a Systemd service
```shell
sudo systemctl restart [NAME_OF_SERVICE]
```
<strong>Ex</strong>
```shell
sudo systemctl restart sshd
```
<aid="systemd-history"></a>
#### systemd History [🔝](#contents)
Systemd was introduced by RedHat to replace / compliment SYSV for Linux distributions. Traditionally SYSV used syslogd or rsyslogd to log errors and output to log files for services and daemons running on the system. However systemd provides own logging facility which interacts for special journal files.
#### Systemd Working with journaling [🔝](#contents)
To control the journaling settings for **systemd** edit the below
```conf
/etc/systemd/journald.conf
```
> To get journaling to work properly on Debian 9.x I had to explicity set compression for the journal files to `off`
To allow a standard user to access and manipulate log files controlled by the journaling capabilities provided by systemd, perform the below
1. Add the standard user, ie. `whoami` to the **systemd-journal** group.
> See notes above, about working with users and groups, if needed.
2. Make sure the path to the **journal** directory and all sub folders and files are writeable by the **systemd-journal** group, ie. 775 for `/var/log/journal`
To **rotate** logs generated by systemd, one can use the **--vacuum-time=2weeks** flag / option to the `journalctl` command to trim / shrink the size of log files.
On Debian 9.x **systemd** stores system services in the below location
```conf
/lib/systemd/system/[mr-fancy-system.service]
```
**systemd** also stores `$USER` services in the below location
```conf
/usr/lib/systemd/user/[mr-fancy-user.service]
```
To start a systemd service for a particular `$USER`
```shell
systemctl --user start [service-name]
```
> The full path and the `.service` extension can be omitted if the **unit** file, (ie. that is what systemd calls the `.service` files.) is either of the above mentioned paths.
To add a additionaly systemd unit file the preferred location to add a unit file is within the `/etc/systemd/system` directory on Debian.
The unit files located in the `/etc/systemd/system` will take precedence over the ones that are located in `/lib/systemd/system`. Also, if there is a already a Debian provided systemd unit file in `/lib/systemd/system` the user provided will be chosen if located in the `/etc/systemd/system` directory. 👍
To install libpam-systemd
```shell
apt-get install libpam-systemd
```
> See, [🙈](https://unix.stackexchange.com/a/120086/33002) for more details
To list all enabled services from systemctl
```shell
systemctl list-unit-files | grep enabled
```
To analyze system boot time, ie. measure startup time of GNU+Linux using **systemd** performance
```shell
systemd-analyze
```
> When modifying a systemd `.service` file, the below command will need to be executed in order for systemd to pick up on the changes.
```shell
systemctl daemon-reload
```
<aid="troubleshooting-systemd"></a>
### Troubleshooting systemd
To remove / delete old systemd log files
```shell
sudo journalctl --flush--rotate
sudo journalctl --vacuum-time=2d
```
> The above command will remove all non active logs, and delete all active logs that 2 days or older.
<aid="working-with-man-pages">
### Working with man pages [🔝](#contents)
**man** pages are divided into various groups, ie. man pages about using a specific command, ie. `uname` will be located within **Section 1** where as getting a understanding of how the programming works for the `uname` command will be located within **Section 2**
#### `man` page directory layout [🔝](#contents)
| Directory | Section | Name |
| --------- | ------- | ---- |
| man1 | (1) | User Commands |
| man2 | (2) | System Calls |
| man3 | (3) | Subroutines |
| man4 | (4) | Devices |
| man5 | (5) | File Formats |
| man6 | (6) | Games |
| man7 | (7) | Miscellaneous |
| man8 | (8) | Sys. Administration |
| manl | (l) | Local |
| mann | (n) | New |
| mano | (o) | Old |
When working with man pages on macOS user added CLI commands can store their man page in `/usr/local/share/man/man1`. The `man1` directory is reserved for user added CLI commands, but **make certain** the man page has a `.1` or a `.1.gz` unless you don't want to read 📖 the man page ಠ_ಠ
<aid="useful-links-man-pages"></a>
#### Useful Links man pages [🔝](#contents)
-[cyberciti.biz - HowTo: Linux / UNIX Create a Manpage](https://www.cyberciti.biz/faq/linux-unix-creating-a-manpage/)
<aid="working-with-ld"></a>
### Working with `ld` [🔝](#contents)
To print the ld search path, see [🙈](https://stackoverflow.com/questions/9922949/)
