Commit ab9d26c1 authored by Mamadou Babaei's avatar Mamadou Babaei
Browse files

add documentation

parent 147f0ca4
OmniBackup
==========
# OmniBackup: One Script to back them all up
One Script to back them all up
OmniBackup is a MIT-licensed Bash script which delivers the following set of features:
* Support for *NIX operating systems (e.g. *BSD and GNU/Linux)
* Configuration and customization of backup mechanism through JSON
* Support for OpenLDAP backups
* Support for PostgreSQL backups as a whole or per database
* Support for MariaDB and MySQL backups as a whole or per database
* Support for filesystem backups with optional ability to follow symbolic links
* Support for pluggable customized scripts to extend OmniBackup functionality beyond its original design which allows support for many different backup scenarios that has not been built into OmniBackup, yet
* Backup file name tagging which allows including date or host name in the archive name
* Online backup without a prerequisite to suspend any service
* Support for customized backup tasks priority order
* Support for multiple backup servers
* Ability to always keep a copy of backups offline
* Ability to keep a copy of backups offline if no backup server is available, or in case of an error such as a file transfer error
* Secure file transfer through SSH / SCP protocol
* LZMA2, gzip and bzip2 compression algorithms along with different compression levels to maintain a balance between speed and file size
* Ability to preserve permissions inside backup files
* Support for symmetric cryptography algorithms AES-128, AES-192 and AES-256 (a.k.a Rijndael or Advanced Encryption Standard)
* Random passphrase generation for encrypted backups with variable length and patterns or a unique passphrase for all backups
* Support for RSA signatures to verify the backup origin and integrity
* Passphrase encryption using RSA public keys for individual backup servers
* Backup integrity verification by offering hash algorithms such as MD4, MD5, MDC-2, RIPEMD160, SHA, SHA-1, SHA-224, SHA-256, SHA-384, SHA-512 and WHIRLPOOL
* Optional Base64 encoding
* System logs and a standalone log file including all details
* Reporting through email to a list of recipients with ability to include passphrases
* Customized mail subject for successful and failed backup operations
* Customized support message for reports
* Crontab integration
* Custom temporary / working directory
* Automatic and smart clean-up ability
* One instance only policy which avoids running multiple instances by mistake at the same time, therefore avoids system slow-down
* An example configuration file in JSON format to get you up and running
## Disclaimer
_Please be wary of the fact that this script has approximately <code>3.5 K</code> lines of Bash code and devoured hell of a time from me to write and debug. You should also consider that this is my first heavy Bash experiment and I may not write quality code in the language since I'm a newcomer to Bash. I do not claim that OmniBackup is production ready, that's why I did version the first release at <code>0.1.0</code>. Also keep in mind that OmniBackup heavily relies on 3rd-party software which increases the chance for fatal bugs, therefore losing data. So, I provide OmniBackup without any warranties, guarantees or conditions, of any kind and I accept no liability or responsibility for any misuse or damage. Please use it at your own risk and remember you are solely responsible for any resulting damage or data loss._
## Documentation
Check out the documentation at
[OmniBackup: One Script to Back Them All Up](http://www.babaei.net/blog/2015/08/08/omnibackup-one-script-to-back-them-all-up/)
* [Message Types and Their Meanings](#MessageTypes)
* [Requirements](#Requirements)
* [Installation](#Installation)
* [Configuring Temporary Directory](#ConfigTempDirectory)
* [Configuring Compression](#ConfigCompression)
* [Configuring Security](#ConfigSecurity)
* [Configuring Reports](#ConfigReports)
* [Configuring Remote Backup Servers](#ConfigRemoteBackupServers)
* [Configuring Backup Tasks](#ConfigBackupTasks)
* [Configuring Backup Priority and Order](#ConfigBackupPriorityOrder)
* [Configuring OpenLDAP Backups](#ConfigOpenLdapBackups)
* [Configuring Database Backups](#ConfigDatabaseBackups)
* [Configuring PostgreSQL Database Backups](#ConfigPostgreSqlBackups)
* [Configuring MariaDB and MySQL Databases Backups](#ConfigMariaDbMySqlBackups)
* [Configuring Filesystem Backups](#ConfigFilesystemBackups)
* [Configuring Other Backups](#ConfigMiscBackups)
* [3rd-party Commands status codes](#Config3rdPartyCommandsStatusCode)
* [Generating RSA Keys](#GeneratingRSAKeys)
* [Password-less SSH Login](#PasswordlessSshLogin)
* [SMTP Relay for Hosts with Private IPs](#SmtpRelay)
* [First Run](#FirstRun)
* [Crontab](#Crontab)
* [Restore](#Restore)
* [Restoring Encrypted Archives](#RestoringEncryptedArchives)
* [Restoring Archives](#RestoringArchives)
* [Restoring OpenLDAP Backups](#RestoringOpenLDAP)
* [Restoring PostgreSQL Backups](#RestoringPostgreSQL)
* [Restoring MariaDB or MySQL Backups](#RestoringMariaDbMySQL)
* [Staying Away From Disaster](#StayingAwayFromDisaster)
* [Example Report](#ExampleReport)
* [ToDo](#ToDo)
* [License](#License)
<br />
<a name="MessageTypes"></a>
### Message Types and Their Meanings ###
Before we go any further, you may want to know that other than regular logs there are different kinds of messages in OmniBackup which may appear on screen, system logs or mail reports:
* <code>DEBUG</code> it's for development purpose only, so you should not see any message of this kind.
* <code>ERROR</code> some error happened, but hopefully the program is able to continue.
* <code>FATAL</code> indicates that something went south due to unrecoverable errors and the program terminates immediately.
* <code>INFO</code> informational notices to inform you about regular events inside the program.
* <code>SUCCESS</code> indicates an operation has been successful.
* <code>TRACE</code> it's for development purpose only, so you should not see any message of this kind.
* <code>WARNING</code> indicates that a dangerous situation might happen or already happened which is not desired.
<br />
<a name="Requirements"></a>
### Requirements ###
Originally OmniBackup developed on FreeBSD. Recently, I had the time to test it on Funtoo Linux (a variant of Gentoo) and without any modification to the original code it worked just fine. In my estimation, the current code should work out of the box inside any GNU/Linux distribution since I tried my best to write it in a platform-independent manner by reading pile of man pages. I assume if it works on FreeBSD it should also work on other BSDs and GNU/Linuxes. But I'm only 99.99% sure and when it comes to programming computers 0.01% human error is really too much and risky. So, I really appreciate feedbacks and possible patches or pull-requests.
Aside from Bash, OmniBackup requires other tools and utilities. Although most of these programs are found in a base installation of your operating system, it relies on a few other programs which has to be installed before you go any further. Note that if OmniBackup cannot find a program it gives you a fatal error message and exits.
Requirements for OmniBackup include:
* <code>basename</code>
* <code>bash</code> version <code>4.2+</code>
* <code>bzip2</code>
* <code>caller</code>
* <code>cat</code>
* <code>cd</code>
* <code>chown</code>
* <code>cut</code>
* <code>date</code>
* <code>dirname</code>
* <code>du</code>
* <code>echo</code>
* <code>expr</code>
* <code>flock</code>
* <code>grep</code>
* <code>gzip</code>
* <code>head</code>
* <code>hostname</code>
* <code>jq</code> version <code>1.4+</code>
* <code>logger</code>
* <code>mail</code>
* <code>mkdir</code>
* <code>mysqldump</code>
* <code>openssl</code>
* <code>pg_dump</code>
* <code>pg_dumpall</code>
* <code>printf</code>
* <code>readlink</code>
* <code>rm</code>
* <code>scp</code>
* <code>slapcat</code>
* <code>ssh</code>
* <code>strings</code>
* <code>sudo</code>
* <code>tar</code>
* <code>tr</code>
* <code>xz</code>
I should add, not all of the above dependencies are required in order for OmniBackup to work. At runtime, it dynamically decides which dependencies are required and then search for them. For example, if you did not enabled PostgreSQL database backup, it won't look for <code>pg_dump</code>, <code>pg_dumpall</code> and <code>sudo</code> binaries. Or, if you choose to go with LZMA2 compression algorithm it only looks for <code>xz</code> and ignore both <code>bzip2</code> and <code>gzip</code> binaries. Of course some of these commands like <code>cd</code> are internal and there is no need to lookup the filesystem to find them. On my FreeBSD system I only had to install the following ports in order to have all the dependencies complete:
* <code>databases/postgresql9*-client</code> one of these ports provide <code>pg_dump</code> and <code>pg_dumpall</code>
* <code>databases/mariadb*-client</code> one of these ports provides <code>mysqldump</code>
* <code>databases/mysql5*-client</code> one of these ports provides <code>mysqldump</code>
* <code>openldap24-server</code> provides <code>slapcat</code>
* <code>security/sudo</code> provides <code>sudo</code>
* <code>shells/bash</code> provides <code>bash</code>
* <code>sysutils/flock</code> probably a default on GNU/Linux, provides <code>flock</code> executable on FreeBSD
* <code>textproc/jq</code> provides <code>jq</code>
Note that from the above list <code>flock</code> and [jq](http://stedolan.github.io/jq/) are only mandatory requirements unless based on OmniBackup configurations other dependencies get pulled in. The best way to determine dependencies is to ignore the list of dependencies and first configure your OmniBackup instance. When your done with that, run OmniBackup manually for the first time. If it won't complain about any dependency then you are good to go. However, if it does, then you should resolve the dependencies one by one until you satisfy all the dependencies and good to go.
__Notes for Funtoo/Gentoo Linux:__ From the mandatory requirements list, I had to install <code>app-misc/jq</code>. All other requirements except <code>mail</code> were in place. Both <code>mail-client/mailx</code> and <code>mail-mta/ssmtp</code> provide <code>mail</code> command for you.
```
$ emerge -avt mail-client/mailx
```
or
```
$ emerge -avt mail-mta/ssmtp
```
In addition to that, running OmniBackup for the first time on Funtoo generated lots of this error:
```
logger: socket /dev/log: No such file or directory
```
To fix that install and start <code>app-admin/syslog-ng</code>:
```
$ emerge -avt app-admin/syslog-ng
$ rc-update add syslog-ng default
$ service syslog-ng start
```
<br />
<a name="Installation"></a>
### Installation
Installing OmniBackup is really easy. It consists of two files: a huge script file -- a little less than <code>105 KB</code> -- named <code>backup.sh</code> which looks for the second file at runtime named <code>config.json</code>. So, let's say I want to install OmniBackup inside <code>/usr/local/omnibackup</code>. In addition to that, I assume from now on you do everything as <code>root</code> user:
```
$ cd /usr/local/
$ git clone https://github.com/NuLL3rr0r/omnibackup.git
$ cd omnibackup
$ cp config.json.sample config.json
$ chmod u+rx,go-r,a-w backup.sh
$ chmod u+rw,go-rw,a-x config.json
```
All we did was cloning the code from GitHub, copying over the sample configuration file as a template and assigning the right permissions for both the script and configuration file. I prefer to make this files <code>root</code> accessible only, so no one else can read our configuration or modify it or even triggering the backup process.
You should avoid running the backup script in this step. As I mentioned we just copied a sample file to get you up and running. So, you should only run it after the final configurations are done, since it pickups possible dependencies from the configuration file and it may baffles you with the wrong dependencies errors. So, for now open-up the configuration file in your favorite editor and take a look.
<br />
<a name="ConfigTempDirectory"></a>
### Configuring Temporary Directory ###
The first option inside <code>config.json</code> file is <code>.temp_dir</code> which specifes the temporary or working directory for OmniBackup. <code>/var/tmp</code> seems to be a reasonable place. Feel free to adopt it according to your needs. But, if you are going to change it to a path other than <code>/var/tmp/</code> or <code>/tmp/</code> choose an empty one. Note that each time you run OmniBackup it creates a log file inside <code>/var/tmp/</code> e.g. <code>/var/tmp/backup.2015-07-31.58471.log</code>. You cannot change the path for the log files due to technical limitations. Keep in mind that OmniBackup never removes its log files due to their small footprints. They may come handy when reports won't deliver to your email.
```
{
"temp_dir" : "/var/tmp",
}
```
<br />
<a name="ConfigCompression"></a>
### Configuring Compression
You've been given three options for compression:
```
{
"compression" :
{
"algorithm" : "lzma2",
"level" : "6e",
"preserve_permissions" : "yes"
},
}
```
<code>.compression.algorithm</code> accepts only four possible values: <code>lzma2</code>, <code>gzip</code> and <code>bzip2</code> which determines the compression algorithm, or, you can leave it blank for no compression at all. This affects the extension of the backup file which we call archive from now on. For <code>lzma2</code> it will be <code>.tar.xz</code>, for <code>gzip</code>, <code>.tar.gz</code>, for <code>bzip2</code>, <code>tar.bz2</code> and for no compression mode it will be simply <code>.tar</code>. Also, <code>lzma2</code>, <code>gizp</code> and <code>bzip2</code> values, pull in <code>xz</code>, <code>gizp</code> and <code>bizip2</code> binaries as dependency, respectively.
<code>.compression.level</code> option accepts values between <code>1</code> to <code>9</code> for <code>gizp</code> and <code>bzip2</code> algorithms. For <code>lzma2</code> algorithm it accepts values between <code>1</code> to <code>9</code> and <code>1e</code> to <code>9e</code>. <code>e</code> stands for extreme and aggressive compression which demands more RAM and CPU cycles. In case you choose no compression mode, the <code>level</code> option will be ignored.
<code>.compression.preserve_permissions</code> is self-explanatory, it preserve the archived files permissions inside the final archive file.
<br />
<a name="ConfigSecurity"></a>
### Configuring Security
Security module provides many features that you may not notice at the first sight:
```
{
"security" :
{
"checksum_algorithm" : "sha512",
"encryption" :
{
"enable" : "yes",
"key_size" : "256",
"base64_encode" : "no",
"passphrase" : "",
"random_passphrase_pattern" : "print",
"random_passphrase_length" : 128,
"private_key" : "~/keys/private.pem"
}
},
}
```
Please be wary, this module heavily relies on OpenSSL. So, some of the hash or encryption algorithms may not work in case you or your distribution built OpenSSL with those options excluded.
<code>.security.checksum_algorithm</code> provides various hash algorithms to verify the archive file integrity later on. <code>md4</code>, <code>md5</code>, <code>mdc2</code>, <code>ripemd160</code>, <code>sha</code>, <code>sha1</code>, <code>sha224</code>, <code>sha256</code>, <code>sha384</code>, <code>sha512</code> and <code>whirlpool</code> algorithms are all valid options and well supported. If encryption is disabled or you did not provide a private key, OmniBackup creates a checksum file with extension <code>.sum</code> which includes the archive checksum and uploads it along with your archive file. If encryption is enabled and you did provide a private key it uses the checksum file to sign the archive and instead of uploading <code>.sum</code> file, uploads the signature file with extension <code>.sign</code>, so that it can be verified using your public key at the destination. In either case OmniBackup includes the checksum in reports and send it through email to you.
<code>.security.encryption.enable</code> should either set to <code>yes</code> or <code>no</code>. If you set it to <code>no</code> the rest of the encryption options will be ignored.
<code>.security.encryption.key_size</code> only accepts <code>128</code>, <code>192</code> and <code>256</code> values which in turn enables AES-128, AES-192 and AES-256 encryption.
<code>.security.encryption.base64_encode</code> if you set it to <code>yes</code>, the encrypted archive file, signature file and the encrypted archive passphrase will be base64 encoded, otherwise they will all be in binary format which saves up disk space and bandwidth.
<code>.security.encryption.passphrase</code> is the passphrase to encrypt or decrypt archive files. If you leave it blank, OmniBackup generates a random password for each archive file. Otherwise, it uses the specified password for all archive files and ignore both <code>random_passphrase_pattern</code> and <code>random_passphrase_length</code> options. If you decide to use a unique password for all backups make sure <code>config.json</code> is only readable by its owner. If set to blank, it pulls in <code>grep</code>, <code>head</code>, <code>strings</code> and <code>tr</code> as dependency.
<code>.security.encryption.random_passphrase_pattern</code> indicates the pattern to generate a random passphrase. For more information see the documentation of [GNU](http://www.gnu.org/software/grep/manual/html_node/Character-Classes-and-Bracket-Expressions.html) and [BSD](https://www.freebsd.org/cgi/man.cgi?query=grep) implementations of <code>grep</code>:
* <code>alnum</code> Alphanumeric characters: <code>alpha</code> and <code>digit</code>; in the <code>C</code> locale and ASCII character encoding, this is the same as <code>[0-9A-Za-z]</code>.
* <code>alpha</code> Alphabetic characters: <code>lower</code> and <code>upper</code>; in the <code>C</code> locale and ASCII character encoding, this is the same as <code>[A-Za-z]</code>.
* <code>blank</code> Blank characters: space and tab.
* <code>cntrl</code> Control characters. In ASCII, these characters have octal codes <code>000</code> through <code>037</code>, and <code>177 (DEL)</code>. In other character sets, these are the equivalent characters, if any.
* <code>digit</code> Digits: <code>0 1 2 3 4 5 6 7 8 9</code>.
* <code>graph</code> Graphical characters: <code>alnum</code> and <code>punct</code>.
* <code>lower</code> Lower-case letters; in the <code>C</code> locale and ASCII character encoding, this is <code>a b c d e f g h i j k l m n o p q r s t u v w x y z</code>.
* <code>print</code> Printable characters: <code>alnum</code>, <code>punct</code>, and space.
* <code>punct</code> Punctuation characters; in the <code>C</code> locale and ASCII character encoding, this is <code>! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ _ ` { | } ~</code>.
* <code>space</code> Space characters: in the <code>C</code> locale, this is tab, newline, vertical tab, form feed, carriage return, and space.
* <code>upper</code> Upper-case letters: in the <code>C</code> locale and ASCII character encoding, this is <code>A B C D E F G H I J K L M N O P Q R S T U V W X Y Z</code>.
* <code>xdigit</code> Hexadecimal digits: <code>0 1 2 3 4 5 6 7 8 9 A B C D E F a b c d e f</code>.
<code>.security.encryption.random_passphrase_length</code> a positive number bigger than <code>0</code> that indicates the length of random passphrases.
<code>.security.encryption.private_key</code> is a private key in PEM format used to sign the encrypted archive file so that anyone knows the file originated from you. Note it only accepts absolute path and relative paths starting from home folder (tilde <code>~</code>). Do not start paths relative to OmniBackup directory. Also, keep in mind that avoid space and exotic characters inside path. It's both sane and safe to only include <code>A-Z</code>, <code>a-z</code> and underscore <code>_</code> in path because I cannot guarantee its safety. We'll discuss [RSA key generation](#GeneratingRSAKeys) later on.
<br />
<a name="ConfigReports"></a>
### Configuring Reports
Reports module is here to allow you become aware of all the details of the events that happened during the backup process, through email:
```
{
"reports" :
{
"mailboxes" :
[
{
"email_address" : "backups.archive@babaei.net",
"attach_passphrases" : "yes"
},
{
"email_address" : "backups.verify@babaei.net",
"attach_passphrases" : "no"
}
],
"subject" :
{
"success" : "[{HOST_NAME} {DATE}] BACKUP REPORT / SUCCESS",
"error" : "[{HOST_NAME} {DATE}] BACKUP REPORT / ERROR",
"fatal" : "[{HOST_NAME} {DATE}] BACKUP REPORT / FATAL ERROR"
},
"support_info" : "Have question or need technical assistance, please visit http://www.babaei.net/ or write an email to info [ at ] babaei.net."
},
}
```
<code>.reports.mailboxes</code> is a JSON array of email addresses and their settings who will receive the backup report when it's done, either successful or failed.
<code>.reports.mailboxes.email_address</code> should be a valid email address.
<code>.reports.mailboxes.attach_passphrases</code> determines whether passphrases for archive files should be attached to the reports for that specific email or not.
<code>.reports.subject</code> provides the ability to determine the subject of reports for different scenarios that might happen during the backup process at runtime, so by looking at your inbox you immediately realize what happened and take the appropriate action if it's necessary. <code>{HOST_NAME}</code> and <code>{DATE}</code> are recognized special placeholder keywords. They will be replaced at runtime by the host name OmniBackup backup running on or the date OmniBackup instance starts backup jobs, respectively.
<code>.reports.success</code> is the email subject when backup process finished successfully.
<code>.reports.error</code> is the email subject when at least one error happened during the backup process.
<code>.reports.fatal</code> is the email subject when backup process faced a fatal error so it could not finish the jobs.
<code>.reports.support_info</code> allows adding a customized support message to the end of the reports.
<br />
<a name="ConfigRemoteBackupServers"></a>
### Configuring Remote Backup Servers
Using the <code>.remote</code> section of the configuration file, you can configure as many as remote backup servers that you wish. If you plan to keep backup files locally, this section provides two possible ways to do that which we'll discuss later. Note that you also have to setup [password-less SSH login](#PasswordlessSshLogin) regardless of choosing a remote server or the current host as a backup server.
```
{
"remote" :
{
"keep_backup_locally" : "partial",
"servers" :
[
{
"host" : "10.12.0.4",
"port" : "22",
"user" : "babaei",
"dir" : "~/backups/{HOST_NAME}",
"backups_to_preserve" : 7,
"public_key" : "~/keys/10.12.0.4_babaei.pem"
},
{
"host" : "example.domain",
"port" : "8931",
"user" : "babaei",
"dir" : "~/backups/{HOST_NAME}",
"backups_to_preserve" : 31,
"public_key" : "~/keys/example.domain_babaei.pem"
}
]
},
}
```
<code>.remote.keep_backup_locally</code> accepts four possible values.
* <code>never</code>: always remove the temporary backup file, no matter what.
* <code>error</code>: always remove the temporary backup file unless an upload error happens.
* <code>partial</code>: keep the backup file in case the upload operation for a single backup file was partially successful. This indicates that at least one upload operation for a specific backup file has been successful. So, we are sure we have the backup file on at least one of the backup servers.
* <code>success</code>: do not remove the local backup file and keep it inside <code>.temp_dir</code> directory, even in case of succeeding all upload operations. This option is one way to always keep the backup files locally although it is not recommended. If you're going to use this method anyway, it is recommended to use some other path instead of <code>/var/tmp</code> or <code>/tmp</code> as your <code>.temp_dir</code>.
* <code>.remote.servers</code> is a JSON array of backup servers. The second method to keep a copy of backup files locally is to define the current host as another remote server here which is the recommended method to do so.
* <code>.remote.servers.host</code> is the host name or IP address of a backup server.
* <code>.remote.servers.port</code> is the SSH port for that backup server.
* <code>.remote.servers.user</code> is a user with SSH access on the server.
* <code>.remote.servers.dir</code> specifies a directory in which we keep the backup files. <code>{HOST_NAME}</code> is recognized as a valid placeholder for the current host's name (not the backup server) and should be automatically replaced at runtime.
* <code>.backups_to_preserve</code> determines how many days of older backups should be kept. For example, if I set this to <code>10</code> for one of the backup servers, I should only have <code>10</code> days of backup on that server, plus today or tonight's backup. Be advised that any negative number -- e.g. <code>-1</code> -- has special meanings here. It means do not clean up any older backup, basically keep them forever.
* <code>public_key</code> is a public key in PEM format for encrypting passphrases. As you recall these passphrases are required to decrypt the archive files. If you do not specify a public key for a backup server, you do not have access to encrypted passphrases on that server. It is useful in case that you do not wish to keep the passphrases in the same place as your encrypted archive files, even the encrypted ones. I must warn and assure you, if you are using random passphrases and and you did not provide any public key here, your backups are good for nothing unless you chose to attach passphrases to at least one email in the reports section. In addition to everything, it's possible to only provide one pair of RSA keys instead of one private key and multiple public keys if you are the sole receiver of the backup files on those multiple servers. I'll describe [RSA key generation](#GeneratingRSAKeys) later.
<br />
<a name="ConfigBackupTasks"></a>
### Configuring Backup Tasks
<code>.backup</code> is the actual part of the configuration which determines what has to be backed up. I'll break it down into a few sections for the sake of simplicity.
```
{
"backup" :
{
"archive_name" : "{DATE}_{HOST_NAME}_{TAG}",
},
}
```
<code>.backup.archive_name</code> is a pattern for archive file names. You can prefix or postfix them as you wish by modifying this option. <code>{DATE}</code> shall be replaced by the actual date that OmniBackup script started. The date format is fixed and looks like <code>YYYY-MM-DD</code>. <code>{HOST_NAME}</code> shall be replaced by the host name that OmniBackup runs on. <code>{TAG}</code> is an option for each item that should be backed up, so it shall be replaced with that option's value for each backup task. So, with above configuration in mind, a typical backup archive file should look like this:
2015-07-31_blog.babaei.net_openldap-babaei-net.tar.xz
<code>2015-07-31</code>, <code>blog.babaei.net</code> and <code>openldap-babaei-net</code> are <code>{DATE}</code>, <code>{HOST_NAME}</code> and <code>{TAG}</code> in the archive file name, respectively.
<br />
<a name="ConfigBackupPriorityOrder"></a>
### Configuring Backup Priority and Order
One of the most important steps is to configure what should be backed up, in which order:
```
{
"backup" :
{
"priority_order" :
[
"openldap",
"database",
"filesystem",
"misc"
],
},
}
```
<code>.backup.priority_order</code> only recognizes four elements: <code>openldap</code>, <code>database</code>, <code>filesystem</code> and <code>misc</code>. Although, they should be self-explanatory, I have to add a comment for <code>misc</code> which is a special item in this list. It allows you to run an external script and pass arguments to it. After the script is finished, OmniBackup is able to backup a directory or file as its output. So, we'll be able to plug extra scripts to OmniBackup. We will discuss this later.
In the above example, OmniBackup backups in this order: OpenLDAP, databases, filesystem and finally other customized backups. You can change the order by swapping their order of appearance. If you want for example avoid backing up, filesystem, you should remove it from this list. Otherwise, OmniBackup looks for its configuration.
Let's consider an example if I need to only backup first database and then filesystem:
```
{
"backup" :
{
"priority_order" :
[
"database",
"filesystem"
],
},
}
```
<br />
<a name="ConfigOpenLdapBackups"></a>
### Configuring OpenLDAP Backups
This is all we need to backup OpenLDAP objects and directories using <code>slapcat</code>:
```
{
"backup" :
{
"openldap" :
{
"tag" : "openldap-babaei-net",
"user" : "ldap",
"group" : "ldap",
"flags" : ""
},
},
}
```
<code>.backup.openldap.tag</code> is used to replace <code>{TAG}</code> in <code>.backup.archive_name</code>. Since it's going to be part of a file name, it is highly recommended to avoid space or any other character that needs escaping or quotation.
<code>.backup.openldap.user</code> is optional system user to run <code>slapcat</code> under.
<code>.backup.openldap.group</code> is optional system group to run <code>slapcat</code> under.
<code>.backup.openldap.flags</code> is used to pass arguments directly to <code>slapcat</code>. If you are OK with defaults or do not have any idea what is that, just leave it as it is.
<br />
<a name="ConfigDatabaseBackups"></a>
### Configuring Database Backups
Like the general backup configuration, database section also needs specifying the types of backup jobs and their order of running:
```
{
"backup" :
{
"database" :
{
"priority_order" :
[
"postgres",
"mysql"
],
},
},
}
```
OmniBackup officially only recognizes two types of database to backup. PostgreSQL and MariaDB / MySQL. Of course, it won't stop you from writing your own backup scripts for other databases. You can plug such scripts to OmniBackup using <code>.backup.misc</code> which we'll see later.
<code>.backup.database.priority_order</code> everything for <code>.backup.priority_order</code> also applies here.
<br />
<a name="ConfigPostgreSqlBackups"></a>
### Configuring PostgreSQL Database Backups
Configuring PostgreSQL backups consists of two easy steps: first providing a user name with an optional group name and finally a list of databases to backup:
```
{
"backup" :
{
"database" :
{
"postgres" :
{
"user" : "pgsql",
"group" : "pgsql",
"databases" :
[
{
"tag" : "postgres",
"name" : "*",
"comment" : "All PostgreSQL databases"
},
{
"tag" : "postgres-gitlab",
"name" : "gitlab",
"comment" : "GitLab database"
},
{
"tag" : "postgres-redmine",
"name" : "redmine",
"comment" : "Redmine database"
},
{
"tag" : "postgres-agilo",
"name" : "agilo",
"comment" : "Agilo for Trac database"
},
{
"tag" : "postgres-owncloud",
"name" : "owncloud",
"comment" : "ownCloud database"
},
{
"tag" : "postgres-tracks",
"name" : "tracks",
"comment" : "Get on Tracks database"
}
]
},
},
},
}
```
<code>backup.database.postgres.user</code> is a system user's name which runs our PostgreSQL service. For example, on my FreeBSD system it is called <code>pgsql</code>. This user should be created by Ports, pkgng installation of PostgreSQL or whatever package manager your distro uses. It has different names on different platforms. You can figure it out by investigating <code>/etc/passwd</code> or <code>/etc/master.passwd</code>:
```
$ cat /etc/passwd
```
or
```
$ cat /etc/master.passwd
```
<code>backup.database.postgres.group</code> is a mandatory system group's name which runs our PostgreSQL service. This one is completely optional. You can figure this one out by investigating <code>/etc/group</code>.
<code>backup.database.postgres.databases</code> is a JSON array of PostgreSQL databases.
<code>backup.database.postgres.databases.tag</code> is a tag for archive file name. Everything for <code>.backup.openldap.tag</code> also applies here.
<code>backup.database.postgres.databases.name</code> is the exact database name inside your PostgreSQL instance. * means create a backup of all tables in one go, in one single dump file. I found it a best practice to have an entire database dump and a separate dump for each database. Why? Because, in one hand I sometimes forget to add new databases here, so that * takes care of that for me. On the other hand, sometimes you may face error restoring or importing back all your databases using a single dump file -- due to a bug, human error or anything --, so you have each database backup separately, then you won't loose much in such a scenario.
<code>backup.database.postgres.databases.comment</code> is used inside logs, syslogs and reports to refer to that database instead of name which is not always clear.
<br />
<a name="ConfigMariaDbMySqlBackups"></a>
### Configuring MariaDB and MySQL Databases Backups
The same as PostgreSQL goes for configuring MariaDB or MySQL databases:
```
{
"backup" :
{
"database" :
{
"mysql" :
{
"user" : "mysql",
"group" : "mysql",