README.md 6.73 KB
Newer Older
MrMan's avatar
MrMan committed
1
2
3
4
5
6
# redis-bootleg-backup #

Key enumeration based backups for [`redis`](https://redis.io).

# Why ? #

7
8
9
`redis-bootleg-backup` (`rbb` for short) is for the awkward situation where you have access to a `redis` database but *do not* have the ability to use [`SAVE`](https://redis.io/commands/save)/[`DBSAVE`](https://redis.io/commands/save) or the [`--rdb` option on `redis-cli`](https://redis.io/topics/rediscli#remote-backups-of-rdb-files). If you are in doubt on whether you should be using this tool, you *should not be* -- it's only for those without access to any of the other officially supported options. There are better ways to backup redis and you should use those methods before attempting to use this tool.

**WARNING** This tool does [not yet preserve TTL settings for keys](https://gitlab.com/mrman/redis-bootleg-backup/-/issues/4).
MrMan's avatar
MrMan committed
10

11
# Quick start #
MrMan's avatar
MrMan committed
12

MrMan's avatar
MrMan committed
13
14
15
16
17
18
## Performing a backup ##

Backups are performed with the `backup` subcommand (i.e. `rbb backup`). How you store a backup can be configured via the `--store-type` command line option.

### Folder ###

MrMan's avatar
MrMan committed
19
To take a backup of a server at `redis://localhost:6379` using the default `folder-v1` store type, run the following command:
MrMan's avatar
MrMan committed
20

21
```shell
MrMan's avatar
MrMan committed
22
23
$ export RBB_FOLDER_BASE_DIR=path/to/backup/area
$ rbb backup --name <backup-name>
24
```
MrMan's avatar
MrMan committed
25

26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
If you used `/tmp/test` as your backup area and a backup name of `"test-backup"`, you'd see a file tree like the following:
```shell
$ tree /tmp/test
/tmp/test/
└── backups
    └── test-backup
        └── kv
            ├── test-key
            └── test-key-2

3 directories, 2 files
```

To specify the URI of the server manually, use the `--uri` option to the `backup` subcommand.

MrMan's avatar
MrMan committed
41
42
43
44
45
46
47
48
49
## SQLite ##

To use the `sqlite-v1` store type to perform a backup, run the following command:

```shell
$ export RBB_SQLITE_BASE_DIR=/path/to/backup/area # this tells rbb where to place the created DB file
$ rbb --store-type sqlite-v1 backup --name test --uri redis://localhost:6379
```

MrMan's avatar
MrMan committed
50
51
52
53
54
55
56
57
58
If you used `/tmp/test` as your backup area and a backup name of `"test-backup"` you should see a file tree like the following:
```shell
$ tree /tmp/test
/tmp/test/
└── test-backup.sqlite

0 directories, 1 file
```

MrMan's avatar
MrMan committed
59
## S3/Minio ##
MrMan's avatar
MrMan committed
60

MrMan's avatar
MrMan committed
61
To use the `s3-v1` store type to perform a backup, run the following command:
MrMan's avatar
MrMan committed
62
63

```shell
MrMan's avatar
MrMan committed
64
65
66
67
68
69
$ export RBB_S3_ENDPOINT=http://127.0.0.1:9000
$ export RBB_S3_REGION=us-east-1
$ export RBB_S3_BUCKET=test-bucket
$ export RBB_S3_ACCESS_KEY_ID=****************
$ export RBB_S3_SECRET_ACCESS_KEY=****************
$ rbb --store-type s3-v1 backup --name test --uri redis://localhost:6379
MrMan's avatar
MrMan committed
70
71
```

MrMan's avatar
MrMan committed
72
73
**NOTE** The S3 bucket you specify must exist *before* the backup is attempted. The bucket will hold multiple backups, you are not required to make a bucket per backup.

MrMan's avatar
MrMan committed
74
75
**NOTE** `REGION` is not required if using a Minio instance, but `ENDPOINT` is always required. AWS has a list of the [endpoints to use for AWS](https://docs.aws.amazon.com/general/latest/gr/s3.html)

MrMan's avatar
MrMan committed
76
Since the S3/Minio integration is remote, check the bucket to confirm the backup has been made -- the layout is similar (but slightly different) to the `folder` storage method.
MrMan's avatar
MrMan committed
77

MrMan's avatar
MrMan committed
78
## Performing a Restore ##
79
80
81
82

To restore a backup of a server at `redis://localhost:6379`, run the following command:

```
83
$ RBB_FOLDER_BASE_DIR=<path/to/backup/area> rbb restore --name <backup-name>
84
85
86
87
88
89
```

Pointing to the right backup area which contains a backup with the appropriate name will case all the keys that were backed up to be written on to the server.

To specify the URI of the server manually, use the `--uri` option to the `restore` subcommand.

MrMan's avatar
MrMan committed
90
91
92
93
94
### Performing restores with different storage backends ###

You can perform restores with different storage backends by using the `--store-type` command line option. Of course, this only makes sense if you performed a back up with the same store to begin with. For example to use the `sqlite-v1` store type to perform a restore, run the following command:

```shell
MrMan's avatar
MrMan committed
95
96
$ export RBB_SQLITE_BASE_DIR=/path/to/backup/area
$ rbb --store-type sqlite-v1 restore --name test --uri redis://localhost:6379
MrMan's avatar
MrMan committed
97
98
```

MrMan's avatar
MrMan committed
99
100
**NOTE** Don't forget to set the all environment variables required by the storage method you used.

101
102
# Usage #

103
You can run `rbb --help` to read about the options the
104
105

```shell
MrMan's avatar
MrMan committed
106
redis-bootleg-backup 0.2.0
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
Subcommands for structopt

USAGE:
    redis-bootleg-backup [OPTIONS] <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -d, --debug <debug>                    Activate debug mode [env: DEBUG=]
    -e, --executor-type <executor-type>    Type of executor to use [env: EXECUTOR_TYPE=]  [default: direct-v1]
    -s, --store-type <store-type>          Type of backup store [env: STORE_TYPE=]  [default: folder]

SUBCOMMANDS:
    backup     Perform a backup
    help       Prints this message or the help of the given subcommand(s)
    restore    Perform a restore
```

To see the options available for `backup` and `restore` subcommands, use `--help` on them, most importantly, the `--uri` option (env variable `URI`) can be used to configure the URI to the redis instance.

# Installation #

MrMan's avatar
MrMan committed
131
132
1. Download [the current (v0.2.0) `rbb` binary](https://gitlab.com/mrman/redis-bootleg-backup/-/raw/v0.2.0/releases/rbb?inline=false) (only supported on linux)
2. Validate the binary against the [SHA1 checksum](https://gitlab.com/mrman/redis-bootleg-backup/-/raw/v0.2.0/releases/rbb.sha1sum?inline=false) and/or [SHA256 checksum](https://gitlab.com/mrman/redis-bootleg-backup/-/raw/v0.2.0/releases/rbb.sha256sum?inline=false)
MrMan's avatar
MrMan committed
133
3. Run the `rbb` binary
MrMan's avatar
MrMan committed
134

135
136
## Docker ##

MrMan's avatar
MrMan committed
137
You can also run the dockerized CLI by using the `registry.gitlab.com/mrman/redis-bootleg-backup/cli:0.2.0` image.
138

MrMan's avatar
MrMan committed
139
# Extending `redis-bootleg-backup`: noteworthy upcoming features #
140
141
142
143
144

`redis-bootleg-backup` is built extensibly to allow for new `Executor`s (ways to communicate with `redis` and/or backup keys) and `Store`s (places to store the backup).

Taking advantage of this extensibility a following features are planned for v0.2.0:

MrMan's avatar
MrMan committed
145
- [x] [SQLite support](https://gitlab.com/mrman/redis-bootleg-backup/-/issues/3)
MrMan's avatar
MrMan committed
146
- [x] [S3 support](https://gitlab.com/mrman/redis-bootleg-backup/-/issues/3)
147
148

And for v0.3.0:
MrMan's avatar
MrMan committed
149
- [ ] [Parallelism for the `direct` executor](https://gitlab.com/mrman/redis-bootleg-backup/-/issues/7)
150
- [ ] [Streaming support](https://gitlab.com/mrman/redis-bootleg-backup/-/issues/2) (a third subcommand for streaming changes from a `redis` instance to a backup or vice-versa)
MrMan's avatar
MrMan committed
151
- [ ] [TTL support for all stores (folder, sqlite, s3, etc)](https://gitlab.com/mrman/redis-bootleg-backup/-/issues/4)
152

MrMan's avatar
MrMan committed
153
154
155
156
157
158
# Contributing #

0. Clone the repository
1. Run `make dev-setup` to set up your your local deployment environment
2. Improve/fix the code
3. [Make a pull request](https://gitlab.com/mrman/redis-bootleg-backup/-/issues/new)