qt3.14 collection o' backup scripts for QEMU/KVM-based VMs, MySQL/MariaDB databases and l0cal filesystems
|
7 months ago | |
---|---|---|
components | 7 months ago | |
conf | 7 months ago | |
skrip | 7 months ago | |
.editorconfig | 1 year ago | |
.gitignore | 2 years ago | |
LICENSE | 3 years ago | |
README.md | 7 months ago |
This originally started as just a qt lil collection of scripts I made to back up mein QEMU/KVM-based VMs to Backblaze B2, encrypted and all. Then I decided to add more shit so I can have 1 server handling all muh /b/lackups. :DDDDDDDDDDD
I won't be mentioning these anywhere else:
Keep only the last version of the file
, as we'll handle retention ourselves)Shit should b p self-explanatory:
There's one global config for storing your Backblaze information, cuz no matter what component you use it's always the same. ;];] Copy conf/restic.sh.example
to conf/restic.sh
and edit em, ez.
Every component has its own README so everything specific to that shit will be explained over there. In all cases there will be a conf
dir per component, which contains another restic.sh
. As you can guess, we'll first source
the global restic.sh
followed by the component-specific one. [[==[[= You'll find things like retention and post-actions in the component config.
I have a couple of "global" utility/helper scripts, found in the skrip
dir:
find_restic_file.sh
: given a path specification, list the paths of all backed up files under it (e.g ./find_restic_file.sh /backup/mysql
)list_restic_snapsh0ts.sh
: every backup target is called a snapsh0t in restic, this will simply list all of themlist_restic_usage.sh
: lists the raw size of the backup repository, i.e. this is (close to) what Backblaze will charge you for ;]wrapper_run_backup.sh
: a generic wrapper for running a component's run_backup_all.sh
skrip, so we can make sure only 1 backup runs at a time [[=[=[=[==[=For all of the *restic*.sh
scripts any arguments are simply passed along to restic
($@
in Bash), so you can also filter by component tag: ./list_restic_snapsh0ts.sh --tag muhbackup_mysql
Muy importante: this shit doesn't check if you're running it as root nor does it elevate to it, it simply runs under the current user. Some people may have set up a separate management user that has permissions on everything they want/need backed up, so checking for rewt is pretty pointless imo tbh famalmalmala. ;] The same is true for scripts specific to a component, so I won't be mentioning it in their READMEs y0.
Muy importante 2: you can have it do maintenance tasks such as restic prune
and restic check
after every backup, but you might incur some massive Download Bandwidth
costs with B2 when you do that. I used to run that shit daily and those costs were almost as much as the storage itself. xd So I made a separate maintenance
component for doing this every X weeks instead.
Although restic does incremental backups, it's still possible to delete e.g. the 5th backup out of 10. It will simply remove everything that was present in the 5th backup but not the previous or following ones. So if before the 5th backup you add a file, keep it until the 10th and delete the 5th, restic will repack the st0red bl0bs so only the changes between 4 and 5 will be wiped. When you'd restore e.g. the 9th backup, the file will still be there. [[[==[[=[==[=[[=[=
restic forget <snapsh0t ID[, ...]>
-- keep in mind the data is only actually removed once you run restic prune
;]restic ls <snapsh0t ID>
Since this is likely to differ between components everything has its own Retention
section. Whatever happens though, we will always keep at least 1 backup (restic forget --keep-last 1
). ;] This also generally means that you'll have to manually restic forget <snapsh0t ID(s)>
to remove something the last backup of something (e.g. a deleted database), as well as its associated temporary backup directory. Of course this doesn't apply to individual files within a mount point; since we direct restic
to the root of that, that is what it will use for determining what paths to keep.
# Obviously not necessary, only added this so the specific days from below make more sense ;]
0 1 1,15 * * zpool scrub muhpool01
0 2 * * * /mnt/muhpool01/muhbackup/skrip/wrapper_run_task.sh mysql 43200 /mnt/muhpool01/log/muhbackup 10
0 3 * * * /mnt/muhpool01/muhbackup/skrip/wrapper_run_task.sh filesystem_local 43200 /mnt/muhpool01/log/muhbackup 10
0 4 * * * /mnt/muhpool01/muhbackup/skrip/wrapper_run_task.sh filesystem_remote 43200 /mnt/muhpool01/log/muhbackup 10
0 5 5,12,19,26 * * /mnt/muhpool01/muhbackup/skrip/wrapper_run_task.sh kvm 43200 /mnt/muhpool01/log/muhbackup 10
0 6 7,21 * * /mnt/muhpool01/muhbackup/skrip/wrapper_run_task.sh maintenance 43200 /mnt/muhpool01/log/muhbackup 10
For VM backups: