qt3.14 collection o' backup scripts for QEMU/KVM-based VMs, MySQL/MariaDB databases and l0cal filesystems

Wazakindjes 8739699dcf Updated main restic.sh example conf with more details about --keep-* flags, also that shit was in the wrong place anyways xd 3 months ago
components 38bea874df Added new c0mp0nent: filesystem_remote [=[=[=[=[=[=[=[ 3 months ago
conf 8739699dcf Updated main restic.sh example conf with more details about --keep-* flags, also that shit was in the wrong place anyways xd 3 months ago
skrip e91f66457b Renamed wrapper_run_backup.sh to wrapper_run_task.sh so it's more general =] 3 months ago
.gitignore dc04dc2321 Let's also gitignore *.log lm0a 5 months ago
LICENSE ad13eccf0c Ayy lmao first versi0ne [[==[==[=[[=[==[[==[ 1 year ago
README.md 4d0b34cb8d readmuh: some min0r n0tes 3 months ago

README.md

The fuck is this

This originally started as just a qt lil collection of scripts I made to back up mein QEMU/KVM-based VMs to Backblaze B2, encrypted and all. Then I decided to add more shit so I can have 1 server handling all muh /b/lackups. :DDDDDDDDDDD

Requirements for ALL /b/lackup components

I won't be mentioning these anywhere else:

  • Obviously, a Backblaze B2 account and a bucket to store your shit in (make sure to set the lifecycle to Keep only the last version of the file, as we'll handle retention ourselves)
  • The restic utility for actually backing shit up (does deduplication and encryption etc =]]]), refer to their instructions for setting up B2

List of components

Shit should b p self-explanatory:

  • filesystem_local
  • filesystem_remote
  • kvm
  • mysql

Usage

There's one global config for storing your Backblaze information, cuz no matter what component you use it's always the same. ;];] Copy conf/restic.sh.example to conf/restic.sh and edit em, ez.

Every component has its own README so everything specific to that shit will be explained over there. In all cases there will be a conf dir per component, which contains another restic.sh. As you can guess, we'll first source the global restic.sh followed by the component-specific one. [[==[[= You'll find things like retention and post-actions in the component config.

I have a couple of "global" utility/helper scripts, found in the skrip dir:

  • find_restic_file.sh: given a path specification, list the paths of all backed up files under it (e.g ./find_restic_file.sh /backup/mysql)
  • list_restic_snapsh0ts.sh: every backup target is called a snapsh0t in restic, this will simply list all of them
  • list_restic_usage.sh: lists the raw size of the backup repository, i.e. this is (close to) what Backblaze will charge you for ;]
  • wrapper_run_backup.sh: a generic wrapper for running a component's run_backup_all.sh skrip, so we can make sure only 1 backup runs at a time [[=[=[=[==[=

For all of the *restic*.sh scripts any arguments are simply passed along to restic ($@ in Bash), so you can also filter by component tag: ./list_restic_snapsh0ts.sh --tag muhbackup_mysql

Muy importante: this shit doesn't check if you're running it as root nor does it elevate to it, it simply runs under the current user. Some people may have set up a separate management user that has permissions on everything they want/need backed up, so checking for rewt is pretty pointless imo tbh famalmalmala. ;] The same is true for scripts specific to a component, so I won't be mentioning it in their READMEs y0.

Although restic does incremental backups, it's still possible to delete e.g. the 5th backup out of 10. It will simply remove everything that was present in the 5th backup but not the previous or following ones. So if before the 5th backup you add a file, keep it until the 10th and delete the 5th, restic will repack the st0red bl0bs so only the changes between 4 and 5 will be wiped. When you'd restore e.g. the 9th backup, the file will still be there. [[[==[[=[==[=[[=[=

Some n0tes

  • If you want to delete any backup at any time from restic, simply do restic forget <snapsh0t ID> -- keep in mind the data is only actually removed once you run restic prune ;]
  • To view the full contents of a backup: restic ls <snapsh0t ID>
  • It's pr0lly best if you first manually run the backup 2 times (initial full and an incremental), this way you can see how long it would take and determine proper backup time frames ;];];];];];]
  • This shit is currently pretty verb0se, I may make it more compact at some point xd

Retention

Since this is likely to differ between components everything has its own Retention section. Whatever happens though, we will always keep at least 1 backup (restic forget --keep-last 1). ;]

References y0:

For VM backups:

Some future references y0: