diff options
| -rw-r--r-- | docs/backups.md | 89 | ||||
| -rw-r--r-- | docs/tpc.md | 81 |
2 files changed, 108 insertions, 62 deletions
diff --git a/docs/backups.md b/docs/backups.md index c612116..d645207 100644 --- a/docs/backups.md +++ b/docs/backups.md @@ -11,33 +11,68 @@ This mostly has to do with `hydractl` commands such as * But all this does not exempt a team from manually testing backups and to make their own offline copies in external drives. +## Parameters + +Start by running the following commands in your terminal, adjusting to your +case: + + export HYDRA="hydra-name" + export DISK="disk-name" + export DEVICE="/dev/sdc" + export USER="`whoami`" + export DOMAIN="`facter domain`" + + if [ "$USER" != "root" ]; then + export SUDO="sudo" + fi + ## New external drive -Proceed as follows +To add a new external drive into the pool, proceed as follows. + +First make sure to create a passphrase for the new disk into your hydra's keyring: + + keyringer $HYDRA pwgen disks/$DISK.$DOMAIN/luks/root + +If you're using a USB drive, run this before plugging it on the computer: + + hydractl usb-enable + +With the disk plugged in, make a layout with single `${DEVICE}1` partition +(example: a disk with only one partition like `/dev/sdc1`): - disk=new-disk-name - device=/dev/sdc - hydractl usb-enable # run this for USB drives, then connect the disk - sudo cfdisk ${device} # layout with single ${device}1 partition - sudo cryptsetup luksFormat ${device}1 - sudo cryptsetup luksOpen ${device}1 $disk - sudo mkfs.ext4 /dev/mapper/$disk - sudo mkdir /media/$disk - sudo mount /dev/mapper/$disk /media/$disk - sudo mkdir /media/$disk/media.`facter domain` - sudo chown `whoami`: /media/$disk/media.`facter domain` + $SUDO cfdisk ${DEVICE} -## NAS +Now create the LUKS encryption header in this new disk, using the passphrase +generated above by keyringer: + + $SUDO cryptsetup luksFormat ${DEVICE}1 + +Open the encrypted volume and create the basic filesystem structure needed +to sync content: + + $SUDO cryptsetup luksOpen ${DEVICE}1 $DISK + $SUDO mkfs.ext4 /dev/mapper/$DISK + $SUDO mkdir /media/$DISK + $SUDO mount /dev/mapper/$DISK /media/$DISK + $SUDO mkdir /media/$DISK/media.$DOMAIN + $SUDO chown ${USER}: /media/$DISK/media.$DOMAIN + +Finally, umount the drive: + + hydractl umount-media $DISK + +## Regular sync These commands should be enough to sync all media archives: hydractl usb-enable # run this for USB drives, then connect the disk - hydractl mount-media $MEDIA - hydractl sync-media $MEDIA + hydractl mount-media $DISK + hydractl sync-media $DISK As this should handle syncing all backups: - hydractl sync-backups $MEDIA + hydractl sync-backups $DISK ## TPC @@ -57,6 +92,11 @@ Then do the following: hydractl mount-media $TPC hydractl sync-tpc $TPC +If you don't want to do a full TPC sync, but just want to sync the home folders +(which is faster), use the following instead of the last command above: + + hydractl sync-home $TPC + To sync archives and remote backups, proceed with as explained in the NAS section. @@ -71,10 +111,10 @@ homedir backups in the external archive/backup volume. You might want to backup the whole SSD, M-SATA or microSD from your appliances. If so, proceed as follows with the appliance device connected in your TPC: - export appliance=appliance-name - export dest="/var/backups/remote/$appliance.`facter domain`/image/`date +%Y%m%d`" - sudo mkdir -p $dest - dcfldd if=/dev/sdb | bzip2 > $dest/$appliance.img.bz2 + export APPLIANCE="appliance-name" + export DEST="/var/backups/remote/${APPLIANCE}.${DOMAIN}/image/`date +%Y%m%d`" + $SUDO mkdir -p $dest + dcfldd if=/dev/sdb | bzip2 > $DEST/${APPLIANCE}.img.bz2 ## Smartphone @@ -100,7 +140,9 @@ A Backup Kit is a box with the following items: * External encrypted archive/backup disk. * Case for SSD transportation. * Laptop power adapter and cables. -* Dockstation SATA/USB/M.2/microSD/etc (with power adapter). +* Dockstation SATA/USB/M.2/microSD/etc (with power adapter -- usually 12V). +* Power adapter for external hard drives (usually 12V, and might be compatible + with the dockstationadapter, so you might carry just one). * USB power adapter and cable (including USB 2, USB 3 and USB C). * USB cables (USB 2, USB 3 and USB C) for the Dockstation and the external drive. * TPC laptop with battery and TPS (SSD, M.2 etc), optionally with a UltraBase/Dockstation. @@ -128,10 +170,11 @@ For [duplicity][]: For [Borg][]: + export PATH_TO_RESTORE="path/to/be/restored" mkdir ~/temp/misc/restore cd ~/temp/misc/restore borg list ssh://$USER@$SERVER:$PORT//var/backups/users/$USER/borg - borg extract ssh://$USER@$SERVER:$PORT//var/backups/users/$USER/borg::$USER-2018-06-11T17:07:39 mnt/crypt/home/$USER/$FILE_OR_FOLDER + borg extract ssh://$USER@$SERVER:$PORT//var/backups/users/$USER/borg::$USER-2018-06-11T17:07:39 mnt/crypt/home/$USER/$PATH_TO_RESTORE Make sure to cleanup `~/temp/misc/restore` after recovering what you need. @@ -162,6 +205,6 @@ Note on backup keys: For [eCryptfs][]: - sudo ecryptfs-recover-private /media/$MEDIA/home/.ecryptfs/$USER/.Private + $SUDO ecryptfs-recover-private /media/$DISK/home/.ecryptfs/$USER/.Private [eCryptfs]: https://www.ecryptfs.org/ diff --git a/docs/tpc.md b/docs/tpc.md index d3e7e06..2d8a125 100644 --- a/docs/tpc.md +++ b/docs/tpc.md @@ -6,7 +6,10 @@ The Hydra Suite was made with the [Trusted Personal Computer (TPC)][] This guide documents the lifecycle of a TPC storage device, or a "TPS Cartridge": - Cartridge = TPS = Trusted Personal Storage (like and SSD, microSD or M.2) + Cartridge = TPS = Trusted Personal Storage + +A TPS can usually be whatever removable block device like an SSD, microSD, M.2 +etc. The idea is that a number of TPCs can have interchangeable cartridges, as long as each cartridge has an unique name. @@ -19,12 +22,12 @@ as each cartridge has an unique name. Start by running the following commands in your terminal, adjusting to your case: - export HYDRA="nome-da-hydra" - export VOLNAME="nome-do-disco" - export DOMAIN="dominio.example.org" + export HYDRA="hydra-name" + export DISK="disk-name" export DEVICE="/dev/sdb" - export USER=`whoami` - export DEST=/media/$VOLNAME + export DOMAIN="`facter domain`" + export USER="`whoami`" + export DEST="/media/$DISK" export CONFIG_FOLDER="`hydra $HYDRA folder`" if [ "$USER" != "root" ]; then @@ -46,52 +49,52 @@ procedures from this document can be done from another TPC. If non-existing, create entries and keys: - hydra $HYDRA newnode $VOLNAME tpc + hydra $HYDRA newnode $DISK tpc Customize: - vim $CONFIG_FOLDER/config/provision/$VOLNAME.conf - vim $CONFIG_FOLDER/puppet/config/node/$VOLNAME.$DOMAIN.yaml - vim $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml + vim $CONFIG_FOLDER/config/provision/$DISK.conf + vim $CONFIG_FOLDER/puppet/config/node/$DISK.$DOMAIN.yaml + vim $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml ### Secrets Proceed adding other passwords and secrets at -`$CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN`, make sure to +`$CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN`, make sure to encode them with [hiera-yaml][]: - hydra $HYDRA eyaml $VOLNAME encrypt -q -o block -p -l some::password + hydra $HYDRA eyaml $DISK encrypt -q -o block -p -l some::password If you already have the secret somewhere, you can use a construction like this which already adds the eyaml block into the hiera config file: keyringer $HYDA decrypt /path/to/some/secret | \ tr -d '\n' | \ - hydra $HYDRA eyaml $VOLNAME encrypt -q -o block --stdin -l some::password >> \ - $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml + hydra $HYDRA eyaml $DISK encrypt -q -o block --stdin -l some::password >> \ + $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml You can also ensure a fresh random passphrase is used, using your favourite generator like this: head -c ${1:-20} /dev/urandom | base64 | \ tr -d '\n' | \ - hydra $HYDRA eyaml $VOLNAME encrypt -q -o block --stdin -l some::random:password >> \ - $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml + hydra $HYDRA eyaml $DISK encrypt -q -o block --stdin -l some::random:password >> \ + $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml For `passwd(5)` and `shadow(5)` hashed passphrases, use something like this: mkpasswd -m sha-512 | \ tr -d '\n' | \ - hydra $HYDRA eyaml $VOLNAME encrypt -q -o block --stdin -l some::random:password >> \ - $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml + hydra $HYDRA eyaml $DISK encrypt -q -o block --stdin -l some::random:password >> \ + $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml Or this, for a random passphrase: head -c ${1:-20} /dev/urandom | base64 | \ mkpasswd -m sha-512 --stdin | \ tr -d '\n' | \ - hydra $HYDRA eyaml $VOLNAME encrypt -q -o block --stdin -l some::random:password >> \ - $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml + hydra $HYDRA eyaml $DISK encrypt -q -o block --stdin -l some::random:password >> \ + $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml [hiera-yaml]: https://github.com/voxpupuli/hiera-eyaml @@ -100,7 +103,7 @@ Or this, for a random passphrase: The proceed with the system installation, which should create all volumes in the TPS and setup the base Operating System: - hydra $HYDRA provision $VOLNAME + hydra $HYDRA provision $DISK ### Metadata @@ -112,23 +115,23 @@ can be useful during recovery procedures. 1. Write down UUIDS, models and serials from the new disk. Examples: - lsblk $DEVICE -n -o +UUID | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/lsblk - udevadm info --query=all --name=$DEVICE | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/udevadm + lsblk $DEVICE -n -o +UUID | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/lsblk + udevadm info --query=all --name=$DEVICE | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/udevadm # For non-USB and non-microSD disks only - $SUDO hdparm -I $DEVICE | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/hdparm - $SUDO smartctl -i $DEVICE | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/smartctl + $SUDO hdparm -I $DEVICE | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/hdparm + $SUDO smartctl -i $DEVICE | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/smartctl 2. Save partition tables, the LUKS header and other relevant information: - $SUDO LC_ALL=C sfdisk -d $DEVICE | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/sfdisk + $SUDO LC_ALL=C sfdisk -d $DEVICE | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/sfdisk - $SUDO cryptsetup luksDump /dev/mapper/$VOLNAME-root | keyringer $HYDRA encrypt nodes/houdini.fluxo.info/luksDump-root - $SUDO cryptsetup luksDump /dev/mapper/$VOLNAME-swap | keyringer $HYDRA encrypt nodes/houdini.fluxo.info/luksDump-swap + $SUDO cryptsetup luksDump /dev/mapper/$DISK-root | keyringer $HYDRA encrypt nodes/houdini.fluxo.info/luksDump-root + $SUDO cryptsetup luksDump /dev/mapper/$DISK-swap | keyringer $HYDRA encrypt nodes/houdini.fluxo.info/luksDump-swap - $SUDO cryptsetup luksHeaderBackup /dev/mapper/$VOLNAME-root --header-backup-file luksHeaderBackup-root + $SUDO cryptsetup luksHeaderBackup /dev/mapper/$DISK-root --header-backup-file luksHeaderBackup-root $SUDO chown $(whoami) luksHeaderBackup-root - keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/luksHeaderBackup-root luksHeaderBackup-root + keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/luksHeaderBackup-root luksHeaderBackup-root chmod +w luksHeaderBackup-root wipe luksHeaderBackup-root @@ -137,15 +140,15 @@ can be useful during recovery procedures. * Optionally, configure ATA passphrases for the new disk. If so, make sure to save the passphrase in the keyring: - keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/ata + keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/ata ## Mounting Use the following whenever you need to mount the cartridge in another system: - hydractl mount-media $VOLNAME + hydractl mount-media $DISK -This will make the volume available at `/media/$VOLNAME`. +This will make the volume available at `/media/$DISK`. ## Initial user @@ -161,7 +164,7 @@ home folder encrypted, backup your data and then proceed running ### Syncing the user and encrypted home in the new cartridge -Then, with the new TPS cartridge mounted (`hydractl mount-media $VOLNAME`), do +Then, with the new TPS cartridge mounted (`hydractl mount-media $DISK`), do the following to replicate your user and homedir there: $SUDO chroot $DEST adduser $USER @@ -183,7 +186,7 @@ and is logged in there as `root`. From another system, and with the cartridge mounted: - hydractl sync-tpc $VOLNAME + hydractl sync-tpc $DISK This might involve a lot of data being transmitted. If you want `sync-tpc` to ignore all your QEMU virtual machine images hosted at `/var/cache/quemu`, setup @@ -201,7 +204,7 @@ Now that basic provisioning is completed, you can unmount the cartridge. From another system, and with the cartridge mounted: - hydractl umount-media $VOLNAME + hydractl umount-media $DISK ## Booting @@ -209,10 +212,10 @@ The current provisioning procedure (as of 2024-02) may yield into a partially bootable system, and may need some additional manual steps right after installation: - hydractl mount-media $VOLNAME + hydractl mount-media $DISK sudo chroot $DEST update-initramfs -v -u - hydractl umount-media $VOLNAME + hydractl umount-media $DISK During boot, at the `(initramfs)` prompt, type this to successfully unlock the encrypted volumes: @@ -221,7 +224,7 @@ unlock the encrypted volumes: cryptsetup luksOpen /dev/mapper/volumename-swap swap ^Ctrl-D -Where `volumename` is `$VOLNAME`. +Where `volumename` is `$DISK`. Then, in the running system, do: |