aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorSilvio Rhatto <rhatto@riseup.net>2024-07-12 13:38:51 -0300
committerSilvio Rhatto <rhatto@riseup.net>2024-07-12 13:38:51 -0300
commit10912a5f713fc40473e4870f524dc8934f5559e7 (patch)
tree5dcae9e8d7a528829195fa73fcf341c85900e9f5
parentc6dde9cfcc68531c58394d9806b2c83dbd4fae8d (diff)
downloadhydra-10912a5f713fc40473e4870f524dc8934f5559e7.tar.gz
hydra-10912a5f713fc40473e4870f524dc8934f5559e7.tar.bz2
Feat: docs: improvements about backups and TPC volume handling
-rw-r--r--docs/backups.md89
-rw-r--r--docs/tpc.md81
2 files changed, 108 insertions, 62 deletions
diff --git a/docs/backups.md b/docs/backups.md
index c612116..d645207 100644
--- a/docs/backups.md
+++ b/docs/backups.md
@@ -11,33 +11,68 @@ This mostly has to do with `hydractl` commands such as
* But all this does not exempt a team from manually testing backups and to make
their own offline copies in external drives.
+## Parameters
+
+Start by running the following commands in your terminal, adjusting to your
+case:
+
+ export HYDRA="hydra-name"
+ export DISK="disk-name"
+ export DEVICE="/dev/sdc"
+ export USER="`whoami`"
+ export DOMAIN="`facter domain`"
+
+ if [ "$USER" != "root" ]; then
+ export SUDO="sudo"
+ fi
+
## New external drive
-Proceed as follows
+To add a new external drive into the pool, proceed as follows.
+
+First make sure to create a passphrase for the new disk into your hydra's keyring:
+
+ keyringer $HYDRA pwgen disks/$DISK.$DOMAIN/luks/root
+
+If you're using a USB drive, run this before plugging it on the computer:
+
+ hydractl usb-enable
+
+With the disk plugged in, make a layout with single `${DEVICE}1` partition
+(example: a disk with only one partition like `/dev/sdc1`):
- disk=new-disk-name
- device=/dev/sdc
- hydractl usb-enable # run this for USB drives, then connect the disk
- sudo cfdisk ${device} # layout with single ${device}1 partition
- sudo cryptsetup luksFormat ${device}1
- sudo cryptsetup luksOpen ${device}1 $disk
- sudo mkfs.ext4 /dev/mapper/$disk
- sudo mkdir /media/$disk
- sudo mount /dev/mapper/$disk /media/$disk
- sudo mkdir /media/$disk/media.`facter domain`
- sudo chown `whoami`: /media/$disk/media.`facter domain`
+ $SUDO cfdisk ${DEVICE}
-## NAS
+Now create the LUKS encryption header in this new disk, using the passphrase
+generated above by keyringer:
+
+ $SUDO cryptsetup luksFormat ${DEVICE}1
+
+Open the encrypted volume and create the basic filesystem structure needed
+to sync content:
+
+ $SUDO cryptsetup luksOpen ${DEVICE}1 $DISK
+ $SUDO mkfs.ext4 /dev/mapper/$DISK
+ $SUDO mkdir /media/$DISK
+ $SUDO mount /dev/mapper/$DISK /media/$DISK
+ $SUDO mkdir /media/$DISK/media.$DOMAIN
+ $SUDO chown ${USER}: /media/$DISK/media.$DOMAIN
+
+Finally, umount the drive:
+
+ hydractl umount-media $DISK
+
+## Regular sync
These commands should be enough to sync all media archives:
hydractl usb-enable # run this for USB drives, then connect the disk
- hydractl mount-media $MEDIA
- hydractl sync-media $MEDIA
+ hydractl mount-media $DISK
+ hydractl sync-media $DISK
As this should handle syncing all backups:
- hydractl sync-backups $MEDIA
+ hydractl sync-backups $DISK
## TPC
@@ -57,6 +92,11 @@ Then do the following:
hydractl mount-media $TPC
hydractl sync-tpc $TPC
+If you don't want to do a full TPC sync, but just want to sync the home folders
+(which is faster), use the following instead of the last command above:
+
+ hydractl sync-home $TPC
+
To sync archives and remote backups, proceed with as explained in the NAS
section.
@@ -71,10 +111,10 @@ homedir backups in the external archive/backup volume.
You might want to backup the whole SSD, M-SATA or microSD from your appliances.
If so, proceed as follows with the appliance device connected in your TPC:
- export appliance=appliance-name
- export dest="/var/backups/remote/$appliance.`facter domain`/image/`date +%Y%m%d`"
- sudo mkdir -p $dest
- dcfldd if=/dev/sdb | bzip2 > $dest/$appliance.img.bz2
+ export APPLIANCE="appliance-name"
+ export DEST="/var/backups/remote/${APPLIANCE}.${DOMAIN}/image/`date +%Y%m%d`"
+ $SUDO mkdir -p $dest
+ dcfldd if=/dev/sdb | bzip2 > $DEST/${APPLIANCE}.img.bz2
## Smartphone
@@ -100,7 +140,9 @@ A Backup Kit is a box with the following items:
* External encrypted archive/backup disk.
* Case for SSD transportation.
* Laptop power adapter and cables.
-* Dockstation SATA/USB/M.2/microSD/etc (with power adapter).
+* Dockstation SATA/USB/M.2/microSD/etc (with power adapter -- usually 12V).
+* Power adapter for external hard drives (usually 12V, and might be compatible
+ with the dockstationadapter, so you might carry just one).
* USB power adapter and cable (including USB 2, USB 3 and USB C).
* USB cables (USB 2, USB 3 and USB C) for the Dockstation and the external drive.
* TPC laptop with battery and TPS (SSD, M.2 etc), optionally with a UltraBase/Dockstation.
@@ -128,10 +170,11 @@ For [duplicity][]:
For [Borg][]:
+ export PATH_TO_RESTORE="path/to/be/restored"
mkdir ~/temp/misc/restore
cd ~/temp/misc/restore
borg list ssh://$USER@$SERVER:$PORT//var/backups/users/$USER/borg
- borg extract ssh://$USER@$SERVER:$PORT//var/backups/users/$USER/borg::$USER-2018-06-11T17:07:39 mnt/crypt/home/$USER/$FILE_OR_FOLDER
+ borg extract ssh://$USER@$SERVER:$PORT//var/backups/users/$USER/borg::$USER-2018-06-11T17:07:39 mnt/crypt/home/$USER/$PATH_TO_RESTORE
Make sure to cleanup `~/temp/misc/restore` after recovering what you need.
@@ -162,6 +205,6 @@ Note on backup keys:
For [eCryptfs][]:
- sudo ecryptfs-recover-private /media/$MEDIA/home/.ecryptfs/$USER/.Private
+ $SUDO ecryptfs-recover-private /media/$DISK/home/.ecryptfs/$USER/.Private
[eCryptfs]: https://www.ecryptfs.org/
diff --git a/docs/tpc.md b/docs/tpc.md
index d3e7e06..2d8a125 100644
--- a/docs/tpc.md
+++ b/docs/tpc.md
@@ -6,7 +6,10 @@ The Hydra Suite was made with the [Trusted Personal Computer (TPC)][]
This guide documents the lifecycle of a TPC storage device, or a "TPS
Cartridge":
- Cartridge = TPS = Trusted Personal Storage (like and SSD, microSD or M.2)
+ Cartridge = TPS = Trusted Personal Storage
+
+A TPS can usually be whatever removable block device like an SSD, microSD, M.2
+etc.
The idea is that a number of TPCs can have interchangeable cartridges, as long
as each cartridge has an unique name.
@@ -19,12 +22,12 @@ as each cartridge has an unique name.
Start by running the following commands in your terminal, adjusting to your
case:
- export HYDRA="nome-da-hydra"
- export VOLNAME="nome-do-disco"
- export DOMAIN="dominio.example.org"
+ export HYDRA="hydra-name"
+ export DISK="disk-name"
export DEVICE="/dev/sdb"
- export USER=`whoami`
- export DEST=/media/$VOLNAME
+ export DOMAIN="`facter domain`"
+ export USER="`whoami`"
+ export DEST="/media/$DISK"
export CONFIG_FOLDER="`hydra $HYDRA folder`"
if [ "$USER" != "root" ]; then
@@ -46,52 +49,52 @@ procedures from this document can be done from another TPC.
If non-existing, create entries and keys:
- hydra $HYDRA newnode $VOLNAME tpc
+ hydra $HYDRA newnode $DISK tpc
Customize:
- vim $CONFIG_FOLDER/config/provision/$VOLNAME.conf
- vim $CONFIG_FOLDER/puppet/config/node/$VOLNAME.$DOMAIN.yaml
- vim $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml
+ vim $CONFIG_FOLDER/config/provision/$DISK.conf
+ vim $CONFIG_FOLDER/puppet/config/node/$DISK.$DOMAIN.yaml
+ vim $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml
### Secrets
Proceed adding other passwords and secrets at
-`$CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN`, make sure to
+`$CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN`, make sure to
encode them with [hiera-yaml][]:
- hydra $HYDRA eyaml $VOLNAME encrypt -q -o block -p -l some::password
+ hydra $HYDRA eyaml $DISK encrypt -q -o block -p -l some::password
If you already have the secret somewhere, you can use a construction like this
which already adds the eyaml block into the hiera config file:
keyringer $HYDA decrypt /path/to/some/secret | \
tr -d '\n' | \
- hydra $HYDRA eyaml $VOLNAME encrypt -q -o block --stdin -l some::password >> \
- $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml
+ hydra $HYDRA eyaml $DISK encrypt -q -o block --stdin -l some::password >> \
+ $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml
You can also ensure a fresh random passphrase is used, using your favourite
generator like this:
head -c ${1:-20} /dev/urandom | base64 | \
tr -d '\n' | \
- hydra $HYDRA eyaml $VOLNAME encrypt -q -o block --stdin -l some::random:password >> \
- $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml
+ hydra $HYDRA eyaml $DISK encrypt -q -o block --stdin -l some::random:password >> \
+ $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml
For `passwd(5)` and `shadow(5)` hashed passphrases, use something like this:
mkpasswd -m sha-512 | \
tr -d '\n' | \
- hydra $HYDRA eyaml $VOLNAME encrypt -q -o block --stdin -l some::random:password >> \
- $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml
+ hydra $HYDRA eyaml $DISK encrypt -q -o block --stdin -l some::random:password >> \
+ $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml
Or this, for a random passphrase:
head -c ${1:-20} /dev/urandom | base64 | \
mkpasswd -m sha-512 --stdin | \
tr -d '\n' | \
- hydra $HYDRA eyaml $VOLNAME encrypt -q -o block --stdin -l some::random:password >> \
- $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml
+ hydra $HYDRA eyaml $DISK encrypt -q -o block --stdin -l some::random:password >> \
+ $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml
[hiera-yaml]: https://github.com/voxpupuli/hiera-eyaml
@@ -100,7 +103,7 @@ Or this, for a random passphrase:
The proceed with the system installation, which should create all volumes in
the TPS and setup the base Operating System:
- hydra $HYDRA provision $VOLNAME
+ hydra $HYDRA provision $DISK
### Metadata
@@ -112,23 +115,23 @@ can be useful during recovery procedures.
1. Write down UUIDS, models and serials from the new disk. Examples:
- lsblk $DEVICE -n -o +UUID | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/lsblk
- udevadm info --query=all --name=$DEVICE | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/udevadm
+ lsblk $DEVICE -n -o +UUID | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/lsblk
+ udevadm info --query=all --name=$DEVICE | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/udevadm
# For non-USB and non-microSD disks only
- $SUDO hdparm -I $DEVICE | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/hdparm
- $SUDO smartctl -i $DEVICE | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/smartctl
+ $SUDO hdparm -I $DEVICE | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/hdparm
+ $SUDO smartctl -i $DEVICE | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/smartctl
2. Save partition tables, the LUKS header and other relevant information:
- $SUDO LC_ALL=C sfdisk -d $DEVICE | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/sfdisk
+ $SUDO LC_ALL=C sfdisk -d $DEVICE | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/sfdisk
- $SUDO cryptsetup luksDump /dev/mapper/$VOLNAME-root | keyringer $HYDRA encrypt nodes/houdini.fluxo.info/luksDump-root
- $SUDO cryptsetup luksDump /dev/mapper/$VOLNAME-swap | keyringer $HYDRA encrypt nodes/houdini.fluxo.info/luksDump-swap
+ $SUDO cryptsetup luksDump /dev/mapper/$DISK-root | keyringer $HYDRA encrypt nodes/houdini.fluxo.info/luksDump-root
+ $SUDO cryptsetup luksDump /dev/mapper/$DISK-swap | keyringer $HYDRA encrypt nodes/houdini.fluxo.info/luksDump-swap
- $SUDO cryptsetup luksHeaderBackup /dev/mapper/$VOLNAME-root --header-backup-file luksHeaderBackup-root
+ $SUDO cryptsetup luksHeaderBackup /dev/mapper/$DISK-root --header-backup-file luksHeaderBackup-root
$SUDO chown $(whoami) luksHeaderBackup-root
- keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/luksHeaderBackup-root luksHeaderBackup-root
+ keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/luksHeaderBackup-root luksHeaderBackup-root
chmod +w luksHeaderBackup-root
wipe luksHeaderBackup-root
@@ -137,15 +140,15 @@ can be useful during recovery procedures.
* Optionally, configure ATA passphrases for the new disk. If so, make sure to
save the passphrase in the keyring:
- keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/ata
+ keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/ata
## Mounting
Use the following whenever you need to mount the cartridge in another system:
- hydractl mount-media $VOLNAME
+ hydractl mount-media $DISK
-This will make the volume available at `/media/$VOLNAME`.
+This will make the volume available at `/media/$DISK`.
## Initial user
@@ -161,7 +164,7 @@ home folder encrypted, backup your data and then proceed running
### Syncing the user and encrypted home in the new cartridge
-Then, with the new TPS cartridge mounted (`hydractl mount-media $VOLNAME`), do
+Then, with the new TPS cartridge mounted (`hydractl mount-media $DISK`), do
the following to replicate your user and homedir there:
$SUDO chroot $DEST adduser $USER
@@ -183,7 +186,7 @@ and is logged in there as `root`.
From another system, and with the cartridge mounted:
- hydractl sync-tpc $VOLNAME
+ hydractl sync-tpc $DISK
This might involve a lot of data being transmitted. If you want `sync-tpc` to
ignore all your QEMU virtual machine images hosted at `/var/cache/quemu`, setup
@@ -201,7 +204,7 @@ Now that basic provisioning is completed, you can unmount the cartridge.
From another system, and with the cartridge mounted:
- hydractl umount-media $VOLNAME
+ hydractl umount-media $DISK
## Booting
@@ -209,10 +212,10 @@ The current provisioning procedure (as of 2024-02) may yield into
a partially bootable system, and may need some additional manual
steps right after installation:
- hydractl mount-media $VOLNAME
+ hydractl mount-media $DISK
sudo chroot $DEST
update-initramfs -v -u
- hydractl umount-media $VOLNAME
+ hydractl umount-media $DISK
During boot, at the `(initramfs)` prompt, type this to successfully
unlock the encrypted volumes:
@@ -221,7 +224,7 @@ unlock the encrypted volumes:
cryptsetup luksOpen /dev/mapper/volumename-swap swap
^Ctrl-D
-Where `volumename` is `$VOLNAME`.
+Where `volumename` is `$DISK`.
Then, in the running system, do: