My Dotfile Workflow: Stow, Git Filters, and Encryption

There are dozens of ways to manage your dotfiles. From manually copying files from one machine to another, to putting them in a git repository and manually linking the files to their targets, to using tools that copy files into place and keep them in sync. Managing dotfiles can become a constant challenge, especially when you regularly use more than one machine.

For my part I primarily work on 2 machines, my desktop and my laptop. I have various VPSes and Servers in my homelab that need some of my configuration, but only the parts that deal with shell interaction. I also have my .emacs configuration that I do at times use elsewhere (such as on a work machine if it is permitted) so it belongs in a separate repo for ease of access.

Of course just because I only have 2 systems that I really need to worry about doesn’t mean that I don’t struggle keeping them in sync. I start using one tool and incorporate it but then can’t use it elsewhere because that machine is turned off and I don’t remember how I set things up. So as part of my homelab rebuild I decided to declare configuration bankruptcy and start over, at least as far as how I manage synchronization.

What I tried before

Last time I went down the path of trying to keep my dotfiles in sync I set up Chezmoi, I actually still have it managing certain files as I complete the migration to my new setup. There was absolutely nothing wrong with the tool, it did everything I wanted and did it well.

1. Configuration encryption in the repository, allowing for sensitive data to be stored.
2. Configuration templating to allow for variations between systems.
3. Ability to add external repositories to clone and update when applying chezmoi changes.

My new setup only has the first feature. It has a very basic version of the second but will require work or additional tooling to add the third.

Chezmoi had 2 real downsides for my use-case. The first was a workflow problem where I would make changes to my configuration, forget to synchronize it with Chezmoi, or remember to make the change in Chezmoi but never commit and/or re-apply it on my other machines. This led to my configuration being out of sync and not having the same experience on my Laptop as I did on my desktop. Being more diligent would help, but it was just enough steps that I found myself being very inconsistent.

The second was that there is no easy way to only apply a subset of the configuration. Yes, you can leverage templates to only apply a file if some setting is true, but that means turning files into templates solely for the purpose of “Only apply this file if my setup says to”. I don’t need to manage my SSH keys if the machine is only an SSH target, I don’t need my GPG keys applied on a VPS that will never actually use GPG, there isn’t any reason to include my backup configuration for my home (~/) directory when there isn’t anything relevant in it.

A new beginning

Based on my past experiences and desires I had 3 criteria I definitely wanted to meet and a 4th that would be nice to have.

1. File encryption so that sensitive data could be stored in the repo
2. Ability to only apply a subset of the configuration as opposed to everything without having to modify every optional file to flag it.
3. Simpler workflow. This probably meant symbolic links (symlink) rather than files being copied.
4. (Optional) File templating to allow for per-machine differences.

Right away GNU Stow stood out by satisfying requirements 2 and 3. Instead of the typical duplication of your configuration layout it uses an intermediate folder layer to define packages. Each package has it’s own set of files that it stows so if you don’t need the ssh package, you don’t apply it.

As a comparison (these are only a subset of configuration files meant purely to show the difference in layouts).

 1  .config/
 2  ├── awesome/
 3  │   ├── config.lua
 4  │   ├── rc.lua
 5  │   └── themes/
 6  ├── borg-space/
 7  │   └── settings.nt
 8  ├── chezmoi/
 9  │   └── chezmoi.yaml
10  ├── dconf/
11  │   └── user
12  └── dunst/
13    └── dunstrc

 1dotfiles/
 2├── aspell/
 3│   └── .aspell.en.pws
 4├── atuin/
 5│   └── .config/
 6│       └── atuin/
 7├── git/
 8│   ├── .gitattributes_global
 9│   ├── .gitconfig
10│   └── .gitignore_global
11├── gopass/
12│   └── .config/
13│       └── gopass/
14├── swhkd/
15│   └── .config/
16│       └── sxhkd/
17├── vale/
18│   ├── .config/
19│   │   └── vale/
20│   └── .vale.ini
21└── wakatime/
22    ├── org/
23    │   └── .wakatime-project
24    └── .wakatime.cfg

Everything within a package is applied relative to the root. By default this is the parent folder of your repository so the suggestion is to put your repository at ~/.dotfiles/. This can be configured so I have it pointing to $HOME to avoid any confusion.

If the folder can be symlinked when stowing it will be. If there are other files that it doesn’t manage that prevent this then it will link the individual files. It can also be configured to always link files and not directories.

Encryption and Templating

Stow on it’s own handled all the configuration synchronization and simplified the workflow. Once a file or folder was stowed, any changes were made in the repo and all that I need to do is git commit && git push to capture the changes and git pull on my other machines to retrieve them.

All that was left on my list of requirements was encryption and ideally templating. One of my main interests with encryption was to be able to keep my GPG and SSH keys in sync. Migrating to a new machine but having to retrieve your keys means setting up the machine takes extra steps, especially if your old machine is not accessible. Age allows for both key based and password based encryption and doesn’t rely on existing keys so it felt like a great fit for managing my in-repo encryption. The Chezmoi FAQ even has a guide on how to keep your age key in the repo while encrypted so that you only need to decrypt it once and then all other encryption is transparent. This just left the actual encrypt/decrypt process to figure out. Symlinks meant I couldn’t decrypt while applying the configuration the way Chezmoi did but rather had to keep the configuration in a decrypted state locally but ensure it was encrypted when pushed to the repository.

The final piece of the puzzle was git clean/smudge filters¹ combined with git diff filters² because Age encryption is not deterministic³.

1git config filter.ageEncrypt.clean  "age -a -R key.pub"
2git config filter.ageEncrypt.smudge "age -d -i key.txt"
3git config diff.ageEncrypt.textconv "cat"

When the file gets checked out (smudged) it decrypts the content and then when committing it encrypts (cleans) it. Using cat as the diff filter compares the raw encrypted version to itself if there are no changes, avoiding the need to re-encrypt unless the contents actually changed.

The only templated value I have found myself needing so far while moving my configuration over is the machine hostname. Simple sed replacements let me put in a template string then remove it.

1git config filter.setHostname.clean  'sed -e "s/$(hostname)/<<hostname>>/g"'
2git config filter.setHostname.smudge 'sed -e "s/<<hostname>>/$(hostname)/g"'

Setup Script

Because I will not need to perform setup tasks very often, other than when adding new templating filters, I wrote a a justfile that can handle all the setup tasks to get the local copy of the dotfiles repo configured and ready to use.

The unlock_key task decrypts the key (key.txt.age is the encrypted version of the key) while the install_git_filters and adds applicable git filters and re_checkout_git resets the entire repository to ensure the filters are applied. This simplifies the setup and reduces the risk of errors. And if I am on a machine that does not need the filtered content I can still use all the other configuration without issue.

default:
    @just --list

install_git_filters:
    @git config filter.ageEncrypt.clean   "age -a -R key.pub"
    @git config filter.ageEncrypt.smudge  "age -d -i key.txt"
    @git config diff.ageEncrypt.textconv  "cat"
    @git config filter.setHostname.clean  'sed -e "s/$(hostname)/<<hostname>>/g"'
    @git config filter.setHostname.smudge 'sed -e "s/<<hostname>>/$(hostname)/g"'

re_checkout_git:
    @git checkout -f

unlock_key:
    @[ -f key.txt ] || age -d -o key.txt key.txt.age

setup:
    just unlock_key
    just install_git_filters
    just re_checkout_git'

Future Additions

Managing my dotfiles and configuration only goes so far. Having my terminal and Tiling Window Manager setups loaded is of little use if the software isn’t installed to go along with it. I also need to clone additional repositories for content that isn’t being kept with the dotfiles, which Chezmoi used to manage.

There are 2 options that stand out to me for handling my user setup end-to-end: Nix Home Manager and Guix Home. Nix is probably the more commonly referenced one however I intend to start by taking another look at Guix (I had tried it prior to using Chezmoi but never got fully comfortable with it). Based on some research Nix has a complex Domain-Specific Language (DSL) for managing a system while Guix uses Guile⁴. I’m already familiar with Lisp through my use and configuration of Emacs so I’m not worried about the learning curve. It also has direct integration with Stow for handling dotfiles which means my current setup can work for systems that include Guix as well as for those where I only need my dotfiles and nothing else.