Coding with Titans

Yet again, during my engineering journey to the future I was hit by a practical problem. When I tried to build and host a docker image on a small and slow volume (precisely on an old Raspberry 3 that still boots from SD-card), I ended up very disappointed with the final experience. And even though this device seems to be a bit outdated, it’s still ideal for small hobby project with simple API served via FastAPI, Python and PostgreSQL.

First try - NOT a solution

My first approach was to attach an SSD drive into the device, mount it under /volumes/red (yes, as it has the red color case) and then use symlinks to move all Docker-related storage folders down to the new drive. It goes down to 3 folders (containers, overlay2 and volumes are places, where layers and images plus some runtime changes are stored).

I ended up having a structure at default installation location like this one (of course I copied the contents of original folders under the new paths):

$ ls /var/lib/docker/

drwx--x--- 12 root root 4096 Jan  2 11:41 .
drwxr-xr-x 41 root root 4096 Dec  1 16:37 ..
lrwxrwxrwx  1 root root   30 Dec  1 17:00 containers -> /volumes/red/docker/containers/
lrwxrwxrwx  1 root root   28 Jan  2 11:37 overlay2 -> /volumes/red/docker/overlay2/
lrwxrwxrwx  1 root root   28 Jan  2 11:41 volumes -> /volumes/red/docker/volumes/

I won’t put here commands leading to this result, cause it didn’t work at the end. Any docker command seemed to fail due to docker not supporting symlinks. For example building my sample image (within a folder with my FastAPI project and with Dockerfile in it):

$ docker build .

always ended up with the following error message:

failed to create shim task: OCI runtime create failed: runc create failed: invalid rootfs: not an absolute path, or a symlink: unknown

Revert all changes and go back to drawing board.

Working solution

In this approach let’s try to move the whole Docker installation into new folder residing on SSD drive, so we won’t need any new symlinks. It’s even easier than the previous failed one. Single command could be used to copy the original application tree (with all folder permissions and contents):

$ sudo rsync -aqxP /var/lib/docker/ /volumes/red/docker

First path of the rsync command points to source, the following is the destination folder. Additional arguments are:

• a - enables archive mode to synchronize directories recursively, while keeping the ownerships and permissions
• q - enables quiet mode to print less messages during the operation
• x - copies only from current file-system (avoids crossing boundaries to another mounted drive)
• P - enables mode to keep partially copied files, to complete it on subsequent re-run, in case of any failure in the meantime

Final step needs to be done now to make it all work flawlessly after device reboots. Since Docker is started via a dedicated daemon, we need to instruct it, where the new executable resides. The path has to be passed via -g parameter via docker.service file:

Let’s edit the file:

$ sudo vi /lib/systemd/system/docker.service

Change the line starting the docker daemon to look for binary inside /volumes/red/:

ExecStart=/usr/bin/dockerd -g /volumes/red/docker -H fd://

Reload the updated service and restart Docker:

$ sudo systemctl daemon-reload
$ sudo systemctl start docker

Many thanks to Luke Reynolds as I created mine based on his tutorial from here.

Note: If you use Docker also on Windows, I have another guide explaining, how to change the Docker files location over WSL.

🌹