9:53 a.m.
Hi all. Late last year, we managed to deploy one of our web applications as a rootless
Podman container and a systemd user service on a Ubuntu 22.04 VPS, using Podman 4.2.0,
also building a full GitLab CI pipeline around it.
Now Podman 4.4.0 is out, featuring Quadlet. There's no mention of the release on
podman.io, let alone any docs for the Podman-integrated version of Quadlet that I can
find. (Any ETA on these? I only realized it was out thanks to this mailing list.)
Below I'll describe our deployment workflow and show an annotated version of the
systemd service we use. What I'd like to know is: is there a way of using Quadlet for
this use case? Would it be an improvement over the current setup in terms of using
systemd/Podman "optimally"?
It would especially be nice if Quadlet could give us a working Type=notify unit file, to
eliminate the need for PID files. I was previously unable to get it to work (I no longer
remember why), and had to use Type=forking instead.
- - - - -
Container deployment is done with an Ansible playbook, run by GitLab CI. It connects to
the VPS, updates the image from our private registry, builds an entirely new container out
of it with a unique suffix (appname-<imagetag>_<githash>) and templates out a
unit file with a matching name (the Jinja2 template is based on 'podman generate
systemd' output). The old systemd service is stopped and disabled, and the new one
started and enabled.
We do this because we want deploys to be "atomic", minimizing downtime. Building
a new container instead of updating the old one lets us quickly revert to the previous
version if the new container is faulty somehow. (Old containers and their unit files are
eventually removed with a cron job.)
Here's the unit file, with some annotations about our changes:
[Unit]
Description=AppName
Wants=network-online.target
After=network-online.target
RequiresMountsFor=%t/containers
[Service]
# No Environment=PODMAN_SYSTEMD_UNIT=%n clause, because we don't use
podman-auto-update
Restart=on-failure
TimeoutStopSec=10
# Don't try to start unless the container exists that this service is named after.
ExecCondition=/bin/bash -c "/usr/bin/podman container inspect %N -f
'{{.State.Status}}' 2> /dev/null"
# Symlink the PID file to a more convenient location (not strictly necessary,
# looks nicer here in PIDFile and ExecStopPost).
ExecStartPre=/bin/bash -c "ln -sf \
$XDG_RUNTIME_DIR/containers/vfs-containers/$(podman container inspect %N -f
'{{.Id}}')/userdata/conmon.pid \
$XDG_RUNTIME_DIR/.%N_conmon.pid"
# Type=notify would be nicer
Type=forking
PIDFile=%t/.%N_conmon.pid
ExecStart=/usr/bin/podman start %N
# This pattern of running 'podman stop' in both ExecStop and ExecStopPost
# is from podman-generate-systemd, but I never understood the reasoning for it.
ExecStop=/usr/bin/podman stop --ignore -t 10 %N
ExecStopPost=/usr/bin/podman stop --ignore -t 10 %N
# Clean up the PID file symlink
ExecStopPost=/bin/rm -f $XDG_RUNTIME_DIR/.%N_conmon.pid
[Install]
# Saves us from having to deal with the full unit name with the image tag and the
# git hash; the symlink to this name is replaced to point to the new unit file
# during the Ansible deployment.
Alias=appname.service
WantedBy=default.target
- - - - -
This is pretty robust for our purposes, but my systemd and overall Podman knowledge is
limited, so I don't know what I could be doing better. Quadlet has a rather different
philosophy overall than what we're used to, but can it be leveraged in this workflow,
for CI-driven replacements of rootless containers running as systemd user services?