Logging to journald from arbitrary processes, not just the main process
I've got a bunch of Quadlet-based containers with LogDriver=journald. This correctly captures the stdout/stderr of the main process of the container, and the process output is then viewable using journalctl on the host side. We now have a need to run cron jobs in one of our containers. We're running `podman exec containername python /path/to/script.py` as the job on the host, but the stdout prints from the script are not forwarded to the logging driver. I see two choices: 1. Run the cron scripts as short-lived containers of their own, making them the main process for the duration and presumably causing the journald driver to capture their stdout as expected. IIRC there's a way for systemd to replace cron; what would be the correct method of using Quadlet-based containers here? (We would create a separate quadlet config per script.) 2. Somehow enable arbitrary script runs with `podman exec` against a running container to interface with the journald driver. Is this possible to do?
I think perhaps what you're reaching for is systemd timers. They can run whatever command you like (including containers) and since you already have the journald bits configured the logging part should "just work". See: man systemd.timer --- Chris Evich (he/him) Senior Quality Assurance Engineer If there's a "hard-way", I'm the first one to implement it. On 8/6/24 09:39, jklaiho@iki.fi wrote:
I've got a bunch of Quadlet-based containers with LogDriver=journald. This correctly captures the stdout/stderr of the main process of the container, and the process output is then viewable using journalctl on the host side.
We now have a need to run cron jobs in one of our containers. We're running `podman exec containername python /path/to/script.py` as the job on the host, but the stdout prints from the script are not forwarded to the logging driver.
I see two choices:
1. Run the cron scripts as short-lived containers of their own, making them the main process for the duration and presumably causing the journald driver to capture their stdout as expected. IIRC there's a way for systemd to replace cron; what would be the correct method of using Quadlet-based containers here? (We would create a separate quadlet config per script.)
2. Somehow enable arbitrary script runs with `podman exec` against a running container to interface with the journald driver. Is this possible to do? _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
Thanks, Chris; systemd timers were indeed what I was looking for. I got a cron-like repeating Quadlet service functioning. I'll detail the process below since someone else may find this useful in the future. Before I do that: I'm still interested in how to interface with the journald logging driver from arbitrary processes inside long-running containers, instead of just the one that's Exec'd in the .container file. Any input would be welcome. - - - - - - - - Here's the setup. Worth mentioning that this is all rootless and using systemd user services, so lingering is enabled etc. ~/.config/containers/systemd/temp.container: [Unit] Description=Temporary test container [Container] Image=someimage:latest ContainerName=temp LogDriver=journald Exec=python /path/to/script.py ~/.config/systemd/user/temp.timer: [Unit] Description=Run the temp container every minute [Timer] OnCalendar=*:0/1 Persistent=true [Install] WantedBy=timers.target By default (i.e. without Unit= under [Timer]), `temp.timer` expects `temp.service` to exist as its target. Generate it from the `temp.container` Quadlet file into `$XDG_RUNTIME_DIR/systemd/generator` by running `systemctl --user daemon-reload`. Repeat whenever you change the Quadlet file. `systemctl --user start temp.timer` starts the timer. To auto-start it during reboots, `systemctl --user enable temp.timer`. The timer will show up in `systemctl --user list-timers` when started; with `--all` in case it's enabled but stopped. The process output should show up in `journalctl --user -f`.
On Tuesday, Aug 06, 2024 at 11:09 PM, Chris Evich <cevich@redhat.com (mailto:cevich@redhat.com)> wrote:
I think perhaps what you're reaching for is systemd timers. They can run whatever command you like (including containers) and since you already have the journald bits configured the logging part should "just work".
See:
man systemd.timer
--- Chris Evich (he/him) Senior Quality Assurance Engineer If there's a "hard-way", I'm the first one to implement it. _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
Nice! I am by far not a systemd or journald expert, but a very long time ago...while working with a certain other container engine...we use to volume-mount /dev/log in from the host to expose logging in the container to to the outside. Not sure if that has any relevance whatsoever today, but maybe it'll help google find you an answer (if somebody else doesn't know). Oh and thanks for the test write-up, I'm sure that will be useful to someone else. Might you consider crafting it into a short podman.io blog? If so, I can connect you with people that will get it published :) --- Chris Evich (he/him) Senior Quality Assurance Engineer If there's a "hard-way", I'm the first one to implement it. On 8/7/24 03:52, Jarkko Laiho wrote:
Thanks, Chris; systemd timers were indeed what I was looking for.
I got a cron-like repeating Quadlet service functioning. I'll detail the process below since someone else may find this useful in the future.
Before I do that: I'm still interested in how to interface with the journald logging driver from arbitrary processes inside long-running containers, instead of just the one that's Exec'd in the .container file. Any input would be welcome.
- - - - - - - -
Here's the setup. Worth mentioning that this is all rootless and using systemd user services, so lingering is enabled etc.
~/.config/containers/systemd/temp.container:
[Unit] Description=Temporary test container
[Container] Image=someimage:latest ContainerName=temp LogDriver=journald Exec=python /path/to/script.py
~/.config/systemd/user/temp.timer:
[Unit] Description=Run the temp container every minute
[Timer] OnCalendar=*:0/1 Persistent=true
[Install] WantedBy=timers.target
By default (i.e. without Unit= under [Timer]), `temp.timer` expects `temp.service` to exist as its target. Generate it from the `temp.container` Quadlet file into `$XDG_RUNTIME_DIR/systemd/generator` by running `systemctl --user daemon-reload`. Repeat whenever you change the Quadlet file.
`systemctl --user start temp.timer` starts the timer. To auto-start it during reboots, `systemctl --user enable temp.timer`.
The timer will show up in `systemctl --user list-timers` when started; with `--all` in case it's enabled but stopped. The process output should show up in `journalctl --user -f`.
On Tuesday, Aug 06, 2024 at 11:09 PM, Chris Evich <cevich@redhat.com <mailto:cevich@redhat.com>> wrote:
I think perhaps what you're reaching for is systemd timers. They can run whatever command you like (including containers) and since you already have the journald bits configured the logging part should "just work".
See:
man systemd.timer
--- Chris Evich (he/him) Senior Quality Assurance Engineer If there's a "hard-way", I'm the first one to implement it. _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
_______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
participants (3)
-
Chris Evich -
Jarkko Laiho -
jklaiho＠iki.fi