Thanks for reaching out.
Hi,
I'm working with a pod where 2 containers depend on one another. I'm
using podman v4.3.0 and the pod is started using systemd
(podman-kube@.service).
When one of the containers is in an improper state (crashes, stops
unexpectedly, or some other condition is not met), I would like to
restart the whole pod automatically (just restarting the one container
that stopped is not enough in my case).
Ideally, systemd would be in charge of restarting the pod (using the
Restart= directive to control precisely when it must be restarted).
So far, I've tried using the healthchecks mechanism, but while podman
correctly detects that the container is unhealthy, no further action is
taken.
Is there any way to achieve the desired result?
That is a very cool use case, thanks for elaborating on it. At the time of writing, it is not possible with Kube. The restart policy in the Kubernetes YAML only relates to the containers only but not to the entire pod. Hence, when container A has a failure and gets restarted, other containers won't be touched. It would be nice to integrate health checks into the Kubernetes support in the future. Feel free to open an issue on
github.com/containers/podman for that.
But there may be a solution (I see space for improvement though) when using Podman pods outside the context of `podman kube play` and Kubernetes. It's a big complicated at first glance, so I'll go through step by step:
- `podman generate systemd --new` can create systemd units for a pod and its containers.
- `podman pod create --name=foo --exit-policy=stop` creates a pod called "foo" that will "stop" once all containers have exited. We can exploit that in the systemd unit.
- `podman create --pod=foo [...]` creates a container inside the pod "foo". This way, you can add your two containers to the pod.
- Starting with Podman v4.3, there is a cool new feature called "on-failure actions" for health checks [1]. These allow, among other things, to kill a container once it turns "unhealthy".
- So it's quite easy to deal with container A. How to deal with container B really depends on the use case. Maybe there's a way to detect inside container B if container A is still doing its job (e.g., via health check) and do the same trick. `podman create` has a `--requires` flag but that only works when starting containers but does not take stopping or restarting into account yet. I am pulling in @Matt Heon to discuss this use case.
- If you want to run these things in systemd, you can use `podman generate systemd --new foo`. In that case, you should use the "kill" on-failure action. If you want to run it outside the context of systemd, I recommend using the "restart" action.
I hope this gives you enough pointers for a working solution.
Kind regards,
Valentin
Regards,
François
_______________________________________________
Podman mailing list -- podman@lists.podman.io
To unsubscribe send an email to podman-leave@lists.podman.io