concurrent podman invocations hang
Hi, I'm using podman 2.1.1 and noticed an odd behavior of podman. I created a test image that takes several minutes to stop when the container got signaled to stop. So when I call podman stop with a long timeout the call hangs a few minutes until the container stops. When I use a second terminal while the podman stop call is wating for the container to stop some podman calls hang as well. For example a podman inspect on the same container or a simple podman ps hangs. If I use ctrl-c to kill the earlier podman stop call then those hanging commands continue straight away. Are things supposed to behave that way? regards, Hendrik
Hi Hendrik, Thanks for reaching out! On Thu, Nov 26, 2020 at 11:38 PM Hendrik Haddorp <hendrik.haddorp@gmx.net> wrote:
Hi,
I'm using podman 2.1.1 and noticed an odd behavior of podman. I created a test image that takes several minutes to stop when the container got signaled to stop. So when I call podman stop with a long timeout the call hangs a few minutes until the container stops. When I use a second terminal while the podman stop call is wating for the container to stop some podman calls hang as well. For example a podman inspect on the same container or a simple podman ps hangs. If I use ctrl-c to kill the earlier podman stop call then those hanging commands continue straight away. Are things supposed to behave that way?
Yes, that's expected behaviour. The container is locked while it's being stopped. The lock blocks other commands from using the container. I am optimistic that we can improve the locking and introduce an intermediate state between "running" and "stopped" and unlock the container. I am pulling Matt Heon who likely has the best understanding how we can achieve it. Matt, what do you think? Kind regards, Valentin
regards, Hendrik _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
Thanks, I have also opened https://github.com/containers/podman/issues/8501 in the meantime as suggested by Daniel Walsh. On 11/30/20 11:19 AM, Valentin Rothberg wrote:
Hi Hendrik,
Thanks for reaching out!
On Thu, Nov 26, 2020 at 11:38 PM Hendrik Haddorp <hendrik.haddorp@gmx.net <mailto:hendrik.haddorp@gmx.net>> wrote:
Hi,
I'm using podman 2.1.1 and noticed an odd behavior of podman. I created a test image that takes several minutes to stop when the container got signaled to stop. So when I call podman stop with a long timeout the call hangs a few minutes until the container stops. When I use a second terminal while the podman stop call is wating for the container to stop some podman calls hang as well. For example a podman inspect on the same container or a simple podman ps hangs. If I use ctrl-c to kill the earlier podman stop call then those hanging commands continue straight away. Are things supposed to behave that way?
Yes, that's expected behaviour. The container is locked while it's being stopped. The lock blocks other commands from using the container.
I am optimistic that we can improve the locking and introduce an intermediate state between "running" and "stopped" and unlock the container. I am pulling Matt Heon who likely has the best understanding how we can achieve it.
Matt, what do you think?
Kind regards, Valentin
regards, Hendrik _______________________________________________ Podman mailing list -- podman@lists.podman.io <mailto:podman@lists.podman.io> To unsubscribe send an email to podman-leave@lists.podman.io <mailto:podman-leave@lists.podman.io>
On Mon, 2020-11-30 at 11:19 +0100, Valentin Rothberg wrote:
Hi Hendrik,
Thanks for reaching out!
On Thu, Nov 26, 2020 at 11:38 PM Hendrik Haddorp < hendrik.haddorp@gmx.net> wrote:
Hi,
I'm using podman 2.1.1 and noticed an odd behavior of podman. I created a test image that takes several minutes to stop when the container got signaled to stop. So when I call podman stop with a long timeout the call hangs a few minutes until the container stops. When I use a second terminal while the podman stop call is wating for the container to stop some podman calls hang as well. For example a podman inspect on the same container or a simple podman ps hangs. If I use ctrl-c to kill the earlier podman stop call then those hanging commands continue straight away. Are things supposed to behave that way?
Yes, that's expected behaviour. The container is locked while it's being stopped. The lock blocks other commands from using the container.
i agree. there are actually several operations like stop where this can happen.
I am optimistic that we can improve the locking and introduce an intermediate state between "running" and "stopped" and unlock the container. I am pulling Matt Heon who likely has the best understanding how we can achieve it.
Matt, what do you think?
Kind regards, Valentin
regards, Hendrik _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
On 2020-11-30 11:19, Valentin Rothberg wrote:
Hi Hendrik,
Thanks for reaching out!
On Thu, Nov 26, 2020 at 11:38 PM Hendrik Haddorp <hendrik.haddorp@gmx.net> wrote:
Hi,
I'm using podman 2.1.1 and noticed an odd behavior of podman. I created a test image that takes several minutes to stop when the container got signaled to stop. So when I call podman stop with a long timeout the call hangs a few minutes until the container stops. When I use a second terminal while the podman stop call is wating for the container to stop some podman calls hang as well. For example a podman inspect on the same container or a simple podman ps hangs. If I use ctrl-c to kill the earlier podman stop call then those hanging commands continue straight away. Are things supposed to behave that way?
Yes, that's expected behaviour. The container is locked while it's being stopped. The lock blocks other commands from using the container.
I am optimistic that we can improve the locking and introduce an intermediate state between "running" and "stopped" and unlock the container. I am pulling Matt Heon who likely has the best understanding how we can achieve it.
Matt, what do you think?
The reason for the lock is to ensure that no other operations on the container can happen at the same time - we don't want a user to be able to, say, checkpoint a container that is already in the process of being stopped. We could probably work around this with an intermediate Stopping state, as Valentin said, which would not allow any API calls against the container except for basic status queries. It shouldn't be that bad to add the extra state, but it could be a bit of work to ensure that it only allows the right set of calls. Would you mind opening an RFE on our Github to track this? Thanks, Matt Heon
Kind regards, Valentin
regards, Hendrik _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
Hi, I had already opened https://github.com/containers/podman/issues/8501 On November 30, 2020 3:31:45 PM GMT+01:00, Matt Heon <mheon@redhat.com> wrote:
On 2020-11-30 11:19, Valentin Rothberg wrote:
Hi Hendrik,
Thanks for reaching out!
On Thu, Nov 26, 2020 at 11:38 PM Hendrik Haddorp <hendrik.haddorp@gmx.net> wrote:
Hi,
I'm using podman 2.1.1 and noticed an odd behavior of podman. I created a test image that takes several minutes to stop when the container got signaled to stop. So when I call podman stop with a long timeout the call hangs a few minutes until the container stops. When I use a second terminal while the podman stop call is wating for the container to stop some podman calls hang as well. For example a podman inspect on the same container or a simple podman ps hangs. If I use ctrl-c to kill the earlier podman stop call then those hanging commands continue straight away. Are things supposed to behave that way?
Yes, that's expected behaviour. The container is locked while it's being stopped. The lock blocks other commands from using the container.
I am optimistic that we can improve the locking and introduce an intermediate state between "running" and "stopped" and unlock the container. I am pulling Matt Heon who likely has the best understanding how we can achieve it.
Matt, what do you think?
The reason for the lock is to ensure that no other operations on the container can happen at the same time - we don't want a user to be able to, say, checkpoint a container that is already in the process of being stopped. We could probably work around this with an intermediate Stopping state, as Valentin said, which would not allow any API calls against the container except for basic status queries.
It shouldn't be that bad to add the extra state, but it could be a bit of work to ensure that it only allows the right set of calls. Would you mind opening an RFE on our Github to track this?
Thanks, Matt Heon
Kind regards, Valentin
regards, Hendrik _______________________________________________ Podman mailing list -- podman@lists.podman.io To unsubscribe send an email to podman-leave@lists.podman.io
participants (4)
-
Brent Baude -
Hendrik Haddorp -
Matt Heon -
Valentin Rothberg