Uploaded image for project: 'OpenVZ'
  1. OpenVZ
  2. OVZ-4943

vzctl enter <VEID> hangs with bash 4.2

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Fix Version/s: OpenVZ-legacy
    • Component/s: Containers::Userspace
    • Security Level: Public
    • Environment:
      Operating System: RHEL/CentOS 5
      Platform: x86 (i386)

      Description

      Hi!

      Problem:

      When running "vzctl enter <VEID>" the command will hang with latest
      version of bash i.e 4.2.7 ( happens with all 4.2.x versions )

      I first thought this was bash bug and filed a report to the maintainer
      of bash (Chet Ramey) but got back the response below which indicates this
      might be a bug in vzctl instead,

      Let me know if you need any more information.

      There is also a thread discussing this issue at:

      https://groups.google.com/group/gnu.bash.bug
      https://groups.google.com/group/gnu.bash.bug/browse_thread/thread/0be5df8f41c8b88c#

      //Michael


      > Hi Chet!
      >
      > I run 3 different strace sessions (see attached file)
      >
      > 1. A working session (bash 3.2.25)
      >
      > # strace -ff -o /tmp/bash_strace/bash_working/bash_working.log vzctl enter 152
      >
      > 2. A failing session (bash 4.2.7):
      >
      > # strace -ff -o /tmp/bash_strace/bash_not_working/bash_not_working.log
      > vzctl enter 152
      >
      > for the failing session pstree shows:
      >
      > bash(23067)---strace(23230)---vzctl(23231)---vzctl(23232)---bash(23233)

      I suspect this is a bug in vzctl that was masked by bash-4.1 and previous
      versions.

      The only change of any significance here is that bash-4.1 closed file
      descriptors 3-20 at startup. That's a bug; you can't close fds out
      from under libraries like that. This caused mysterious crashes on Mac
      OS X, for example when running bash as a login shell under iTerm.
      Bash-4.2 sets the fds to close-on-exec instead.

      The problem is that vzctl plays fast and loose with file descriptors.
      It leaves read and write ends of pipes open in the child process it
      forks to exec bash when it uses the other ends internally to communicate
      with that child through the pty it opens as the controlling terminal.
      The big difference between the non-working and working versions is that
      bash-4.2 inherits file descriptors 3, 7, 9, and 10 and leaves them open,
      where bash-4.1 closed them.

      This results in the the process group that bash-4.2 is using being
      orphaned, which makes read() return EOF and the kernel send SIGHUP and
      SIGCONT to bash. This is consistent with the strace output.

      You can test this by changing shell.c to call close(i) instead of
      SET_CLOSE_ON_EXEC(i) around like 541. That's just to prove vzctl has
      a bug, however -- I'm not going to revert that change.

      Keep in mind that I haven't looked at the vzctl source code, and so don't
      have any patches for it. Somehow, though, the file descriptors that
      get closed in process 23231 after forking 23232 (in the bash-not-working
      set of traces, fds 3,7,9,10) need to get closed in 23233 after 23232 forks
      it and before it execs bash.

      Let me know how it goes. If you can make the right changes to vzctl and
      that fixes the problem, so much the better.

        Activity

        Hide
        michaelk@linuxmail.org michaelk added a comment -

        Attachment bash_strace.tar.gz has been added with description: strace sessions of vzctl enter

        Show
        michaelk@linuxmail.org michaelk added a comment - Attachment bash_strace.tar.gz has been added with description: strace sessions of vzctl enter
        Hide
        openvz-bugs@vandekamer.com Henk van de Kamer added a comment -

        I've raised the severity because when an owner of a virtual machine installs Bash 4.2 patchlevel 0 or higher, it is a DoS against the owner of the physical machine .

        There are times when it is handy to have the vzctl enter option and this bug closes that route.

        Show
        openvz-bugs@vandekamer.com Henk van de Kamer added a comment - I've raised the severity because when an owner of a virtual machine installs Bash 4.2 patchlevel 0 or higher, it is a DoS against the owner of the physical machine . There are times when it is handy to have the vzctl enter option and this bug closes that route.
        Hide
        kir Kir Kolyshkin added a comment -

        Guys,

        Fix committed into GIT, lightly tested here, works fine for me.
        Can you please test it as well?

        http://git-ovzcore.sw.ru/?p=vzctl;a=commit;h=e7c40ee697606e43d6242fd3c124e370588ae26f

        Will be available in vzctl-3.0.27 (and most probably in vzctl-3.0.26.2, too).

        Show
        kir Kir Kolyshkin added a comment - Guys, Fix committed into GIT, lightly tested here, works fine for me. Can you please test it as well? http://git-ovzcore.sw.ru/?p=vzctl;a=commit;h=e7c40ee697606e43d6242fd3c124e370588ae26f Will be available in vzctl-3.0.27 (and most probably in vzctl-3.0.26.2, too).
        Hide
        openvz-bugs@vandekamer.com Henk van de Kamer added a comment -

        I've just tested the patch and checked if it works:

        hoefnix2:~# vzctl enter 60
        entered into CT 60
        ve60:/# pacman -Q bash
        bash 4.2.008-1

        I hope this speaks for itself . Thanks for the quick patch!

        Show
        openvz-bugs@vandekamer.com Henk van de Kamer added a comment - I've just tested the patch and checked if it works: hoefnix2:~# vzctl enter 60 entered into CT 60 ve60:/# pacman -Q bash bash 4.2.008-1 I hope this speaks for itself . Thanks for the quick patch!
        Hide
        kir Kir Kolyshkin added a comment -

        Is that ArchLinux that comes with bash-4.2? Are you aware of any other distros already using it? I mean, I am trying to assess does it makes sense to release vzctl-3.0.36.2 immediately, or can it wait (will I have some more fixes)?

        Show
        kir Kir Kolyshkin added a comment - Is that ArchLinux that comes with bash-4.2? Are you aware of any other distros already using it? I mean, I am trying to assess does it makes sense to release vzctl-3.0.36.2 immediately, or can it wait (will I have some more fixes)?
        Hide
        yettyn@astrocalc.com yettyn added a comment -

        (In reply to comment #4)
        > Is that ArchLinux that comes with bash-4.2? Are you aware of any other distros
        > already using it? I mean, I am trying to assess does it makes sense to release
        > vzctl-3.0.36.2 immediately, or can it wait (will I have some more fixes)?

        Gentoo also provides it and yes I (and other Gentoo users) would very much need vzctl-3.0.36.2, thank you.

        Show
        yettyn@astrocalc.com yettyn added a comment - (In reply to comment #4) > Is that ArchLinux that comes with bash-4.2? Are you aware of any other distros > already using it? I mean, I am trying to assess does it makes sense to release > vzctl-3.0.36.2 immediately, or can it wait (will I have some more fixes)? Gentoo also provides it and yes I (and other Gentoo users) would very much need vzctl-3.0.36.2, thank you.
        Hide
        openvz-bugs@vandekamer.com Henk van de Kamer added a comment -

        I'm indeed using Arch Linux for my servers because it is easier to upgrade with small changes than once per two year with very big changes .

        If I had to make the decission, I probbably would wait. People who use bleeding edge like Arch Linux – and there I'm probably one of few who uses this distribution – or Gentoo will find the solution in this bugfix. So I don't see a big hurry. I raised the importance because there was no solution and could raise on DoS against the owner of the physical machine. It has nothing to do with the importance of a new version of vzctl .

        Show
        openvz-bugs@vandekamer.com Henk van de Kamer added a comment - I'm indeed using Arch Linux for my servers because it is easier to upgrade with small changes than once per two year with very big changes . If I had to make the decission, I probbably would wait. People who use bleeding edge like Arch Linux – and there I'm probably one of few who uses this distribution – or Gentoo will find the solution in this bugfix. So I don't see a big hurry. I raised the importance because there was no solution and could raise on DoS against the owner of the physical machine. It has nothing to do with the importance of a new version of vzctl .
        Show
        kir Kir Kolyshkin added a comment - Too late, vzctl 3.0.26.2 released http://openvz.org/pipermail/announce/2011-March/000200.html http://wiki.openvz.org/Download/vzctl/3.0.26.2
        Hide
        sergeyb Sergey Bronnikov (Inactive) added a comment -

        Bug was fixed more than one year ago and there were no complains from reporter after fix. We believe bug fix helped and mark bug as closed.

        Show
        sergeyb Sergey Bronnikov (Inactive) added a comment - Bug was fixed more than one year ago and there were no complains from reporter after fix. We believe bug fix helped and mark bug as closed.

          People

          • Assignee:
            kir Kir Kolyshkin
            Reporter:
            michaelk@linuxmail.org michaelk
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: