Uploaded image for project: 'OpenVZ'
  1. OpenVZ
  2. OVZ-6790

can't restore bindmount on socket-over-tmpfs /dev/log (MariaDB)

    XMLWordPrintable

    Details

      Description

      >Description of problem:
      Can't resume container after suspend

      >Steps to Reproduce:
      1. Create new container with last CentOS 7 template (centos-7-x86_64.tar.gz )
      2. Install MariaDB server from it repository and start it.
      https://downloads.mariadb.org/mariadb/repositories/#mirror=best&distro=CentOS&distro_release=centos7-amd64--centos7&version=10.1

      > yum install MariaDB-server MariaDB-client
      > systemctl start mariadb
      3. Suspend container
      vzctl suspend <CTID>
      4. Resume container
      vzctl resume <CTID>

      >Actual results:
      [root@localhost]# vzctl suspend 111
      Setting up checkpoint...
      suspend...
      dump...
      kill...
      Checkpointing completed successfully
      Unmounting file system at /vz/root/111
      Unmounting device /dev/ploop60715
      Container is unmounted
      [root@localhost]# vzctl resume 111
      Restoring container ...
      Opening delta /vz/private/111/root.hdd/root.hdd
      Adding delta dev=/dev/ploop60715 img=/vz/private/111/root.hdd/root.hdd (rw)
      Mounting /dev/ploop60715p1 at /vz/root/111 fstype=ext4 data='balloon_ino=12,'
      Container is mounted
      undump...
      Adding IP address(es): 10.1.0.1
      Setting CPU units: 1000
      Restore error, undump failed: No such file or directory
      Error: bindmount lookup failed: @14648 /dev/log
      Error: Failed to restore mount point @135072 dev 'devtmpfs', type 'devtmpfs', path '/dev/log'
      Error: rst_namespace: -2
      Error: BUG: ctx->objcount=1
      Container restore failed
      Unmounting file system at /vz/root/111
      Unmounting device /dev/ploop60715
      Container is unmounted

      >Host OS:
      CentOS 6, 042stab117.6

      >Guest OS:
      CentOS 7, centos-7-x86_64.tar.gz

      In my opinion, problem in mount namespaces, that created by systemd for mysqld process.

      [root@localhost /]# ps aux|grep mysql
      mysql 148 0.6 34.4 835972 90248 ? Ssl 07:06 0:00 /usr/sbin/mysqld
      root 461 0.0 0.3 8992 812 pts/0 S+ 07:07 0:00 grep --color=auto mysql
      [root@localhost /]# cat /proc/148/mountinfo
      242 241 0:63 / /proc rw,relatime shared:73 - proc proc rw
      243 241 0:61 / /sys rw,relatime shared:74 - sysfs sysfs rw
      244 243 0:68 / /sys/fs/cgroup ro,nosuid,nodev,noexec shared:75 - tmpfs tmpfs ro,mode=755
      245 244 0:31 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:76 - cgroup cgroup rw,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd
      246 244 0:14 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime shared:77 - cgroup cgroup rw,blkio,name=beancounter
      247 244 0:15 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime shared:78 - cgroup cgroup rw,memory
      248 241 0:60 / /dev rw,nosuid shared:79 - devtmpfs devtmpfs rw,mode=755
      249 248 0:66 / /dev/shm rw,nosuid,nodev shared:80 - tmpfs tmpfs rw
      250 248 0:65 / /dev/pts rw,nosuid,noexec,relatime shared:81 - devpts devpts rw,gid=5,mode=620,ptmxmode=000
      251 248 0:62 / /dev/mqueue rw,relatime shared:82 - mqueue mqueue rw
      252 241 0:67 / /run rw,nosuid,nodev shared:89 - tmpfs tmpfs rw,mode=755
      241 240 182:971441 / / rw,relatime shared:72 - ext4 /dev/ploop60715p1 rw,barrier=1,data=ordered,balloon_ino=12
      253 248 0:69 / /dev rw,nosuid,relatime shared:84 - tmpfs tmpfs rw,mode=755,gid=27
      254 253 0:65 / /dev/pts rw,nosuid,noexec,relatime shared:85 - devpts devpts rw,gid=5,mode=620,ptmxmode=000
      255 253 0:66 / /dev/shm rw,nosuid,nodev,relatime shared:86 - tmpfs tmpfs rw
      256 253 0:62 / /dev/mqueue rw,relatime shared:87 - mqueue mqueue rw
      257 253 0:60 /log /dev/log rw,nosuid,relatime shared:88 - devtmpfs devtmpfs rw,mode=755
      258 241 0:67 /systemd/inaccessible /home ro,nosuid,nodev,relatime shared:91 - tmpfs tmpfs rw,mode=755
      259 241 0:67 /systemd/inaccessible /root ro,nosuid,nodev,relatime shared:92 - tmpfs tmpfs rw,mode=755
      260 252 0:67 /systemd/inaccessible /run/user ro,nosuid,nodev,relatime shared:90 - tmpfs tmpfs rw,mode=755
      261 241 182:971441 /boot /boot ro,relatime shared:93 - ext4 /dev/ploop60715p1 rw,barrier=1,data=ordered,balloon_ino=12
      262 241 182:971441 /etc /etc ro,relatime shared:94 - ext4 /dev/ploop60715p1 rw,barrier=1,data=ordered,balloon_ino=12
      263 241 182:971441 /usr /usr ro,relatime shared:95 - ext4 /dev/ploop60715p1 rw,barrier=1,data=ordered,balloon_ino=12

      Mount with a problem at this line:
      257 253 0:60 /log /dev/log rw,nosuid,relatime shared:88 - devtmpfs devtmpfs rw,mode=755

      For older version MariaDB from official centos repository problem does not occurs. This mounts appends by systemd. If disable PrivateDevices=true option in mysqld.service (/etc/systemd/system/mysqld.service), resuming completed fine.

      Also if make snapshot several times for this container and stop it, kernel hung with soft lockup. I described this at https://bugs.openvz.org/browse/OVZ-6779

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              vvs Vasily Averin
              Reporter:
              butov Sergei Butov
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: