Uploaded image for project: 'OpenVZ'
  1. OpenVZ
  2. OVZ-4888

UBCs contain CTIDs that have are deleted unmounted down, causes vzmemcheck to return incorrect data

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Fix Version/s: OpenVZ-legacy
    • Component/s: Containers::Userspace
    • Security Level: Public
    • Environment:
      Operating System: RHEL/CentOS 5
      Platform: x86_64 (AMD64)

      Description

      Attempting to calculate utilisation, commitment & limit indices for our OpenVZ HNs is inaccurate given that vzmemcheck is including containers that are deleted:

      pdzwart@atlassian45:~/OpenVZ[23:16:08](0,0)$ sudo /usr/sbin/vzlist
      Container(s) not found
      pdzwart@atlassian45:~/OpenVZ[23:16:32](0,1)$ sudo /usr/sbin/vzmemcheck -v
      Output values in %
      veid LowMem LowMem RAM MemSwap MemSwap Alloc Alloc Alloc
                    util commit util util commit util commit limit
      1031 0.00 13.13 0.00 0.00 12.70 0.00 12.70 57.78
      1005 0.00 13.13 0.00 0.00 12.70 0.00 12.70 57.78
      1010 0.00 13.13 0.00 0.00 12.70 0.00 12.70 57.78
      1017 0.00 13.13 0.00 0.00 12.70 0.00 12.70 57.78
      1016 0.00 13.13 0.00 0.00 12.70 0.00 12.70 57.78
      1015 0.00 13.13 0.00 0.00 12.70 0.00 12.70 57.78
      1014 0.00 13.13 0.00 0.00 12.70 0.00 12.70 57.78
      1013 0.00 13.13 0.00 0.00 12.70 0.00 12.70 57.78
      1012 0.00 13.13 0.00 0.00 12.70 0.00 12.70 57.78
      1009 0.00 13.13 0.00 0.00 12.70 0.00 12.70 57.78
      1008 0.00 13.13 0.00 0.00 12.70 0.00 12.70 57.78
      -------------------------------------------------------------------------
      Summary: 0.00 144.38 0.00 0.00 139.75 0.00 139.75 635.53
      pdzwart@atlassian45:~/OpenVZ[23:16:35](0,0)$ sudo /usr/sbin/vzctl status 1031
      CTID 1031 deleted unmounted down
      pdzwart@atlassian45:~/OpenVZ[23:21:26](0,0)$

      There are also entries in both /proc/user_beancounters and /proc/bc/${CTID}/ that show allocations held:

      pdzwart@atlassian45:~/OpenVZ[23:24:56](0,0)$ export CTID=1010
      pdzwart@atlassian45:~/OpenVZ[23:25:44](0,0)$ sudo egrep -A23 "${CTID}:" /proc/user_beancounters
           1010: kmemsize 45366 20936957 841784627 925963089 0
                  lockedpages 0 0 41102 41102 0
                  privvmpages 0 1438840 4476066 4923672 0
                  shmpages 0 3872 447606 447606 0
                  dummy 0 0 0 0 0
                  numproc 0 321 20550 20550 0
                  physpages 0 685472 0 9223372036854775807 0
                  vmguarpages 0 0 746011 9223372036854775807 0
                  oomguarpages 0 685472 746011 9223372036854775807 0
                  numtcpsock 0 430 20550 20550 0
                  numflock 0 10 1000 1100 0
                  numpty 0 6 512 512 0
                  numsiginfo 0 11 1024 1024 0
                  tcpsndbuf 156608 30611240 196422075 280594875 0
                  tcprcvbuf 0 117644640 196422075 280594875 0
                  othersockbuf 0 295872 98211037 182383837 0
                  dgramrcvbuf 0 28400 98211037 98211037 0
                  numothersock 0 41 20550 20550 0
                  dcachesize 3756 62913 183872621 189388800 0
                  numfile 0 7046 328800 328800 0
                  dummy 0 0 0 0 0
                  dummy 0 0 0 0 0
                  dummy 0 0 0 0 0
                  numiptent 0 14 200 200 0
      pdzwart@atlassian45:~/OpenVZ[23:25:48](0,0)$ sudo cat /proc/bc/${CTID}/resources
                  kmemsize 45366 20936957 841784627 925963089 0
                  lockedpages 0 0 41102 41102 0
                  privvmpages 0 1438840 4476066 4923672 0
                  shmpages 0 3872 447606 447606 0
                  numproc 0 321 20550 20550 0
                  physpages 0 685472 0 9223372036854775807 0
                  vmguarpages 0 0 746011 9223372036854775807 0
                  oomguarpages 0 685472 746011 9223372036854775807 0
                  numtcpsock 0 430 20550 20550 0
                  numflock 0 10 1000 1100 0
                  numpty 0 6 512 512 0
                  numsiginfo 0 11 1024 1024 0
                  tcpsndbuf 156608 30611240 196422075 280594875 0
                  tcprcvbuf 0 117644640 196422075 280594875 0
                  othersockbuf 0 295872 98211037 182383837 0
                  dgramrcvbuf 0 28400 98211037 98211037 0
                  numothersock 0 41 20550 20550 0
                  dcachesize 3756 62913 183872621 189388800 0
                  numfile 0 7046 328800 328800 0
                  numiptent 0 14 200 200 0
                  swappages 0 0 9223372036854775807 9223372036854775807 0
      pdzwart@atlassian45:~/OpenVZ[23:25:58](0,0)$ sudo /usr/sbin/vzctl status ${CTID}
      CTID 1010 deleted unmounted down
      pdzwart@atlassian45:~/OpenVZ[23:26:04](0,0)$

      Package versions are as follows:

      pdzwart@atlassian45:~/OpenVZ[23:26:26](0,1)$ rpm -qa |grep vz
      vzctl-lib-3.0.24.1-1
      vzpkg-2.7.0-18
      vztmpl-fedora-9-1.1-1
      vzrpm44-4.4.1-22.5
      vztmpl-fedora-core-3-2.0-2
      vzctl-3.0.24.1-1
      vzrpm43-python-4.3.3-7_nonptl.6
      vztmpl-centos-5-2.0-3
      vztmpl-fedora-core-6-1.2-1
      vzquota-3.0.12-1
      vzrpm44-python-4.4.1-22.5
      vztmpl-centos-4-2.0-2
      vztmpl-fedora-core-5-2.0-2
      vzrpm43-4.3.3-7_nonptl.6
      vztmpl-fedora-core-4-2.0-2
      ovzkernel-2.6.18-194.8.1.el5.028stab070.2
      vzyum-2.4.0-11
      vztmpl-fedora-7-1.1-1
      pdzwart@atlassian45:~/OpenVZ[23:26:36](0,0)$

      Our workaround is a python script that parses /proc/user_beancounters correlating running containers from a vzlist execution.

        Attachments

          Activity

            People

            Assignee:
            kir Kir Kolyshkin
            Reporter:
            pdzwart@atlassian.com Pete de Zwart
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: