Uploaded image for project: 'OpenVZ'
  1. OpenVZ
  2. OVZ-5386

BUG: soft lockup - CPU#2 stuck for 67s! : sock_alloc / new_inode / _spin_lock+0x1e/0x30

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Fix Version/s: OpenVZ-legacy
    • Component/s: Containers::Kernel
    • Security Level: Public
    • Environment:
      Operating System: Other
      Platform: x86_64 (AMD64)

      Description

      Getting a kernel panic every week:

      [root@vz ~]# uname -a
      Linux vz.xyz.com 2.6.32-042stab049.6 #1 SMP Mon Feb 6 19:17:43 MSK 2012 x86_64 x86_64 x86_64 GNU/Linux

      [root@vz ~]# cat /etc/redhat-release
      CentOS release 6.2 (Final)

      This is as much as my terminal server logs captured:

      th xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597276.142067] CPU 1
      [597276.142067] Modules linked in: vzethdev simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota ip6table_mangle nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597276.142067]
      [597276.142067] Pid: 946200, comm: set_keyserver_u veid: 1035 Not tainted 2.6.32-042stab049.6 #1 042stab049_6 empty empty/S2927
      [597276.142067] RIP: 0010:[<ffffffff814e191e>] [<ffffffff814e191e>] _spin_lock+0x1e/0x30
      [597276.142067] RSP: 0018:ffff88000babfe98 EFLAGS: 00000293
      [597276.142067] RAX: 0000000000003c92 RBX: ffff88000babfe98 RCX: 0000000000000000
      [597276.142067] RDX: 0000000000003c90 RSI: ffffffff817a1438 RDI: ffffffff823cca00
      [597276.142067] RBP: ffffffff8100bcce R08: 0000000000000000 R09: 000000000000000e
      [597276.142067] R10: 0000000000000000 R11: 0000000000000246 R12: ffffffff81042b54
      [597276.142067] R13: ffff88000babff18 R14: 0000000000000014 R15: ffff8803c5c004a8
      [597276.142067] FS: 00007f54f714b6e0(0000) GS:ffff880028280000(0000) knlGS:00000000b774f6c0
      [597276.142067] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      [597276.142067] CR2: 00007f54f66747e0 CR3: 00000002a8a5b000 CR4: 00000000000006e0
      [597276.142067] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      [597276.142067] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      [597276.142067] Process set_keyserver_u (pid: 946200, veid: 1035, threadinfo ffff88000babe000, task ffff8800ad7b3290)

      [597276.142067] Stack:
      [597276.142067] ffff88000babfec8 ffffffff811a93fb 000000001d12a781 ffff880336840980
      [597276.142067] <0> 0000000000000000 0000000000000000 ffff88000babfed8 ffffffff81410fd9
      [597276.142067] <0> ffff88000babff28 ffffffff8141143e ffff880000000001 ffff880000000002
      [597276.142067] Call Trace:
      [597276.142067] [<ffffffff811a93fb>] ? new_inode+0x3b/0xb0
      [597276.142067] [<ffffffff81410fd9>] ? sock_alloc+0x19/0x60
      [597276.142067] [<ffffffff8141143e>] ? __sock_create+0x9e/0x210
      [597276.142067] [<ffffffff81411610>] ? sock_create+0x30/0x40
      [597276.142067] [<ffffffff8141182c>] ? sys_socket+0x3c/0x80
      [597276.142067] [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
      [597276.142067] Code: 00 00 00 01 74 05 e8 12 d1 d8 ff c9 c3 55 48 89 e5 0f 1f 44 00 00 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89
      [597276.142067] Call Trace:
      [597276.142067] [<ffffffff811a93fb>] ? new_inode+0x3b/0xb0
      [597276.142067] [<ffffffff81410fd9>] ? sock_alloc+0x19/0x60
      [597276.142067] [<ffffffff8141143e>] ? __sock_create+0x9e/0x210
      [597276.142067] [<ffffffff81411610>] ? sock_create+0x30/0x40
      [597276.142067] [<ffffffff8141182c>] ? sys_socket+0x3c/0x80
      [597276.142067] [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
      [597276.206006] BUG: soft lockup - CPU#2 stuck for 67s! [update_deploy_p:946201]
      [597276.206009] Modules linked in: vzethdev simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota ip6table_mangle nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4
      mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597276.206085] CPU 2
      [597276.206087] Modules linked in: vzethdev simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota ip6table_mangle nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597276.206133]
      [597276.206133] Pid: 946201, comm: update_deploy_p veid: 1035 Not tainted 2.6.32-042stab049.6 #1 042stab049_6 empty empty/S2927
      [597276.206133] RIP: 0010:[<ffffffff814e191e>] [<ffffffff814e191e>] _spin_lock+0x1e/0x30
      [597276.206133] RSP: 0018:ffff8800ad635e98 EFLAGS: 00000297
      [597276.206133] RAX: 0000000000003c94 RBX: ffff8800ad635e98 RCX: 0000000000000000
      [597276.206133] RDX: 0000000000003c90 RSI: ffffffff817a1438 RDI: ffffffff823cca00
      [597276.206133] RBP: ffffffff8100bcce R08: 0000000000000000 R09: 0000000000000003
      [597276.206133] R10: 0000000000000000 R11: 0000000000000246 R12: ffffffff81042b54
      [597276.206133] R13: ffff8800ad635f18 R14: 0000000000000014 R15: ffff8803151d8c78
      [597276.206133] FS: 00007f4f0dc176e0(0000) GS:ffff880230400000(0000) knlGS:00000000b774f6c0
      [597276.206133] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      [597276.206133] CR2: 00007f4f0d1407e0 CR3: 00000002310a1000 CR4: 00000000000006e0
      [597276.206133] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      [597276.206133] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      [597276.206133] Process update_deploy_p (pid: 946201, veid: 1035, threadinfo ffff8800ad634000, task ffff8803d03a4800)
      [597276.206133] Stack:
      [597276.206133] ffff8800ad635ec8 ffffffff811a93fb 000000001d12a781 ffff880336840980

      [597276.206133] <0> 00 ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597276.268102] CPU 3
      [597276.268103] Modules 0000(0000) knlGS:00000000b774f6c0
      [597276.268192] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      [597276.268192] CR2: 00007f246f7067e0 CR3: 0000000407807000 CR4: 00000000000006e0
      [597276.268192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      [597276.268192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      [597276.268192] Process update_deploy_p (pid: 946188, veid: 1063, threadinfo ffff8800c232e000, task ffff8804239049c0)
      [597276.268192] Stack:
      [597276.268192] ffff8800c232fec8 ffffffff811a93fb 0000000009ffd874 ffff880225129200
      [597276.268192] <0> 0000000000000000 0000000000000000 ffff8800c232fed8 ffffffff81410fd9
      [597276.268192] <0> ffff8800c232ff28 ffffffff8141143e ffff880000000001 ffff880000000002
      [597276.268192] Call Trace:
      [597276.268192] [<ffffffff811a93fb>] ? new_inode+0x3b/0xb0
      [597276.268192] [<ffffffff81410fd9>] ? sock_alloc+0x19/0x60
      [597276.268192] [<ffffffff8141143e>] ? __sock_create+0x9e/0x210
      [597276.268192] [<ffffffff81411610>] ? sock_crestring xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597276.079003]
      [597276.079003] Pid: 390166, comm: apache2 veid: 1025 Not tainted 2.6.32-042stab049.6 #1 042stab049_6 empty empty/S2927
      [597276.079003] RIP: 0010:[<ffffffff814e191e>] [<ffffffff814e191e>] _spin_lock+0x1e/0x30
      [597276.079003] RSP: 0018:ffff8803cc2b df c9 c3 0f 1f 40 00 55 48 89

      [597276.079003] Call Trace:
      [597276.079003] [<ffffffff81261685>] ? _atomic_dec_and_lock+0x55/0x80
      [597276.079003] [<ffffffff811a77b0>] ? iput+0x30/0x70
      [597276.079003] [<ffffffff811a4028>] ? dentry_iput+0x98/0x110
      [597276.079003] [<ffffffff811a419c>] ? d_kill+0x3c/0x70
      [597276.079003] [<ffffffff811a5933>] ? dput+0xa3/0x1d0
      [597276.079003] [<ffffffff8118e9e1>] ? __fput+0x1d1/0x280
      [597276.079003] [<ffffffff8118eab5>] ? fput+0x25/0x30
      [597276.079003] [<fffffff
                                [597360.268180] RIP: 0010:[<ffffffff814e1921>] [<ffffffff814e1921>] _spin_lock+0x21/0x30
      [597360.268191] RSP: 0018:ffff8800c232fe98 EFLAGS: 00000293
      [597360.268192] RAX: 0000000000003c93 RBX: ffff8800c232fe98 RCX: 0000000000000000
      [597360.268192] RDX: 0000000000003c90 RSI: ffffffff817a1438 RDI: ffffffff823cca00
      [597360.268192] RBP: ffffffff8100bcce R08: 0000000000000000 R09: 000000000000000f
      [597360.268192] R10: 0000000000000000 R11: 0000000000000246 R12: ffffffff81042b54
      [597360.268192] R13: ffff8800c232ff18 R14: 0000000000000014 R15: ffff880336b5ce90
      [597360.268192] FS: 00007f24701dd6e0(0000) GS:ffff880230480000(0000) knlGS:00000000b774f6c0
      [597360.268192] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      [597360.268192] CR2: 00007f246f7067e0 CR3: 0000000407807000 CR4: 00000000000006e0
      [597360.268192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      [597360.268192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      [597360.268192] Process update_deploy_p (pid: 946188, veid: 1063, threadinfo ffff8800c232e000, task ffff8804239049c0)

      [597360.268192] Stack:
      [597360.268192] ffff8800c232fec8 ffffffff811a93fb 0000000009ffd874 ffff880225129200
      [597360.268192] <0> 0000000000000000 0000000000000000 ffff8800c232fed8 ffffffff81410fd9
      [597360.268192] <0> ffff8800c232ff28 ffffffff8141143e ffff880000000001 ffff880000000002
      [597360.268192] Call Trace:
      [597360.268192] [<ffffffff811a93fb>] ? new_inode+0x3b/0xb0
      [597360.268192] [<ffffffff81410fd9>] ? sock_alloc+0x19/0x60
      [597360.268192] [<ffffffff8141143e>] ? __sock_create+0x9e/0x210
      [597360.268192] [<ffffffff81411610>] ? sock_create+0x30/0x40
      [597360.268192] [<ffffffff8141182c>] ? sys_socket+0x3c/0x80
      [597360.268192] [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
      [597360.268192] Code: 01 74 05 e8 12 d1 d8 ff c9 c3 55 48 89 e5 0f 1f 44 00 00 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 <eb> f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f
      [597360.268192] Call Trace:
      [597360.268192] [<ffffffff811a93fb>] ? new_inode+0x3b/0xb0
      [597360.268192] [<ffffffff81410fd9>] ? sock_alloc+0x19/0x60
      [597360.268192] [<ffffffff8141143e>] ? __sock_create+0x9e/0x210
      [597360.268192] [<ffffffff81411610>] ? sock_create+0x30/0x40
      [597360.268192] [<ffffffff8141182c>] ? sys_socket+0x3c/0x80
      [597360.268192] [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
      [597360.079003] xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597360.079003] CPU 0

      [597360.079003] Modules linked in: vzethdev simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota ip6table_mangle nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597360.079003]
      [597360.079003] Pid: 390166, comm: apache2 veid: 1025 Not tainted 2.6.32-042stab049.6 #1 042stab049_6 empty empty/S2927
      [597360.079003] RIP: 0010:[<ffffffff814e191e>] [<ffffffff814e191e>] _spin_lock+0x1e/0x30
      [597360.079003] RSP: 0018:ffff8803cc201df8 EFLAGS: 00000297
      [597360.079003] RAX: 0000000000003c91 RBX: ffff8803cc201df8 RCX: 0000000000000002
      [597360.079003] RDX: 0000000000003c90 RSI: ffffffff823cca00 RDI: ffffffff823cca00
      [597360.079003] RBP: ffffffff8100bcce R08: 0000000000000000 R09: 0000000000000000
      [597360.079003] R10: 0000000000000000 R11: 0000000000000206 R12: ffff8802243416b0
      [597360.079003] R13: ffff8802243416b0 R14: ffff8802243416b0 R15: ffff8802243416b0
      [597360.079003] FS: 000000004d377960(0063) GS:ffff880028200000(0000) knlGS:00000000b774f6c0
      [597360.079003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
      [597360.079003] CR2: 000000004d36cf90 CR3: 0000000111323000 CR4: 00000000000006f0
      [597360.079003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      [597360.079003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      [597360.079003] Process apache2 (pid: 390166, veid: 1025, threadinfo ffff8803cc200000, task ffff880329357410)
      [597360.079003] Stack:
      [597360.079003] ffff8803cc201e28 ffffffff81261685 ffff8803cc201e08 ffffffff823cca00
      [597360.079003] <0> ffff880012918150 ffff8801862a8790 ffff8803cc201e48 ffffffff811a77b0
      [597360.079003] <0> ffff8803cc201e48 ffff880012918150 ffff8803cc201e68 ffffffff811a4028

      [597360.079003] Call Trace:
      [597360.079003] [<ffffffff81261685>] ? _atomic_dec_and_lock+0x55/0x80
      [597360.079003] [<ffffffff811a77b0>] ? iput+0x30/0x70
      [597360.079003] [<ffffffff811a4028>] ? dentry_iput+0x98/0x110
      [597360.079003] [<ffffffff811a419c>] ? d_kill+0x3c/0x70
      [597360.079003] [<ffffffff811a5933>] ? dput+0xa3/0x1d0
      [597360.079003] [<ffffffff8118e9e1>] ? __fput+0x1d1/0x280
      [597360.079003] [<ffffffff8118eab5>] ? fput+0x25/0x30
      [597360.079003] [<ffffffff8118a01d>] ? filp_close+0x5d/0x90
      [597360.079003] [<ffffffff8118a0f5>] ? sys_close+0xa5/0x100
      [597360.079003] [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
      [597360.079003] Code: 00 00 00 01 74 05 e8 12 d1 d8 ff c9 c3 55 48 89 e5 0f 1f 44 00 00 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89
      [597360.079003] Call Trace:
      [597360.079003] [<ffffffff81261685>] ? _atomic_dec_and_lock+0x55/0x80
      [597360.079003] [<ffffffff811a77b0>] ? iput+0x30/0x70
      [597360.079003] [<ffffffff811a4028>] ? dentry_iput+0x98/0x110
      [597360.079003] [<ffffffff811a419c>] ? d_kill+0x3c/0x70
      [597360.079003] [<ffffffff811a5933>] ? dput+0xa3/0x1d0
      [597360.079003] [<ffffffff8118e9e1>] ? __fput+0x1d1/0x280
      [597360.079003] [<ffffffff8118eab5>] ? fput+0x25/0x30
      [597360.079003] [<ffffffff8118a01d>] ? filp_close+0x5d/0x90
      [597360.079003] [<ffffffff8118a0f5>] ? sys_close+0xa5/0x100

      [597360.079003] [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
      1. softlock.txt
        5 kB
        Yasuyuki Nakamura
      2. vz3-042stab055.7.debug-lockup.txt
        655 kB
        Dave
      3. vz3-softlockup-may-4-2012.txt
        105 kB
        Dave
      4. vzdq-fix-sleep-under-inode_lock-in-vzdq_aquotq_lookset
        2 kB
        Konstantin Khlebnikov

        Issue Links

          Activity

          Hide
          deltaanime@gmail.com Francisco Dias added a comment -

          Any update on if anyone has seen any crashes yet?

          I've pushed this patch to a couple production boxes and haven't seen anything yet, but figured i'd check in

          Francisco

          Show
          deltaanime@gmail.com Francisco Dias added a comment - Any update on if anyone has seen any crashes yet? I've pushed this patch to a couple production boxes and haven't seen anything yet, but figured i'd check in Francisco
          Hide
          dave-openvz@dubkat.com Dave added a comment -

          vzkernel-2.6.32-042stab055.7.debug.2.x86_64.rpm has been solid with no debug output to console for 4 days. I usually see the soft lockup within 6-7 days at the longest. I am 99% sure this kernel fixed it - I will definitely report any further issues.

          Show
          dave-openvz@dubkat.com Dave added a comment - vzkernel-2.6.32-042stab055.7.debug.2.x86_64.rpm has been solid with no debug output to console for 4 days. I usually see the soft lockup within 6-7 days at the longest. I am 99% sure this kernel fixed it - I will definitely report any further issues.
          Hide
          deltaanime@gmail.com Francisco Dias added a comment -

          Awesome to hear that

          All goes well this will address a bug report I had in December regarding this same thing.

          Francisco

          Show
          deltaanime@gmail.com Francisco Dias added a comment - Awesome to hear that All goes well this will address a bug report I had in December regarding this same thing. Francisco
          Hide
          khlebnikov@openvz.org Konstantin Khlebnikov added a comment -

          fixed in 042stab055.10

          Show
          khlebnikov@openvz.org Konstantin Khlebnikov added a comment - fixed in 042stab055.10
          Hide
          sergeyb Sergey Bronnikov (Inactive) added a comment -

          Bug was fixed more than one year ago and there were no complains from reporter after fix. We treat bug as fixed and close bug.

          Show
          sergeyb Sergey Bronnikov (Inactive) added a comment - Bug was fixed more than one year ago and there were no complains from reporter after fix. We treat bug as fixed and close bug.

            People

            • Assignee:
              khlebnikov@openvz.org Konstantin Khlebnikov
              Reporter:
              dave-openvz@dubkat.com Dave
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: