Uploaded image for project: 'OpenVZ'
  1. OpenVZ
  2. OVZ-5386

BUG: soft lockup - CPU#2 stuck for 67s! : sock_alloc / new_inode / _spin_lock+0x1e/0x30

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Fix Version/s: OpenVZ-legacy
    • Component/s: Containers::Kernel
    • Security Level: Public
    • Environment:
      Operating System: Other
      Platform: x86_64 (AMD64)

      Description

      Getting a kernel panic every week:

      [root@vz ~]# uname -a
      Linux vz.xyz.com 2.6.32-042stab049.6 #1 SMP Mon Feb 6 19:17:43 MSK 2012 x86_64 x86_64 x86_64 GNU/Linux

      [root@vz ~]# cat /etc/redhat-release
      CentOS release 6.2 (Final)

      This is as much as my terminal server logs captured:

      th xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597276.142067] CPU 1
      [597276.142067] Modules linked in: vzethdev simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota ip6table_mangle nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597276.142067]
      [597276.142067] Pid: 946200, comm: set_keyserver_u veid: 1035 Not tainted 2.6.32-042stab049.6 #1 042stab049_6 empty empty/S2927
      [597276.142067] RIP: 0010:[<ffffffff814e191e>] [<ffffffff814e191e>] _spin_lock+0x1e/0x30
      [597276.142067] RSP: 0018:ffff88000babfe98 EFLAGS: 00000293
      [597276.142067] RAX: 0000000000003c92 RBX: ffff88000babfe98 RCX: 0000000000000000
      [597276.142067] RDX: 0000000000003c90 RSI: ffffffff817a1438 RDI: ffffffff823cca00
      [597276.142067] RBP: ffffffff8100bcce R08: 0000000000000000 R09: 000000000000000e
      [597276.142067] R10: 0000000000000000 R11: 0000000000000246 R12: ffffffff81042b54
      [597276.142067] R13: ffff88000babff18 R14: 0000000000000014 R15: ffff8803c5c004a8
      [597276.142067] FS: 00007f54f714b6e0(0000) GS:ffff880028280000(0000) knlGS:00000000b774f6c0
      [597276.142067] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      [597276.142067] CR2: 00007f54f66747e0 CR3: 00000002a8a5b000 CR4: 00000000000006e0
      [597276.142067] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      [597276.142067] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      [597276.142067] Process set_keyserver_u (pid: 946200, veid: 1035, threadinfo ffff88000babe000, task ffff8800ad7b3290)

      [597276.142067] Stack:
      [597276.142067] ffff88000babfec8 ffffffff811a93fb 000000001d12a781 ffff880336840980
      [597276.142067] <0> 0000000000000000 0000000000000000 ffff88000babfed8 ffffffff81410fd9
      [597276.142067] <0> ffff88000babff28 ffffffff8141143e ffff880000000001 ffff880000000002
      [597276.142067] Call Trace:
      [597276.142067] [<ffffffff811a93fb>] ? new_inode+0x3b/0xb0
      [597276.142067] [<ffffffff81410fd9>] ? sock_alloc+0x19/0x60
      [597276.142067] [<ffffffff8141143e>] ? __sock_create+0x9e/0x210
      [597276.142067] [<ffffffff81411610>] ? sock_create+0x30/0x40
      [597276.142067] [<ffffffff8141182c>] ? sys_socket+0x3c/0x80
      [597276.142067] [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
      [597276.142067] Code: 00 00 00 01 74 05 e8 12 d1 d8 ff c9 c3 55 48 89 e5 0f 1f 44 00 00 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89
      [597276.142067] Call Trace:
      [597276.142067] [<ffffffff811a93fb>] ? new_inode+0x3b/0xb0
      [597276.142067] [<ffffffff81410fd9>] ? sock_alloc+0x19/0x60
      [597276.142067] [<ffffffff8141143e>] ? __sock_create+0x9e/0x210
      [597276.142067] [<ffffffff81411610>] ? sock_create+0x30/0x40
      [597276.142067] [<ffffffff8141182c>] ? sys_socket+0x3c/0x80
      [597276.142067] [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
      [597276.206006] BUG: soft lockup - CPU#2 stuck for 67s! [update_deploy_p:946201]
      [597276.206009] Modules linked in: vzethdev simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota ip6table_mangle nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4
      mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597276.206085] CPU 2
      [597276.206087] Modules linked in: vzethdev simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota ip6table_mangle nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597276.206133]
      [597276.206133] Pid: 946201, comm: update_deploy_p veid: 1035 Not tainted 2.6.32-042stab049.6 #1 042stab049_6 empty empty/S2927
      [597276.206133] RIP: 0010:[<ffffffff814e191e>] [<ffffffff814e191e>] _spin_lock+0x1e/0x30
      [597276.206133] RSP: 0018:ffff8800ad635e98 EFLAGS: 00000297
      [597276.206133] RAX: 0000000000003c94 RBX: ffff8800ad635e98 RCX: 0000000000000000
      [597276.206133] RDX: 0000000000003c90 RSI: ffffffff817a1438 RDI: ffffffff823cca00
      [597276.206133] RBP: ffffffff8100bcce R08: 0000000000000000 R09: 0000000000000003
      [597276.206133] R10: 0000000000000000 R11: 0000000000000246 R12: ffffffff81042b54
      [597276.206133] R13: ffff8800ad635f18 R14: 0000000000000014 R15: ffff8803151d8c78
      [597276.206133] FS: 00007f4f0dc176e0(0000) GS:ffff880230400000(0000) knlGS:00000000b774f6c0
      [597276.206133] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      [597276.206133] CR2: 00007f4f0d1407e0 CR3: 00000002310a1000 CR4: 00000000000006e0
      [597276.206133] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      [597276.206133] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      [597276.206133] Process update_deploy_p (pid: 946201, veid: 1035, threadinfo ffff8800ad634000, task ffff8803d03a4800)
      [597276.206133] Stack:
      [597276.206133] ffff8800ad635ec8 ffffffff811a93fb 000000001d12a781 ffff880336840980

      [597276.206133] <0> 00 ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597276.268102] CPU 3
      [597276.268103] Modules 0000(0000) knlGS:00000000b774f6c0
      [597276.268192] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      [597276.268192] CR2: 00007f246f7067e0 CR3: 0000000407807000 CR4: 00000000000006e0
      [597276.268192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      [597276.268192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      [597276.268192] Process update_deploy_p (pid: 946188, veid: 1063, threadinfo ffff8800c232e000, task ffff8804239049c0)
      [597276.268192] Stack:
      [597276.268192] ffff8800c232fec8 ffffffff811a93fb 0000000009ffd874 ffff880225129200
      [597276.268192] <0> 0000000000000000 0000000000000000 ffff8800c232fed8 ffffffff81410fd9
      [597276.268192] <0> ffff8800c232ff28 ffffffff8141143e ffff880000000001 ffff880000000002
      [597276.268192] Call Trace:
      [597276.268192] [<ffffffff811a93fb>] ? new_inode+0x3b/0xb0
      [597276.268192] [<ffffffff81410fd9>] ? sock_alloc+0x19/0x60
      [597276.268192] [<ffffffff8141143e>] ? __sock_create+0x9e/0x210
      [597276.268192] [<ffffffff81411610>] ? sock_crestring xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597276.079003]
      [597276.079003] Pid: 390166, comm: apache2 veid: 1025 Not tainted 2.6.32-042stab049.6 #1 042stab049_6 empty empty/S2927
      [597276.079003] RIP: 0010:[<ffffffff814e191e>] [<ffffffff814e191e>] _spin_lock+0x1e/0x30
      [597276.079003] RSP: 0018:ffff8803cc2b df c9 c3 0f 1f 40 00 55 48 89

      [597276.079003] Call Trace:
      [597276.079003] [<ffffffff81261685>] ? _atomic_dec_and_lock+0x55/0x80
      [597276.079003] [<ffffffff811a77b0>] ? iput+0x30/0x70
      [597276.079003] [<ffffffff811a4028>] ? dentry_iput+0x98/0x110
      [597276.079003] [<ffffffff811a419c>] ? d_kill+0x3c/0x70
      [597276.079003] [<ffffffff811a5933>] ? dput+0xa3/0x1d0
      [597276.079003] [<ffffffff8118e9e1>] ? __fput+0x1d1/0x280
      [597276.079003] [<ffffffff8118eab5>] ? fput+0x25/0x30
      [597276.079003] [<fffffff
                                [597360.268180] RIP: 0010:[<ffffffff814e1921>] [<ffffffff814e1921>] _spin_lock+0x21/0x30
      [597360.268191] RSP: 0018:ffff8800c232fe98 EFLAGS: 00000293
      [597360.268192] RAX: 0000000000003c93 RBX: ffff8800c232fe98 RCX: 0000000000000000
      [597360.268192] RDX: 0000000000003c90 RSI: ffffffff817a1438 RDI: ffffffff823cca00
      [597360.268192] RBP: ffffffff8100bcce R08: 0000000000000000 R09: 000000000000000f
      [597360.268192] R10: 0000000000000000 R11: 0000000000000246 R12: ffffffff81042b54
      [597360.268192] R13: ffff8800c232ff18 R14: 0000000000000014 R15: ffff880336b5ce90
      [597360.268192] FS: 00007f24701dd6e0(0000) GS:ffff880230480000(0000) knlGS:00000000b774f6c0
      [597360.268192] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      [597360.268192] CR2: 00007f246f7067e0 CR3: 0000000407807000 CR4: 00000000000006e0
      [597360.268192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      [597360.268192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      [597360.268192] Process update_deploy_p (pid: 946188, veid: 1063, threadinfo ffff8800c232e000, task ffff8804239049c0)

      [597360.268192] Stack:
      [597360.268192] ffff8800c232fec8 ffffffff811a93fb 0000000009ffd874 ffff880225129200
      [597360.268192] <0> 0000000000000000 0000000000000000 ffff8800c232fed8 ffffffff81410fd9
      [597360.268192] <0> ffff8800c232ff28 ffffffff8141143e ffff880000000001 ffff880000000002
      [597360.268192] Call Trace:
      [597360.268192] [<ffffffff811a93fb>] ? new_inode+0x3b/0xb0
      [597360.268192] [<ffffffff81410fd9>] ? sock_alloc+0x19/0x60
      [597360.268192] [<ffffffff8141143e>] ? __sock_create+0x9e/0x210
      [597360.268192] [<ffffffff81411610>] ? sock_create+0x30/0x40
      [597360.268192] [<ffffffff8141182c>] ? sys_socket+0x3c/0x80
      [597360.268192] [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
      [597360.268192] Code: 01 74 05 e8 12 d1 d8 ff c9 c3 55 48 89 e5 0f 1f 44 00 00 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 <eb> f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f
      [597360.268192] Call Trace:
      [597360.268192] [<ffffffff811a93fb>] ? new_inode+0x3b/0xb0
      [597360.268192] [<ffffffff81410fd9>] ? sock_alloc+0x19/0x60
      [597360.268192] [<ffffffff8141143e>] ? __sock_create+0x9e/0x210
      [597360.268192] [<ffffffff81411610>] ? sock_create+0x30/0x40
      [597360.268192] [<ffffffff8141182c>] ? sys_socket+0x3c/0x80
      [597360.268192] [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
      [597360.079003] xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597360.079003] CPU 0

      [597360.079003] Modules linked in: vzethdev simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota ip6table_mangle nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sr_mod cdrom sd_mod crc_t10dif forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]
      [597360.079003]
      [597360.079003] Pid: 390166, comm: apache2 veid: 1025 Not tainted 2.6.32-042stab049.6 #1 042stab049_6 empty empty/S2927
      [597360.079003] RIP: 0010:[<ffffffff814e191e>] [<ffffffff814e191e>] _spin_lock+0x1e/0x30
      [597360.079003] RSP: 0018:ffff8803cc201df8 EFLAGS: 00000297
      [597360.079003] RAX: 0000000000003c91 RBX: ffff8803cc201df8 RCX: 0000000000000002
      [597360.079003] RDX: 0000000000003c90 RSI: ffffffff823cca00 RDI: ffffffff823cca00
      [597360.079003] RBP: ffffffff8100bcce R08: 0000000000000000 R09: 0000000000000000
      [597360.079003] R10: 0000000000000000 R11: 0000000000000206 R12: ffff8802243416b0
      [597360.079003] R13: ffff8802243416b0 R14: ffff8802243416b0 R15: ffff8802243416b0
      [597360.079003] FS: 000000004d377960(0063) GS:ffff880028200000(0000) knlGS:00000000b774f6c0
      [597360.079003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
      [597360.079003] CR2: 000000004d36cf90 CR3: 0000000111323000 CR4: 00000000000006f0
      [597360.079003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      [597360.079003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      [597360.079003] Process apache2 (pid: 390166, veid: 1025, threadinfo ffff8803cc200000, task ffff880329357410)
      [597360.079003] Stack:
      [597360.079003] ffff8803cc201e28 ffffffff81261685 ffff8803cc201e08 ffffffff823cca00
      [597360.079003] <0> ffff880012918150 ffff8801862a8790 ffff8803cc201e48 ffffffff811a77b0
      [597360.079003] <0> ffff8803cc201e48 ffff880012918150 ffff8803cc201e68 ffffffff811a4028

      [597360.079003] Call Trace:
      [597360.079003] [<ffffffff81261685>] ? _atomic_dec_and_lock+0x55/0x80
      [597360.079003] [<ffffffff811a77b0>] ? iput+0x30/0x70
      [597360.079003] [<ffffffff811a4028>] ? dentry_iput+0x98/0x110
      [597360.079003] [<ffffffff811a419c>] ? d_kill+0x3c/0x70
      [597360.079003] [<ffffffff811a5933>] ? dput+0xa3/0x1d0
      [597360.079003] [<ffffffff8118e9e1>] ? __fput+0x1d1/0x280
      [597360.079003] [<ffffffff8118eab5>] ? fput+0x25/0x30
      [597360.079003] [<ffffffff8118a01d>] ? filp_close+0x5d/0x90
      [597360.079003] [<ffffffff8118a0f5>] ? sys_close+0xa5/0x100
      [597360.079003] [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
      [597360.079003] Code: 00 00 00 01 74 05 e8 12 d1 d8 ff c9 c3 55 48 89 e5 0f 1f 44 00 00 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89
      [597360.079003] Call Trace:
      [597360.079003] [<ffffffff81261685>] ? _atomic_dec_and_lock+0x55/0x80
      [597360.079003] [<ffffffff811a77b0>] ? iput+0x30/0x70
      [597360.079003] [<ffffffff811a4028>] ? dentry_iput+0x98/0x110
      [597360.079003] [<ffffffff811a419c>] ? d_kill+0x3c/0x70
      [597360.079003] [<ffffffff811a5933>] ? dput+0xa3/0x1d0
      [597360.079003] [<ffffffff8118e9e1>] ? __fput+0x1d1/0x280
      [597360.079003] [<ffffffff8118eab5>] ? fput+0x25/0x30
      [597360.079003] [<ffffffff8118a01d>] ? filp_close+0x5d/0x90
      [597360.079003] [<ffffffff8118a0f5>] ? sys_close+0xa5/0x100

      [597360.079003] [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
      1. softlock.txt
        5 kB
        Yasuyuki Nakamura
      2. vz3-042stab055.7.debug-lockup.txt
        655 kB
        Dave
      3. vz3-softlockup-may-4-2012.txt
        105 kB
        Dave
      4. vzdq-fix-sleep-under-inode_lock-in-vzdq_aquotq_lookset
        2 kB
        Konstantin Khlebnikov

        Issue Links

          Activity

          Hide
          deltaanime@gmail.com Francisco Dias added a comment -

          Any update on if anyone has seen any crashes yet?

          I've pushed this patch to a couple production boxes and haven't seen anything yet, but figured i'd check in

          Francisco

          Show
          deltaanime@gmail.com Francisco Dias added a comment - Any update on if anyone has seen any crashes yet? I've pushed this patch to a couple production boxes and haven't seen anything yet, but figured i'd check in Francisco
          Hide
          dave-openvz@dubkat.com Dave added a comment -

          vzkernel-2.6.32-042stab055.7.debug.2.x86_64.rpm has been solid with no debug output to console for 4 days. I usually see the soft lockup within 6-7 days at the longest. I am 99% sure this kernel fixed it - I will definitely report any further issues.

          Show
          dave-openvz@dubkat.com Dave added a comment - vzkernel-2.6.32-042stab055.7.debug.2.x86_64.rpm has been solid with no debug output to console for 4 days. I usually see the soft lockup within 6-7 days at the longest. I am 99% sure this kernel fixed it - I will definitely report any further issues.
          Hide
          deltaanime@gmail.com Francisco Dias added a comment -

          Awesome to hear that

          All goes well this will address a bug report I had in December regarding this same thing.

          Francisco

          Show
          deltaanime@gmail.com Francisco Dias added a comment - Awesome to hear that All goes well this will address a bug report I had in December regarding this same thing. Francisco
          Hide
          khlebnikov@openvz.org Konstantin Khlebnikov added a comment -

          fixed in 042stab055.10

          Show
          khlebnikov@openvz.org Konstantin Khlebnikov added a comment - fixed in 042stab055.10
          Hide
          sergeyb Sergey Bronnikov added a comment -

          Bug was fixed more than one year ago and there were no complains from reporter after fix. We treat bug as fixed and close bug.

          Show
          sergeyb Sergey Bronnikov added a comment - Bug was fixed more than one year ago and there were no complains from reporter after fix. We treat bug as fixed and close bug.

            People

            • Assignee:
              khlebnikov@openvz.org Konstantin Khlebnikov
              Reporter:
              dave-openvz@dubkat.com Dave
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: