Remounting root filesystem in read-write mode: [ OK ] Mounting local filesystems: [ 31.060702] EXT4-fs (md0): mounted filesystem with ordered data mode. Opts: [ 31.173407] EXT4-fs (dm-2): mounted filesystem with ordered data mode. Opts: [ OK ] Enabling /etc/fstab swaps: [ 31.506770] Adding 16383992k swap on /dev/mapper/VolGroup00-swap. Priority:-1 extents:1 across:16383992k [ OK ] Entering non-interactive startup Calling the system activity data collector (sadc): Starting monitoring for VG VolGroup00: 3 logical volume(s) in volume group "VolGroup00" monitored [ OK ] ip6tables: Applying firewall rules: [ 32.530777] ip6_tables: (C) 2000-2006 Netfilter Core Team [ 32.625195] nf_conntrack version 0.5.0 (16384 buckets, 65536 max) [ OK ] ip6tables: Loading additional modules: net-pf-16-proto-12 [ 32.666503] Netfilter messages via NETLINK v0.30. ip6t_comment ip6t_CONNSECMARK ip6t_esp ip6t_helper ip6t_length ip6t_limit ip6t_mac ip6t_mark ip6t_MARK ip6t_multiport ip6t_NFQUEUE ip6t_physdev ip6t_pkttype ip6t_policy ip6t_quota ip6t_SECMARK ip6t_state ip6t_statistic ip6t_string ip6_tables [ OK ] iptables: Applying firewall rules: [ 32.856558] ip_tables: (C) 2000-2006 Netfilter Core Team [ OK ] iptables: Loading additional modules: ip_conntrack_netbios_ns tun [ 32.927364] tun: Universal TUN/TAP device driver, 1.6 [ 32.932414] tun: (C) 1999-2004 Max Krasnyansky ipt_REJECT ipt_REDIRECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ip_conntrack ip_conntrack_ftp ip_conntrack_irc ipt_LOG ipt_conntrack ipt_state iptable_nat ip_nat_irc ipt_TOS xt_state ip_nat ip_tables ipt_recent nfnetlink-subsys-4 nfnetlink-subsys-3 ipt_CLASSIFY ipt_comment ipt_connbytes ipt_connmark ipt_CONNMARK ipt_CONNSECMARK ipt_dccp ipt_esp ipt_helper ipt_mac ipt_mark ipt_MARK ipt_NFQUEUE arpt_NFQUEUE ipt_NOTRACK ipt_physdev ipt_pkttype ipt_policy ipt_quota ipt_realm ipt_sctp ipt_SECMARK ipt_statistic ipt_string ip_tables [ OK ] Bringing up loopback interface: [ OK ] Bringing up interface eth0: SIOCADDRT: No such process [ OK ] Bringing up interface venet0: Bringing up interface venet0: Configuring interface venet0: net.ipv4.conf.venet0.send_redirects = 0 [ OK ] Starting auditd: [ OK ] Starting system logger: [ OK ] Starting mdmonitor: [ OK ] Mounting other filesystems: [ OK ] Retrigger failed udev events[ OK ] Starting sshd: [ OK ] Starting ntpd: [ OK ] Starting postfix: [ OK ] Starting crond: [ OK ] Checking vzevent kernel module ...[ OK ] Starting vzeventd: [ OK ] Starting OpenVZ: [ 37.470822] RPC: Registered udp transport module. [ 37.475529] RPC: Registered tcp transport module. [ 37.480230] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 37.568751] Slow work thread pool: Starting up [ 37.573323] Slow work thread pool: Ready [ 37.577331] FS-Cache: Loaded [ 37.633586] FS-Cache: Netfs 'nfs' registered for caching [ 37.733808] ploop_dev: module loaded [ OK ] Starting CT 116: [ 45.787023] warning: `vzctl' uses 32-bit capabilities (legacy support in use) [ 45.796929] CT: 116: started [ OK ] Starting CT 224: [ 345.681482] CT: 224: started [ OK ] Starting CT 317: [ 395.300847] CT: 317: started [ OK ] Starting CT 360: [ 401.310343] BUG: sleeping function called from invalid context at kernel/rwsem.c:21 [ 401.318038] in_atomic(): 1, irqs_disabled(): 0, pid: 3766, name: quotaon [ 401.324744] Pid: 3766, comm: quotaon veid: 317 Not tainted 2.6.32-042stab055.7.debug #1 [ 401.332738] Call Trace: [ 401.335206] [] ? __might_sleep+0xfe/0x120 [ 401.340868] [] ? down_read+0x24/0x40 [ 401.346096] [] ? user_get_super+0x83/0xf0 [ 401.351761] [] ? vzdq_aquotq_lookset+0xb1/0x160 [vzdquota] [ 401.358895] [] ? iget5_locked+0x139/0x1f0 [ 401.364556] [] ? vzdq_aquotq_lookset+0x0/0x160 [vzdquota] [ 401.371602] [] ? vzdq_aquotq_lookup+0xe2/0x160 [vzdquota] [ 401.378648] [] ? do_lookup+0x1ac/0x210 [ 401.384047] [] ? __link_path_walk+0x89e/0x10c0 [ 401.390145] [] ? __link_path_walk+0xe67/0x10c0 [ 401.396242] [] ? path_walk+0x6a/0xe0 [ 401.401466] [] ? do_path_lookup+0x5b/0xa0 [ 401.407124] [] ? do_filp_open+0xfb/0xca0 [ 401.412699] [] ? __do_page_fault+0x264/0x4f0 [ 401.418619] [] ? cpumask_any_but+0x31/0x50 [ 401.424365] [] ? strncpy_from_user+0x5d/0xa0 [ 401.430285] [] ? alloc_fd+0x92/0x170 [ 401.435512] [] ? do_sys_open+0x69/0x140 [ 401.440999] [] ? compat_sys_open+0x1a/0x20 [ 401.446744] [] ? ia32_sysret+0x0/0x5 [ 401.451969] BUG: scheduling while atomic: quotaon/3766/0x10000002 [ 401.458063] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 401.578883] Pid: 3766, comm: quotaon veid: 317 Not tainted 2.6.32-042stab055.7.debug #1 [ 401.586877] Call Trace: [ 401.589329] [] ? __schedule_bug+0x66/0x70 [ 401.594989] [] ? thread_return+0x26a/0x83d [ 401.600737] [] ? __cond_resched+0x2a/0x40 [ 401.606393] [] ? _cond_resched+0x30/0x40 [ 401.611966] [] ? down_read+0x29/0x40 [ 401.617197] [] ? user_get_super+0x83/0xf0 [ 401.622858] [] ? vzdq_aquotq_lookset+0xb1/0x160 [vzdquota] [ 401.629987] [] ? iget5_locked+0x139/0x1f0 [ 401.635648] [] ? vzdq_aquotq_lookset+0x0/0x160 [vzdquota] [ 401.642691] [] ? vzdq_aquotq_lookup+0xe2/0x160 [vzdquota] [ 401.649736] [] ? do_lookup+0x1ac/0x210 [ 401.655135] [] ? __link_path_walk+0x89e/0x10c0 [ 401.661227] [] ? __link_path_walk+0xe67/0x10c0 [ 401.667318] [] ? path_walk+0x6a/0xe0 [ 401.672544] [] ? do_path_lookup+0x5b/0xa0 [ 401.678209] [] ? do_filp_open+0xfb/0xca0 [ 401.683781] [] ? __do_page_fault+0x264/0x4f0 [ 401.689698] [] ? cpumask_any_but+0x31/0x50 [ 401.695448] [] ? strncpy_from_user+0x5d/0xa0 [ 401.701368] [] ? alloc_fd+0x92/0x170 [ 401.706595] [] ? do_sys_open+0x69/0x140 [ 401.712081] [] ? compat_sys_open+0x1a/0x20 [ 401.717826] [] ? ia32_sysret+0x0/0x5 [ 444.352128] CT: 360: started INIT: version 2.85 booting [ OK ] Starting CT 1022: mount: proc already mounted Welcome to Red Hat Enterprise Linux AS Press 'I' to enter interactive startup. Configuring kernel parameters: [ OK ] hwclock is unable to get I/O port access: the iopl(3) call failed. Setting clock : Tue May 8 10:33:39 EDT 2012 [ OK ] Setting hostname www.tourfilter.com: [ OK ] Mounting USB filesystem: mount: mount point /proc/bus/usb does not exist [FAILED] grep: /proc/bus/usb/drivers: No such file or directory Initializing USB HID interface: execvp: No such file or directory [FAILED] Initializing USB keyboard: execvp: No such file or directory [FAILED] Initializing USB mouse: execvp: No such file or directory [FAILED] Your system appears to have shut down uncleanly Press Y within 5 seconds to force file system integrity check... Press Y within 4 seconds to force file system integrity check... Press Y within 3 seconds to force file system integrity check... Press Y within 2 seconds to force file system integrity check... Press Y within 1 seconds to force file system integrity check... Remounting root filesystem in read-write mode: [ OK ] Activating swap partitions: [ OK ] mount: can't find / in /etc/fstab or /etc/mtab mount: can't find /proc in /etc/fstab or /etc/mtab Mounting local filesystems: [ OK ] Enabling swap space: [ OK ] /etc/rc.d/rc.sysinit: line 1: /sbin/modprobe: No such file or directory INIT: Entering runlevel: 3 Entering non-interactive startup Flushing firewall rules: [ OK ] Setting chains to policy ACCEPT: nat mangle filter [ OK ] Unloading iptables modules: /etc/rc3.d/S08iptables: line 1: lsmod: command not found /etc/rc3.d/S08iptables: line 1: lsmod: command not found [FAILED] /etc/rc3.d/S10network: line 1: modprobe: command not found error: permission denied on key 'vm.compact_memory' error: 'fs.quota..fake' is an unknown key error: permission denied on key 'net.ipv4.rout[ 452.585711] process `sysctl' is using deprecated sysctl (syscall) net.ipv6.neigh.lo.retrans_time; Use net.ipv6.neigh.lo.retrans_time_ms instead. e.flush' error: permission denied on key 'net.ipv4.route.flush' error: permission denied on key 'net.ipv6.route.flush' Setting network parameters: [ OK ] Bringing up loopback interface: [ OK ] Bringing up interface venet0: [ OK ] Starting vzquota: [ OK ] Starting default_routing: [ OK ] Starting system logger: [ OK ] Starting kernel logger: [PASSED] Starting sshd:[ OK ] Starting xinetd: [ OK ] Starting postfix: [ OK ] Starting console mouse services: (no mouse is configured) Starting lsws: [ OK ] Starting crond: [ OK ] INIT: no more processes left in this runlevel [ 480.802392] CT: 1022: started [ OK ] Starting CT 1025: [ 525.036377] CT: 1025: started [ OK ] Starting CT 1035: [ 545.667098] CT: 1035: started [ OK ] Starting CT 1048: [ 561.545768] CT: 1048: started [ OK ] Starting CT 1063: [ 588.155913] process `snmpd' is using deprecated sysctl (syscall) net.ipv6.neigh.lo.retrans_time; Use net.ipv6.neigh.lo.retrans_time_ms instead. [ 594.041183] CT: 1063: started [ OK ] Starting CT 1065: [ 617.617342] CT: 1065: started [ OK ] Starting CT 1070: [ 644.505034] CT: 1070: started [ OK ] Starting CT 1072: [ 660.403144] CT: 1072: started [ OK ] Starting CT 1080: [ 672.773482] CT: 1080: started [ OK ] Starting CT 1081: [ 708.871662] CT: 1081: started [ OK ] Starting CT 1082: [ 713.735617] BUG: sleeping function called from invalid context at kernel/rwsem.c:21 [ 713.743278] in_atomic(): 1, irqs_disabled(): 0, pid: 16110, name: quotaon [ 713.750063] Pid: 16110, comm: quotaon veid: 1081 Not tainted 2.6.32-042stab055.7.debug #1 [ 713.758230] Call Trace: [ 713.760690] [] ? __might_sleep+0xfe/0x120 [ 713.766347] [] ? down_read+0x24/0x40 [ 713.771571] [] ? user_get_super+0x83/0xf0 [ 713.777234] [] ? vzdq_aquotq_lookset+0xb1/0x160 [vzdquota] [ 713.784362] [] ? iget5_locked+0x139/0x1f0 [ 713.790019] [] ? vzdq_aquotq_lookset+0x0/0x160 [vzdquota] [ 713.797068] [] ? vzdq_aquotq_lookup+0xe2/0x160 [vzdquota] [ 713.804108] [] ? do_lookup+0x1ac/0x210 [ 713.809503] [] ? __link_path_walk+0x89e/0x10c0 [ 713.815590] [] ? __link_path_walk+0xe67/0x10c0 [ 713.821678] [] ? path_walk+0x6a/0xe0 [ 713.826900] [] ? do_path_lookup+0x5b/0xa0 [ 713.832554] [] ? do_filp_open+0xfb/0xca0 [ 713.838122] [] ? __do_page_fault+0x264/0x4f0 [ 713.844039] [] ? cpumask_any_but+0x31/0x50 [ 713.849782] [] ? strncpy_from_user+0x5d/0xa0 [ 713.855708] [] ? alloc_fd+0x92/0x170 [ 713.860932] [] ? do_sys_open+0x69/0x140 [ 713.866413] [] ? sys_open+0x20/0x30 [ 713.871549] [] ? system_call_fastpath+0x16/0x1b [ 713.877722] BUG: scheduling while atomic: quotaon/16110/0x10000002 [ 713.883906] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 714.004305] Pid: 16110, comm: quotaon veid: 1081 Not tainted 2.6.32-042stab055.7.debug #1 [ 714.012463] Call Trace: [ 714.014909] [] ? __schedule_bug+0x66/0x70 [ 714.020559] [] ? thread_return+0x26a/0x83d [ 714.026294] [] ? __cond_resched+0x2a/0x40 [ 714.031943] [] ? _cond_resched+0x30/0x40 [ 714.037505] [] ? down_read+0x29/0x40 [ 714.042722] [] ? user_get_super+0x83/0xf0 [ 714.048372] [] ? vzdq_aquotq_lookset+0xb1/0x160 [vzdquota] [ 714.055494] [] ? iget5_locked+0x139/0x1f0 [ 714.061143] [] ? vzdq_aquotq_lookset+0x0/0x160 [vzdquota] [ 714.068178] [] ? vzdq_aquotq_lookup+0xe2/0x160 [vzdquota] [ 714.075216] [] ? do_lookup+0x1ac/0x210 [ 714.080608] [] ? __link_path_walk+0x89e/0x10c0 [ 714.086692] [] ? __link_path_walk+0xe67/0x10c0 [ 714.092775] [] ? path_walk+0x6a/0xe0 [ 714.097990] [] ? do_path_lookup+0x5b/0xa0 [ 714.103638] [] ? do_filp_open+0xfb/0xca0 [ 714.109205] [] ? __do_page_fault+0x264/0x4f0 [ 714.115113] [] ? cpumask_any_but+0x31/0x50 [ 714.120847] [] ? strncpy_from_user+0x5d/0xa0 [ 714.126755] [] ? alloc_fd+0x92/0x170 [ 714.131972] [] ? do_sys_open+0x69/0x140 [ 714.137448] [] ? sys_open+0x20/0x30 [ 714.142577] [] ? system_call_fastpath+0x16/0x1b [ 736.299496] CT: 1082: started [ OK ] Starting CT 1083: [ 806.117369] CT: 1083: started [ OK ] Starting CT 1085: [ 825.029840] CT: 1085: started [ OK ] Starting CT 1102: [ 828.227741] BUG: sleeping function called from invalid context at kernel/rwsem.c:21 [ 828.235399] in_atomic(): 1, irqs_disabled(): 0, pid: 23693, name: repquota [ 828.242272] Pid: 23693, comm: repquota veid: 1081 Not tainted 2.6.32-042stab055.7.debug #1 [ 828.250525] Call Trace: [ 828.252986] [] ? __might_sleep+0xfe/0x120 [ 828.258641] [] ? down_read+0x24/0x40 [ 828.263858] [] ? user_get_super+0x83/0xf0 [ 828.269516] [] ? vzdq_aquotq_lookset+0xb1/0x160 [vzdquota] [ 828.276641] [] ? iget5_locked+0x139/0x1f0 [ 828.282296] [] ? vzdq_aquotq_lookset+0x0/0x160 [vzdquota] [ 828.289331] [] ? vzdq_aquotq_lookup+0xe2/0x160 [vzdquota] [ 828.296367] [] ? do_lookup+0x1ac/0x210 [ 828.301761] [] ? __link_path_walk+0x89e/0x10c0 [ 828.307842] [] ? __link_path_walk+0xe67/0x10c0 [ 828.313926] [] ? path_walk+0x6a/0xe0 [ 828.319142] [] ? do_path_lookup+0x5b/0xa0 [ 828.324793] [] ? do_filp_open+0xfb/0xca0 [ 828.330356] [] ? __do_page_fault+0x264/0x4f0 [ 828.336264] [] ? kern_path+0x2a/0x50 [ 828.341482] [] ? strncpy_from_user+0x5d/0xa0 [ 828.347395] [] ? alloc_fd+0x92/0x170 [ 828.352614] [] ? do_sys_open+0x69/0x140 [ 828.358090] [] ? sys_open+0x20/0x30 [ 828.363220] [] ? system_call_fastpath+0x16/0x1b [ 828.369387] BUG: scheduling while atomic: repquota/23693/0x10000002 [ 828.375646] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 828.495709] Pid: 23693, comm: repquota veid: 1081 Not tainted 2.6.32-042stab055.7.debug #1 [ 828.503951] Call Trace: [ 828.506399] [] ? __schedule_bug+0x66/0x70 [ 828.512047] [] ? thread_return+0x26a/0x83d [ 828.517782] [] ? __cond_resched+0x2a/0x40 [ 828.523430] [] ? _cond_resched+0x30/0x40 [ 828.528991] [] ? down_read+0x29/0x40 [ 828.534208] [] ? user_get_super+0x83/0xf0 [ 828.539858] [] ? vzdq_aquotq_lookset+0xb1/0x160 [vzdquota] [ 828.546979] [] ? iget5_locked+0x139/0x1f0 [ 828.552629] [] ? vzdq_aquotq_lookset+0x0/0x160 [vzdquota] [ 828.559665] [] ? vzdq_aquotq_lookup+0xe2/0x160 [vzdquota] [ 828.566699] [] ? do_lookup+0x1ac/0x210 [ 828.572087] [] ? __link_path_walk+0x89e/0x10c0 [ 828.578168] [] ? __link_path_walk+0xe67/0x10c0 [ 828.584250] [] ? path_walk+0x6a/0xe0 [ 828.589469] [] ? do_path_lookup+0x5b/0xa0 [ 828.595117] [] ? do_filp_open+0xfb/0xca0 [ 828.600682] [] ? __do_page_fault+0x264/0x4f0 [ 828.606590] [] ? kern_path+0x2a/0x50 [ 828.611805] [] ? strncpy_from_user+0x5d/0xa0 [ 828.617716] [] ? alloc_fd+0x92/0x170 [ 828.622934] [] ? do_sys_open+0x69/0x140 [ 828.628412] [] ? sys_open+0x20/0x30 [ 828.633543] [] ? system_call_fastpath+0x16/0x1b [ 915.600274] CT: 1102: started [ OK ] Starting CT 1113: [ 919.411333] BUG: sleeping function called from invalid context at kernel/rwsem.c:21 [ 919.418995] in_atomic(): 1, irqs_disabled(): 0, pid: 25673, name: quotaon [ 919.425782] Pid: 25673, comm: quotaon veid: 1102 Not tainted 2.6.32-042stab055.7.debug #1 [ 919.433947] Call Trace: [ 919.436408] [] ? __might_sleep+0xfe/0x120 [ 919.442069] [] ? down_read+0x24/0x40 [ 919.447293] [] ? user_get_super+0x83/0xf0 [ 919.452956] [] ? vzdq_aquotq_lookset+0xb1/0x160 [vzdquota] [ 919.460087] [] ? iget5_locked+0x139/0x1f0 [ 919.465743] [] ? vzdq_aquotq_lookset+0x0/0x160 [vzdquota] [ 919.472784] [] ? vzdq_aquotq_lookup+0xe2/0x160 [vzdquota] [ 919.479827] [] ? do_lookup+0x1ac/0x210 [ 919.485224] [] ? __link_path_walk+0x89e/0x10c0 [ 919.491312] [] ? __link_path_walk+0xe67/0x10c0 [ 919.497400] [] ? path_walk+0x6a/0xe0 [ 919.502622] [] ? do_path_lookup+0x5b/0xa0 [ 919.508277] [] ? do_filp_open+0xfb/0xca0 [ 919.513848] [] ? __do_page_fault+0x264/0x4f0 [ 919.519765] [] ? cpumask_any_but+0x31/0x50 [ 919.525520] [] ? strncpy_from_user+0x5d/0xa0 [ 919.531437] [] ? alloc_fd+0x92/0x170 [ 919.536660] [] ? do_sys_open+0x69/0x140 [ 919.542141] [] ? sys_open+0x20/0x30 [ 919.547282] [] ? system_call_fastpath+0x16/0x1b [ 919.553454] BUG: scheduling while atomic: quotaon/25673/0x10000002 [ 919.559638] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 919.680334] Pid: 25673, comm: quotaon veid: 1102 Not tainted 2.6.32-042stab055.7.debug #1 [ 919.688494] Call Trace: [ 919.690947] [] ? __schedule_bug+0x66/0x70 [ 919.696603] [] ? thread_return+0x26a/0x83d [ 919.702350] [] ? __cond_resched+0x2a/0x40 [ 919.708006] [] ? _cond_resched+0x30/0x40 [ 919.713575] [] ? down_read+0x29/0x40 [ 919.718797] [] ? user_get_super+0x83/0xf0 [ 919.724454] [] ? vzdq_aquotq_lookset+0xb1/0x160 [vzdquota] [ 919.731580] [] ? iget5_locked+0x139/0x1f0 [ 919.737244] [] ? vzdq_aquotq_lookset+0x0/0x160 [vzdquota] [ 919.744294] [] ? vzdq_aquotq_lookup+0xe2/0x160 [vzdquota] [ 919.751336] [] ? do_lookup+0x1ac/0x210 [ 919.756730] [] ? __link_path_walk+0x89e/0x10c0 [ 919.762819] [] ? __link_path_walk+0xe67/0x10c0 [ 919.768907] [] ? path_walk+0x6a/0xe0 [ 919.774137] [] ? do_path_lookup+0x5b/0xa0 [ 919.779792] [] ? do_filp_open+0xfb/0xca0 [ 919.785360] [] ? __do_page_fault+0x264/0x4f0 [ 919.791272] [] ? cpumask_any_but+0x31/0x50 [ 919.797016] [] ? strncpy_from_user+0x5d/0xa0 [ 919.802934] [] ? alloc_fd+0x92/0x170 [ 919.808159] [] ? do_sys_open+0x69/0x140 [ 919.813640] [] ? sys_open+0x20/0x30 [ 919.818780] [] ? system_call_fastpath+0x16/0x1b [ 922.357679] BUG: sleeping function called from invalid context at kernel/rwsem.c:21 [ 922.365344] in_atomic(): 1, irqs_disabled(): 0, pid: 25964, name: quotaon [ 922.372125] Pid: 25964, comm: quotaon veid: 1102 Not tainted 2.6.32-042stab055.7.debug #1 [ 922.380291] Call Trace: [ 922.382743] [] ? __might_sleep+0xfe/0x120 [ 922.388396] [] ? down_read+0x24/0x40 [ 922.393611] [] ? user_get_super+0x83/0xf0 [ 922.399266] [] ? vzdq_aquotq_lookset+0xb1/0x160 [vzdquota] [ 922.406388] [] ? iget5_locked+0x139/0x1f0 [ 922.412037] [] ? vzdq_aquotq_lookset+0x0/0x160 [vzdquota] [ 922.419078] [] ? vzdq_aquotq_lookup+0xe2/0x160 [vzdquota] [ 922.426112] [] ? do_lookup+0x1ac/0x210 [ 922.431500] [] ? __link_path_walk+0x89e/0x10c0 [ 922.437580] [] ? __link_path_walk+0xe67/0x10c0 [ 922.443661] [] ? path_walk+0x6a/0xe0 [ 922.448878] [] ? do_path_lookup+0x5b/0xa0 [ 922.454527] [] ? do_filp_open+0xfb/0xca0 [ 922.460096] [] ? __do_page_fault+0x264/0x4f0 [ 922.466004] [] ? cpumask_any_but+0x31/0x50 [ 922.471740] [] ? strncpy_from_user+0x5d/0xa0 [ 922.477648] [] ? alloc_fd+0x92/0x170 [ 922.482864] [] ? do_sys_open+0x69/0x140 [ 922.488338] [] ? sys_open+0x20/0x30 [ 922.493470] [] ? system_call_fastpath+0x16/0x1b [ 922.499639] BUG: scheduling while atomic: quotaon/25964/0x10000002 [ 922.505812] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 922.626308] Pid: 25964, comm: quotaon veid: 1102 Not tainted 2.6.32-042stab055.7.debug #1 [ 922.634476] Call Trace: [ 922.636929] [] ? __schedule_bug+0x66/0x70 [ 922.642582] [] ? thread_return+0x26a/0x83d [ 922.648327] [] ? __cond_resched+0x2a/0x40 [ 922.653982] [] ? _cond_resched+0x30/0x40 [ 922.659552] [] ? down_read+0x29/0x40 [ 922.664773] [] ? user_get_super+0x83/0xf0 [ 922.670430] [] ? vzdq_aquotq_lookset+0xb1/0x160 [vzdquota] [ 922.677554] [] ? iget5_locked+0x139/0x1f0 [ 922.683216] [] ? vzdq_aquotq_lookset+0x0/0x160 [vzdquota] [ 922.690251] [] ? vzdq_aquotq_lookup+0xe2/0x160 [vzdquota] [ 922.697285] [] ? do_lookup+0x1ac/0x210 [ 922.702677] [] ? __link_path_walk+0x89e/0x10c0 [ 922.708760] [] ? __link_path_walk+0xe67/0x10c0 [ 922.714841] [] ? path_walk+0x6a/0xe0 [ 922.720057] [] ? do_path_lookup+0x5b/0xa0 [ 922.725705] [] ? do_filp_open+0xfb/0xca0 [ 922.731270] [] ? __do_page_fault+0x264/0x4f0 [ 922.737182] [] ? cpumask_any_but+0x31/0x50 [ 922.742925] [] ? strncpy_from_user+0x5d/0xa0 [ 922.748840] [] ? alloc_fd+0x92/0x170 [ 922.754055] [] ? do_sys_open+0x69/0x140 [ 922.759532] [] ? sys_open+0x20/0x30 [ 922.764661] [] ? system_call_fastpath+0x16/0x1b [ 942.047963] CT: 1113: started [ OK ] CentOS release 6.2 (Final) Kernel 2.6.32-042stab055.7.debug on an x86_64 ak1-vz3.$1}$2.net login: CentOS release 6.2 (Final) Kernel 2.6.32-042stab055.7.debug on an x86_64 [ 1128.199063] BUG: sleeping function called from invalid context at kernel/rwsem.c:21 [ 1128.206756] in_atomic(): 1, irqs_disabled(): 0, pid: 28194, name: repquota [ 1128.213634] Pid: 28194, comm: repquota veid: 1102 Not tainted 2.6.32-042stab055.7.debug #1 [ 1128.221900] Call Trace: [ 1128.224363] [] ? __might_sleep+0xfe/0x120 [ 1128.230025] [] ? down_read+0x24/0x40 [ 1128.235258] [] ? user_get_super+0x83/0xf0 [ 1128.240924] [] ? vzdq_aquotq_lookset+0xb1/0x160 [vzdquota] [ 1128.248060] [] ? iget5_locked+0x139/0x1f0 [ 1128.253723] [] ? vzdq_aquotq_lookset+0x0/0x160 [vzdquota] [ 1128.260771] [] ? vzdq_aquotq_lookup+0xe2/0x160 [vzdquota] [ 1128.267818] [] ? do_lookup+0x1ac/0x210 [ 1128.273222] [] ? __link_path_walk+0x89e/0x10c0 [ 1128.279311] [] ? __link_path_walk+0xe67/0x10c0 [ 1128.285397] [] ? path_walk+0x6a/0xe0 [ 1128.290618] [] ? do_path_lookup+0x5b/0xa0 [ 1128.296272] [] ? do_filp_open+0xfb/0xca0 [ 1128.301840] [] ? __do_page_fault+0x264/0x4f0 [ 1128.307759] [] ? kern_path+0x2a/0x50 [ 1128.312987] [] ? strncpy_from_user+0x5d/0xa0 [ 1128.318912] [] ? alloc_fd+0x92/0x170 [ 1128.324139] [] ? do_sys_open+0x69/0x140 [ 1128.329624] [] ? sys_open+0x20/0x30 [ 1128.334760] [] ? system_call_fastpath+0x16/0x1b [ 1128.340935] BUG: scheduling while atomic: repquota/28194/0x10000002 [ 1128.347198] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1128.467576] Pid: 28194, comm: repquota veid: 1102 Not tainted 2.6.32-042stab055.7.debug #1 [ 1128.475826] Call Trace: [ 1128.478275] [] ? __schedule_bug+0x66/0x70 [ 1128.483934] [] ? thread_return+0x26a/0x83d [ 1128.489676] [] ? __cond_resched+0x2a/0x40 [ 1128.495329] [] ? _cond_resched+0x30/0x40 [ 1128.500897] [] ? down_read+0x29/0x40 [ 1128.506119] [] ? user_get_super+0x83/0xf0 [ 1128.511776] [] ? vzdq_aquotq_lookset+0xb1/0x160 [vzdquota] [ 1128.518902] [] ? iget5_locked+0x139/0x1f0 [ 1128.524559] [] ? vzdq_aquotq_lookset+0x0/0x160 [vzdquota] [ 1128.531599] [] ? vzdq_aquotq_lookup+0xe2/0x160 [vzdquota] [ 1128.538639] [] ? do_lookup+0x1ac/0x210 [ 1128.544034] [] ? __link_path_walk+0x89e/0x10c0 [ 1128.550122] [] ? __link_path_walk+0xe67/0x10c0 [ 1128.556214] [] ? path_walk+0x6a/0xe0 [ 1128.561435] [] ? do_path_lookup+0x5b/0xa0 [ 1128.567089] [] ? do_filp_open+0xfb/0xca0 [ 1128.572657] [] ? __do_page_fault+0x264/0x4f0 [ 1128.578571] [] ? kern_path+0x2a/0x50 [ 1128.583793] [] ? strncpy_from_user+0x5d/0xa0 [ 1128.589707] [] ? alloc_fd+0x92/0x170 [ 1128.594929] [] ? do_sys_open+0x69/0x140 [ 1128.600411] [] ? sys_open+0x20/0x30 [ 1128.605545] [] ? system_call_fastpath+0x16/0x1b [ 1428.610262] BUG: sleeping function called from invalid context at kernel/rwsem.c:21 [ 1428.617920] in_atomic(): 1, irqs_disabled(): 0, pid: 29385, name: repquota [ 1428.624789] Pid: 29385, comm: repquota veid: 1102 Not tainted 2.6.32-042stab055.7.debug #1 [ 1428.633039] Call Trace: [ 1428.635495] [] ? __might_sleep+0xfe/0x120 [ 1428.641150] [] ? down_read+0x24/0x40 [ 1428.646374] [] ? user_get_super+0x83/0xf0 [ 1428.652034] [] ? vzdq_aquotq_lookset+0xb1/0x160 [vzdquota] [ 1428.659159] [] ? iget5_locked+0x139/0x1f0 [ 1428.664813] [] ? vzdq_aquotq_lookset+0x0/0x160 [vzdquota] [ 1428.671851] [] ? vzdq_aquotq_lookup+0xe2/0x160 [vzdquota] [ 1428.678889] [] ? do_lookup+0x1ac/0x210 [ 1428.684282] [] ? __link_path_walk+0x89e/0x10c0 [ 1428.690376] [] ? __link_path_walk+0xe67/0x10c0 [ 1428.696462] [] ? path_walk+0x6a/0xe0 [ 1428.701683] [] ? do_path_lookup+0x5b/0xa0 [ 1428.707336] [] ? do_filp_open+0xfb/0xca0 [ 1428.712900] [] ? __do_page_fault+0x264/0x4f0 [ 1428.718811] [] ? kern_path+0x2a/0x50 [ 1428.724030] [] ? strncpy_from_user+0x5d/0xa0 [ 1428.729942] [] ? alloc_fd+0x92/0x170 [ 1428.735160] [] ? do_sys_open+0x69/0x140 [ 1428.740637] [] ? sys_open+0x20/0x30 [ 1428.745771] [] ? system_call_fastpath+0x16/0x1b [ 1428.751940] BUG: scheduling while atomic: repquota/29385/0x10000002 [ 1428.758206] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1428.878470] Pid: 29385, comm: repquota veid: 1102 Not tainted 2.6.32-042stab055.7.debug #1 [ 1428.886717] Call Trace: [ 1428.889164] [] ? __schedule_bug+0x66/0x70 [ 1428.894817] [] ? thread_return+0x26a/0x83d [ 1428.900555] [] ? __cond_resched+0x2a/0x40 [ 1428.906211] [] ? _cond_resched+0x30/0x40 [ 1428.911776] [] ? down_read+0x29/0x40 [ 1428.916994] [] ? user_get_super+0x83/0xf0 [ 1428.922650] [] ? vzdq_aquotq_lookset+0xb1/0x160 [vzdquota] [ 1428.929775] [] ? iget5_locked+0x139/0x1f0 [ 1428.935429] [] ? vzdq_aquotq_lookset+0x0/0x160 [vzdquota] [ 1428.942468] [] ? vzdq_aquotq_lookup+0xe2/0x160 [vzdquota] [ 1428.949507] [] ? do_lookup+0x1ac/0x210 [ 1428.954899] [] ? __link_path_walk+0x89e/0x10c0 [ 1428.960985] [] ? __link_path_walk+0xe67/0x10c0 [ 1428.967075] [] ? path_walk+0x6a/0xe0 [ 1428.972294] [] ? do_path_lookup+0x5b/0xa0 [ 1428.977946] [] ? do_filp_open+0xfb/0xca0 [ 1428.983512] [] ? __do_page_fault+0x264/0x4f0 [ 1428.989424] [] ? kern_path+0x2a/0x50 [ 1428.994647] [] ? strncpy_from_user+0x5d/0xa0 [ 1429.000564] [] ? alloc_fd+0x92/0x170 [ 1429.005783] [] ? do_sys_open+0x69/0x140 [ 1429.011263] [] ? sys_open+0x20/0x30 [ 1429.016397] [] ? system_call_fastpath+0x16/0x1b [ 1512.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 1512.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 1512.139006] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 1512.139009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1512.139069] CPU 1 [ 1512.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1512.139069] [ 1512.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1512.139069] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 1512.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 1512.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 1512.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1512.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 1512.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 1512.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 1512.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 1512.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1512.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 1512.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1512.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1512.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 1512.139069] Stack: [ 1512.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 1512.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 1512.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 1512.139069] Call Trace: [ 1512.139069] [] ? new_inode+0x3b/0xc0 [ 1512.139069] [] ? create_write_pipe+0x35/0x1f0 [ 1512.139069] [] ? do_sigaction+0x1bb/0x200 [ 1512.139069] [] ? do_pipe_flags+0x57/0x130 [ 1512.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 1512.139069] [] ? sys_pipe2+0x24/0x80 [ 1512.139069] [] ? sys_pipe+0x10/0x20 [ 1512.139069] [] ? system_call_fastpath+0x16/0x1b [ 1512.139069] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 1512.139069] Call Trace: [ 1512.139069] [] ? new_inode+0x3b/0xc0 [ 1512.139069] [] ? create_write_pipe+0x35/0x1f0 [ 1512.139069] [] ? do_sigaction+0x1bb/0x200 [ 1512.139069] [] ? do_pipe_flags+0x57/0x130 [ 1512.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 1512.139069] [] ? sys_pipe2+0x24/0x80 [ 1512.139069] [] ? sys_pipe+0x10/0x20 [ 1512.139069] [] ? system_call_fastpath+0x16/0x1b [ 1512.203012] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 1512.203017] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1512.203118] CPU 2 [ 1512.203119] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1512.203129] [ 1512.203129] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1512.203129] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 1512.203129] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 1512.203129] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 1512.203129] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 1512.203129] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 1512.203129] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 1512.203129] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 1512.203129] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 1512.203129] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 1512.203129] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 1512.203129] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1512.203129] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1512.203129] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 1512.203129] Stack: [ 1512.203129] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 1512.203129] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 1512.203129] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 1512.203129] Call Trace: [ 1512.203129] [] ? _atomic_dec_and_lock+0x55/0x90 [ 1512.203129] [] ? iput+0x30/0x70 [ 1512.203129] [] ? dentry_iput+0xb8/0x140 [ 1512.203129] [] ? d_kill+0x3c/0x70 [ 1512.203129] [] ? prune_one_dentry+0xb1/0xd0 [ 1512.203129] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 1512.203129] [] ? shrink_dcache_ub+0x29/0x70 [ 1512.203129] [] ? ub_dcache_shrink+0x3e/0x50 [ 1512.203129] [] ? ub_dcache_charge+0x96/0x120 [ 1512.203129] [] ? d_alloc+0x45/0x280 [ 1512.203129] [] ? do_lookup+0x18d/0x210 [ 1512.203129] [] ? __link_path_walk+0x89e/0x10c0 [ 1512.203129] [] ? vma_prio_tree_insert+0x30/0x50 [ 1512.203129] [] ? path_walk+0x6a/0xe0 [ 1512.203129] [] ? do_path_lookup+0x5b/0xa0 [ 1512.203129] [] ? user_path_at+0x59/0xa0 [ 1512.203129] [] ? __do_page_fault+0x264/0x4f0 [ 1512.203129] [] ? vfs_fstatat+0x46/0x80 [ 1512.203129] [] ? vfs_lstat+0x1e/0x20 [ 1512.203129] [] ? sys32_lstat64+0x24/0x40 [ 1512.203129] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 1512.203129] [] ? ia32_tracesys+0x48/0x8c [ 1512.203129] [] ? ia32_sysret+0x0/0x5 [ 1512.203129] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 1512.203129] Call Trace: [ 1512.203129] [] ? _atomic_dec_and_lock+0x55/0x90 [ 1512.203129] [] ? iput+0x30/0x70 [ 1512.203129] [] ? dentry_iput+0xb8/0x140 [ 1512.203129] [] ? d_kill+0x3c/0x70 [ 1512.203129] [] ? prune_one_dentry+0xb1/0xd0 [ 1512.203129] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 1512.203129] [] ? shrink_dcache_ub+0x29/0x70 [ 1512.203129] [] ? ub_dcache_shrink+0x3e/0x50 [ 1512.203129] [] ? ub_dcache_charge+0x96/0x120 [ 1512.203129] [] ? d_alloc+0x45/0x280 [ 1512.203129] [] ? do_lookup+0x18d/0x210 [ 1512.203129] [] ? __link_path_walk+0x89e/0x10c0 [ 1512.203129] [] ? vma_prio_tree_insert+0x30/0x50 [ 1512.203129] [] ? path_walk+0x6a/0xe0 [ 1512.203129] [] ? do_path_lookup+0x5b/0xa0 [ 1512.203129] [] ? user_path_at+0x59/0xa0 [ 1512.203129] [] ? __do_page_fault+0x264/0x4f0 [ 1512.203129] [] ? vfs_fstatat+0x46/0x80 [ 1512.203129] [] ? vfs_lstat+0x1e/0x20 [ 1512.203129] [] ? sys32_lstat64+0x24/0x40 [ 1512.203129] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 1512.203129] [] ? ia32_tracesys+0x48/0x8c [ 1512.203129] [] ? ia32_sysret+0x0/0x5 [ 1512.265010] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 1512.265014] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1512.265131] CPU 3 [ 1512.265133] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1512.265193] [ 1512.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1512.265193] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 1512.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 1512.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 1512.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1512.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 1512.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 1512.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 1512.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 1512.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1512.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 1512.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1512.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1512.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 1512.265193] Stack: [ 1512.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 1512.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 1512.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 1512.265193] Call Trace: [ 1512.265193] [] ? new_inode+0x3b/0xc0 [ 1512.265193] [] ? create_write_pipe+0x35/0x1f0 [ 1512.265193] [] ? do_sigaction+0x1bb/0x200 [ 1512.265193] [] ? do_pipe_flags+0x57/0x130 [ 1512.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 1512.265193] [] ? sys_pipe2+0x24/0x80 [ 1512.265193] [] ? sys_pipe+0x10/0x20 [ 1512.265193] [] ? system_call_fastpath+0x16/0x1b [ 1512.265193] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 1512.265193] Call Trace: [ 1512.265193] [] ? new_inode+0x3b/0xc0 [ 1512.265193] [] ? create_write_pipe+0x35/0x1f0 [ 1512.265193] [] ? do_sigaction+0x1bb/0x200 [ 1512.265193] [] ? do_pipe_flags+0x57/0x130 [ 1512.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 1512.265193] [] ? sys_pipe2+0x24/0x80 [ 1512.265193] [] ? sys_pipe+0x10/0x20 [ 1512.265193] [] ? system_call_fastpath+0x16/0x1b [ 1512.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1512.076003] CPU 0 [ 1512.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1512.076003] [ 1512.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1512.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 1512.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 1512.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 1512.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1512.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 1512.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 1512.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 1512.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 1512.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1512.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 1512.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1512.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1512.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 1512.076003] Stack: [ 1512.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 1512.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 1512.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 1512.076003] Call Trace: [ 1512.076003] [] ? new_inode+0x3b/0xc0 [ 1512.076003] [] ? create_write_pipe+0x35/0x1f0 [ 1512.076003] [] ? do_sigaction+0x1bb/0x200 [ 1512.076003] [] ? do_pipe_flags+0x57/0x130 [ 1512.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 1512.076003] [] ? sys_pipe2+0x24/0x80 [ 1512.076003] [] ? sys_pipe+0x10/0x20 [ 1512.076003] [] ? system_call_fastpath+0x16/0x1b [ 1512.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 1512.076003] Call Trace: [ 1512.076003] [] ? new_inode+0x3b/0xc0 [ 1512.076003] [] ? create_write_pipe+0x35/0x1f0 [ 1512.076003] [] ? do_sigaction+0x1bb/0x200 [ 1512.076003] [] ? do_pipe_flags+0x57/0x130 [ 1512.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 1512.076003] [] ? sys_pipe2+0x24/0x80 [ 1512.076003] [] ? sys_pipe+0x10/0x20 [ 1512.076003] [] ? system_call_fastpath+0x16/0x1b [ 1596.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 1596.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 1596.139006] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 1596.139009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1596.139069] CPU 1 [ 1596.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1596.139069] [ 1596.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1596.139069] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 1596.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 1596.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 1596.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1596.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 1596.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 1596.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 1596.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 1596.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1596.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 1596.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1596.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1596.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 1596.139069] Stack: [ 1596.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 1596.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 1596.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 1596.139069] Call Trace: [ 1596.139069] [] ? new_inode+0x3b/0xc0 [ 1596.139069] [] ? create_write_pipe+0x35/0x1f0 [ 1596.139069] [] ? do_sigaction+0x1bb/0x200 [ 1596.139069] [] ? do_pipe_flags+0x57/0x130 [ 1596.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 1596.139069] [] ? sys_pipe2+0x24/0x80 [ 1596.139069] [] ? sys_pipe+0x10/0x20 [ 1596.139069] [] ? system_call_fastpath+0x16/0x1b [ 1596.139069] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 1596.139069] Call Trace: [ 1596.139069] [] ? new_inode+0x3b/0xc0 [ 1596.139069] [] ? create_write_pipe+0x35/0x1f0 [ 1596.139069] [] ? do_sigaction+0x1bb/0x200 [ 1596.139069] [] ? do_pipe_flags+0x57/0x130 [ 1596.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 1596.139069] [] ? sys_pipe2+0x24/0x80 [ 1596.139069] [] ? sys_pipe+0x10/0x20 [ 1596.139069] [] ? system_call_fastpath+0x16/0x1b [ 1596.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 1596.203010] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1596.203097] CPU 2 [ 1596.203098] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1596.203131] [ 1596.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1596.203131] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 1596.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 1596.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 1596.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 1596.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 1596.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 1596.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 1596.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 1596.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 1596.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 1596.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1596.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1596.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 1596.203131] Stack: [ 1596.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 1596.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 1596.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 1596.203131] Call Trace: [ 1596.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 1596.203131] [] ? iput+0x30/0x70 [ 1596.203131] [] ? dentry_iput+0xb8/0x140 [ 1596.203131] [] ? d_kill+0x3c/0x70 [ 1596.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 1596.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 1596.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 1596.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 1596.203131] [] ? ub_dcache_charge+0x96/0x120 [ 1596.203131] [] ? d_alloc+0x45/0x280 [ 1596.203131] [] ? do_lookup+0x18d/0x210 [ 1596.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 1596.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 1596.203131] [] ? path_walk+0x6a/0xe0 [ 1596.203131] [] ? do_path_lookup+0x5b/0xa0 [ 1596.203131] [] ? user_path_at+0x59/0xa0 [ 1596.203131] [] ? __do_page_fault+0x264/0x4f0 [ 1596.203131] [] ? vfs_fstatat+0x46/0x80 [ 1596.203131] [] ? vfs_lstat+0x1e/0x20 [ 1596.203131] [] ? sys32_lstat64+0x24/0x40 [ 1596.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 1596.203131] [] ? ia32_tracesys+0x48/0x8c [ 1596.203131] [] ? ia32_sysret+0x0/0x5 [ 1596.203131] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 1596.203131] Call Trace: [ 1596.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 1596.203131] [] ? iput+0x30/0x70 [ 1596.203131] [] ? dentry_iput+0xb8/0x140 [ 1596.203131] [] ? d_kill+0x3c/0x70 [ 1596.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 1596.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 1596.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 1596.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 1596.203131] [] ? ub_dcache_charge+0x96/0x120 [ 1596.203131] [] ? d_alloc+0x45/0x280 [ 1596.203131] [] ? do_lookup+0x18d/0x210 [ 1596.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 1596.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 1596.203131] [] ? path_walk+0x6a/0xe0 [ 1596.203131] [] ? do_path_lookup+0x5b/0xa0 [ 1596.203131] [] ? user_path_at+0x59/0xa0 [ 1596.203131] [] ? __do_page_fault+0x264/0x4f0 [ 1596.203131] [] ? vfs_fstatat+0x46/0x80 [ 1596.203131] [] ? vfs_lstat+0x1e/0x20 [ 1596.203131] [] ? sys32_lstat64+0x24/0x40 [ 1596.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 1596.203131] [] ? ia32_tracesys+0x48/0x8c [ 1596.203131] [] ? ia32_sysret+0x0/0x5 [ 1596.265008] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 1596.265011] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1596.265123] CPU 3 [ 1596.265125] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1596.265193] [ 1596.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1596.265193] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 1596.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 1596.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 1596.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1596.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 1596.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 1596.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 1596.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 1596.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1596.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 1596.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1596.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1596.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 1596.265193] Stack: [ 1596.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 1596.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 1596.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 1596.265193] Call Trace: [ 1596.265193] [] ? new_inode+0x3b/0xc0 [ 1596.265193] [] ? create_write_pipe+0x35/0x1f0 [ 1596.265193] [] ? do_sigaction+0x1bb/0x200 [ 1596.265193] [] ? do_pipe_flags+0x57/0x130 [ 1596.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 1596.265193] [] ? sys_pipe2+0x24/0x80 [ 1596.265193] [] ? sys_pipe+0x10/0x20 [ 1596.265193] [] ? system_call_fastpath+0x16/0x1b [ 1596.265193] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 1596.265193] Call Trace: [ 1596.265193] [] ? new_inode+0x3b/0xc0 [ 1596.265193] [] ? create_write_pipe+0x35/0x1f0 [ 1596.265193] [] ? do_sigaction+0x1bb/0x200 [ 1596.265193] [] ? do_pipe_flags+0x57/0x130 [ 1596.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 1596.265193] [] ? sys_pipe2+0x24/0x80 [ 1596.265193] [] ? sys_pipe+0x10/0x20 [ 1596.265193] [] ? system_call_fastpath+0x16/0x1b [ 1596.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1596.076003] CPU 0 [ 1596.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1596.076003] [ 1596.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1596.076003] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 1596.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 1596.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 1596.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1596.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 1596.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 1596.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 1596.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 1596.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1596.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 1596.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1596.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1596.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 1596.076003] Stack: [ 1596.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 1596.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 1596.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 1596.076003] Call Trace: [ 1596.076003] [] ? new_inode+0x3b/0xc0 [ 1596.076003] [] ? create_write_pipe+0x35/0x1f0 [ 1596.076003] [] ? do_sigaction+0x1bb/0x200 [ 1596.076003] [] ? do_pipe_flags+0x57/0x130 [ 1596.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 1596.076003] [] ? sys_pipe2+0x24/0x80 [ 1596.076003] [] ? sys_pipe+0x10/0x20 [ 1596.076003] [] ? system_call_fastpath+0x16/0x1b [ 1596.076003] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 1596.076003] Call Trace: [ 1596.076003] [] ? new_inode+0x3b/0xc0 [ 1596.076003] [] ? create_write_pipe+0x35/0x1f0 [ 1596.076003] [] ? do_sigaction+0x1bb/0x200 [ 1596.076003] [] ? do_pipe_flags+0x57/0x130 [ 1596.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 1596.076003] [] ? sys_pipe2+0x24/0x80 [ 1596.076003] [] ? sys_pipe+0x10/0x20 [ 1596.076003] [] ? system_call_fastpath+0x16/0x1b [ 1680.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 1680.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 1680.139005] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 1680.139008] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1680.139069] CPU 1 [ 1680.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1680.139069] [ 1680.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1680.139069] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 1680.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 1680.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 1680.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1680.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 1680.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 1680.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 1680.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 1680.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1680.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 1680.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1680.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1680.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 1680.139069] Stack: [ 1680.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 1680.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 1680.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 1680.139069] Call Trace: [ 1680.139069] [] ? new_inode+0x3b/0xc0 [ 1680.139069] [] ? create_write_pipe+0x35/0x1f0 [ 1680.139069] [] ? do_sigaction+0x1bb/0x200 [ 1680.139069] [] ? do_pipe_flags+0x57/0x130 [ 1680.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 1680.139069] [] ? sys_pipe2+0x24/0x80 [ 1680.139069] [] ? sys_pipe+0x10/0x20 [ 1680.139069] [] ? system_call_fastpath+0x16/0x1b [ 1680.139069] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 1680.139069] Call Trace: [ 1680.139069] [] ? new_inode+0x3b/0xc0 [ 1680.139069] [] ? create_write_pipe+0x35/0x1f0 [ 1680.139069] [] ? do_sigaction+0x1bb/0x200 [ 1680.139069] [] ? do_pipe_flags+0x57/0x130 [ 1680.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 1680.139069] [] ? sys_pipe2+0x24/0x80 [ 1680.139069] [] ? sys_pipe+0x10/0x20 [ 1680.139069] [] ? system_call_fastpath+0x16/0x1b [ 1680.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 1680.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1680.203096] CPU 2 [ 1680.203098] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1680.203131] [ 1680.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1680.203131] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 1680.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 1680.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 1680.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 1680.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 1680.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 1680.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 1680.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 1680.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 1680.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 1680.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1680.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1680.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 1680.203131] Stack: [ 1680.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 1680.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 1680.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 1680.203131] Call Trace: [ 1680.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 1680.203131] [] ? iput+0x30/0x70 [ 1680.203131] [] ? dentry_iput+0xb8/0x140 [ 1680.203131] [] ? d_kill+0x3c/0x70 [ 1680.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 1680.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 1680.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 1680.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 1680.203131] [] ? ub_dcache_charge+0x96/0x120 [ 1680.203131] [] ? d_alloc+0x45/0x280 [ 1680.203131] [] ? do_lookup+0x18d/0x210 [ 1680.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 1680.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 1680.203131] [] ? path_walk+0x6a/0xe0 [ 1680.203131] [] ? do_path_lookup+0x5b/0xa0 [ 1680.203131] [] ? user_path_at+0x59/0xa0 [ 1680.203131] [] ? __do_page_fault+0x264/0x4f0 [ 1680.203131] [] ? vfs_fstatat+0x46/0x80 [ 1680.203131] [] ? vfs_lstat+0x1e/0x20 [ 1680.203131] [] ? sys32_lstat64+0x24/0x40 [ 1680.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 1680.203131] [] ? ia32_tracesys+0x48/0x8c [ 1680.203131] [] ? ia32_sysret+0x0/0x5 [ 1680.203131] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 1680.203131] Call Trace: [ 1680.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 1680.203131] [] ? iput+0x30/0x70 [ 1680.203131] [] ? dentry_iput+0xb8/0x140 [ 1680.203131] [] ? d_kill+0x3c/0x70 [ 1680.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 1680.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 1680.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 1680.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 1680.203131] [] ? ub_dcache_charge+0x96/0x120 [ 1680.203131] [] ? d_alloc+0x45/0x280 [ 1680.203131] [] ? do_lookup+0x18d/0x210 [ 1680.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 1680.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 1680.203131] [] ? path_walk+0x6a/0xe0 [ 1680.203131] [] ? do_path_lookup+0x5b/0xa0 [ 1680.203131] [] ? user_path_at+0x59/0xa0 [ 1680.203131] [] ? __do_page_fault+0x264/0x4f0 [ 1680.203131] [] ? vfs_fstatat+0x46/0x80 [ 1680.203131] [] ? vfs_lstat+0x1e/0x20 [ 1680.203131] [] ? sys32_lstat64+0x24/0x40 [ 1680.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 1680.203131] [] ? ia32_tracesys+0x48/0x8c [ 1680.203131] [] ? ia32_sysret+0x0/0x5 [ 1680.265007] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 1680.265010] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1680.265125] CPU 3 [ 1680.265127] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1680.265192] [ 1680.265192] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1680.265192] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 1680.265192] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 1680.265192] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 1680.265192] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1680.265192] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 1680.265192] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 1680.265192] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 1680.265192] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 1680.265192] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1680.265192] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 1680.265192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1680.265192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1680.265192] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 1680.265192] Stack: [ 1680.265192] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 1680.265192] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 1680.265192] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 1680.265192] Call Trace: [ 1680.265192] [] ? new_inode+0x3b/0xc0 [ 1680.265192] [] ? create_write_pipe+0x35/0x1f0 [ 1680.265192] [] ? do_sigaction+0x1bb/0x200 [ 1680.265192] [] ? do_pipe_flags+0x57/0x130 [ 1680.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 1680.265192] [] ? sys_pipe2+0x24/0x80 [ 1680.265192] [] ? sys_pipe+0x10/0x20 [ 1680.265192] [] ? system_call_fastpath+0x16/0x1b [ 1680.265192] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 1680.265192] Call Trace: [ 1680.265192] [] ? new_inode+0x3b/0xc0 [ 1680.265192] [] ? create_write_pipe+0x35/0x1f0 [ 1680.265192] [] ? do_sigaction+0x1bb/0x200 [ 1680.265192] [] ? do_pipe_flags+0x57/0x130 [ 1680.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 1680.265192] [] ? sys_pipe2+0x24/0x80 [ 1680.265192] [] ? sys_pipe+0x10/0x20 [ 1680.265192] [] ? system_call_fastpath+0x16/0x1b [ 1680.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1680.076003] CPU 0 [ 1680.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1680.076003] [ 1680.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1680.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 1680.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 1680.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 1680.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1680.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 1680.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 1680.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 1680.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 1680.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1680.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 1680.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1680.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1680.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 1680.076003] Stack: [ 1680.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 1680.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 1680.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 1680.076003] Call Trace: [ 1680.076003] [] ? new_inode+0x3b/0xc0 [ 1680.076003] [] ? create_write_pipe+0x35/0x1f0 [ 1680.076003] [] ? do_sigaction+0x1bb/0x200 [ 1680.076003] [] ? do_pipe_flags+0x57/0x130 [ 1680.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 1680.076003] [] ? sys_pipe2+0x24/0x80 [ 1680.076003] [] ? sys_pipe+0x10/0x20 [ 1680.076003] [] ? system_call_fastpath+0x16/0x1b [ 1680.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 1680.076003] Call Trace: [ 1680.076003] [] ? new_inode+0x3b/0xc0 [ 1680.076003] [] ? create_write_pipe+0x35/0x1f0 [ 1680.076003] [] ? do_sigaction+0x1bb/0x200 [ 1680.076003] [] ? do_pipe_flags+0x57/0x130 [ 1680.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 1680.076003] [] ? sys_pipe2+0x24/0x80 [ 1680.076003] [] ? sys_pipe+0x10/0x20 [ 1680.076003] [] ? system_call_fastpath+0x16/0x1b [ 1764.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 1764.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 1764.139005] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 1764.139008] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1764.139069] CPU 1 [ 1764.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1764.139069] [ 1764.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1764.139069] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 1764.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 1764.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 1764.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1764.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 1764.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 1764.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 1764.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 1764.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1764.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 1764.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1764.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1764.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 1764.139069] Stack: [ 1764.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 1764.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 1764.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 1764.139069] Call Trace: [ 1764.139069] [] ? new_inode+0x3b/0xc0 [ 1764.139069] [] ? create_write_pipe+0x35/0x1f0 [ 1764.139069] [] ? do_sigaction+0x1bb/0x200 [ 1764.139069] [] ? do_pipe_flags+0x57/0x130 [ 1764.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 1764.139069] [] ? sys_pipe2+0x24/0x80 [ 1764.139069] [] ? sys_pipe+0x10/0x20 [ 1764.139069] [] ? system_call_fastpath+0x16/0x1b [ 1764.139069] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 1764.139069] Call Trace: [ 1764.139069] [] ? new_inode+0x3b/0xc0 [ 1764.139069] [] ? create_write_pipe+0x35/0x1f0 [ 1764.139069] [] ? do_sigaction+0x1bb/0x200 [ 1764.139069] [] ? do_pipe_flags+0x57/0x130 [ 1764.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 1764.139069] [] ? sys_pipe2+0x24/0x80 [ 1764.139069] [] ? sys_pipe+0x10/0x20 [ 1764.139069] [] ? system_call_fastpath+0x16/0x1b [ 1764.203007] BUG: soft lockup - CPU#2 stuck for 68s! [cvs:29276] [ 1764.203010] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1764.203097] CPU 2 [ 1764.203098] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1764.203131] [ 1764.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1764.203131] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 1764.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 1764.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 1764.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 1764.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 1764.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 1764.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 1764.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 1764.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 1764.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 1764.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1764.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1764.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 1764.203131] Stack: [ 1764.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 1764.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 1764.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 1764.203131] Call Trace: [ 1764.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 1764.203131] [] ? iput+0x30/0x70 [ 1764.203131] [] ? dentry_iput+0xb8/0x140 [ 1764.203131] [] ? d_kill+0x3c/0x70 [ 1764.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 1764.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 1764.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 1764.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 1764.203131] [] ? ub_dcache_charge+0x96/0x120 [ 1764.203131] [] ? d_alloc+0x45/0x280 [ 1764.203131] [] ? do_lookup+0x18d/0x210 [ 1764.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 1764.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 1764.203131] [] ? path_walk+0x6a/0xe0 [ 1764.203131] [] ? do_path_lookup+0x5b/0xa0 [ 1764.203131] [] ? user_path_at+0x59/0xa0 [ 1764.203131] [] ? __do_page_fault+0x264/0x4f0 [ 1764.203131] [] ? vfs_fstatat+0x46/0x80 [ 1764.203131] [] ? vfs_lstat+0x1e/0x20 [ 1764.203131] [] ? sys32_lstat64+0x24/0x40 [ 1764.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 1764.203131] [] ? ia32_tracesys+0x48/0x8c [ 1764.203131] [] ? ia32_sysret+0x0/0x5 [ 1764.203131] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 1764.203131] Call Trace: [ 1764.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 1764.203131] [] ? iput+0x30/0x70 [ 1764.203131] [] ? dentry_iput+0xb8/0x140 [ 1764.203131] [] ? d_kill+0x3c/0x70 [ 1764.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 1764.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 1764.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 1764.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 1764.203131] [] ? ub_dcache_charge+0x96/0x120 [ 1764.203131] [] ? d_alloc+0x45/0x280 [ 1764.203131] [] ? do_lookup+0x18d/0x210 [ 1764.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 1764.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 1764.203131] [] ? path_walk+0x6a/0xe0 [ 1764.203131] [] ? do_path_lookup+0x5b/0xa0 [ 1764.203131] [] ? user_path_at+0x59/0xa0 [ 1764.203131] [] ? __do_page_fault+0x264/0x4f0 [ 1764.203131] [] ? vfs_fstatat+0x46/0x80 [ 1764.203131] [] ? vfs_lstat+0x1e/0x20 [ 1764.203131] [] ? sys32_lstat64+0x24/0x40 [ 1764.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 1764.203131] [] ? ia32_tracesys+0x48/0x8c [ 1764.203131] [] ? ia32_sysret+0x0/0x5 [ 1764.265010] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 1764.265014] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1764.265126] CPU 3 [ 1764.265127] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1764.265193] [ 1764.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1764.265193] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 1764.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 1764.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 1764.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1764.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 1764.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 1764.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 1764.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 1764.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1764.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 1764.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1764.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1764.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 1764.265193] Stack: [ 1764.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 1764.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 1764.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 1764.265193] Call Trace: [ 1764.265193] [] ? new_inode+0x3b/0xc0 [ 1764.265193] [] ? create_write_pipe+0x35/0x1f0 [ 1764.265193] [] ? do_sigaction+0x1bb/0x200 [ 1764.265193] [] ? do_pipe_flags+0x57/0x130 [ 1764.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 1764.265193] [] ? sys_pipe2+0x24/0x80 [ 1764.265193] [] ? sys_pipe+0x10/0x20 [ 1764.265193] [] ? system_call_fastpath+0x16/0x1b [ 1764.265193] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 1764.265193] Call Trace: [ 1764.265193] [] ? new_inode+0x3b/0xc0 [ 1764.265193] [] ? create_write_pipe+0x35/0x1f0 [ 1764.265193] [] ? do_sigaction+0x1bb/0x200 [ 1764.265193] [] ? do_pipe_flags+0x57/0x130 [ 1764.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 1764.265193] [] ? sys_pipe2+0x24/0x80 [ 1764.265193] [] ? sys_pipe+0x10/0x20 [ 1764.265193] [] ? system_call_fastpath+0x16/0x1b [ 1764.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1764.076003] CPU 0 [ 1764.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1764.076003] [ 1764.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1764.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 1764.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 1764.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 1764.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1764.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 1764.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 1764.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 1764.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 1764.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1764.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 1764.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1764.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1764.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 1764.076003] Stack: [ 1764.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 1764.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 1764.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 1764.076003] Call Trace: [ 1764.076003] [] ? new_inode+0x3b/0xc0 [ 1764.076003] [] ? create_write_pipe+0x35/0x1f0 [ 1764.076003] [] ? do_sigaction+0x1bb/0x200 [ 1764.076003] [] ? do_pipe_flags+0x57/0x130 [ 1764.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 1764.076003] [] ? sys_pipe2+0x24/0x80 [ 1764.076003] [] ? sys_pipe+0x10/0x20 [ 1764.076003] [] ? system_call_fastpath+0x16/0x1b [ 1764.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 1764.076003] Call Trace: [ 1764.076003] [] ? new_inode+0x3b/0xc0 [ 1764.076003] [] ? create_write_pipe+0x35/0x1f0 [ 1764.076003] [] ? do_sigaction+0x1bb/0x200 [ 1764.076003] [] ? do_pipe_flags+0x57/0x130 [ 1764.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 1764.076003] [] ? sys_pipe2+0x24/0x80 [ 1764.076003] [] ? sys_pipe+0x10/0x20 [ 1764.076003] [] ? system_call_fastpath+0x16/0x1b [ 1848.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 1848.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 1848.139006] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 1848.139009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1848.139069] CPU 1 [ 1848.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1848.139069] [ 1848.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1848.139069] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 1848.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 1848.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 1848.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1848.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 1848.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 1848.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 1848.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 1848.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1848.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 1848.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1848.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1848.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 1848.139069] Stack: [ 1848.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 1848.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 1848.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 1848.139069] Call Trace: [ 1848.139069] [] ? new_inode+0x3b/0xc0 [ 1848.139069] [] ? create_write_pipe+0x35/0x1f0 [ 1848.139069] [] ? do_sigaction+0x1bb/0x200 [ 1848.139069] [] ? do_pipe_flags+0x57/0x130 [ 1848.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 1848.139069] [] ? sys_pipe2+0x24/0x80 [ 1848.139069] [] ? sys_pipe+0x10/0x20 [ 1848.139069] [] ? system_call_fastpath+0x16/0x1b [ 1848.139069] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 1848.139069] Call Trace: [ 1848.139069] [] ? new_inode+0x3b/0xc0 [ 1848.139069] [] ? create_write_pipe+0x35/0x1f0 [ 1848.139069] [] ? do_sigaction+0x1bb/0x200 [ 1848.139069] [] ? do_pipe_flags+0x57/0x130 [ 1848.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 1848.139069] [] ? sys_pipe2+0x24/0x80 [ 1848.139069] [] ? sys_pipe+0x10/0x20 [ 1848.139069] [] ? system_call_fastpath+0x16/0x1b [ 1848.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 1848.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1848.203096] CPU 2 [ 1848.203098] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1848.203131] [ 1848.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1848.203131] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 1848.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 1848.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 1848.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 1848.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 1848.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 1848.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 1848.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 1848.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 1848.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 1848.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1848.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1848.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 1848.203131] Stack: [ 1848.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 1848.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 1848.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 1848.203131] Call Trace: [ 1848.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 1848.203131] [] ? iput+0x30/0x70 [ 1848.203131] [] ? dentry_iput+0xb8/0x140 [ 1848.203131] [] ? d_kill+0x3c/0x70 [ 1848.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 1848.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 1848.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 1848.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 1848.203131] [] ? ub_dcache_charge+0x96/0x120 [ 1848.203131] [] ? d_alloc+0x45/0x280 [ 1848.203131] [] ? do_lookup+0x18d/0x210 [ 1848.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 1848.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 1848.203131] [] ? path_walk+0x6a/0xe0 [ 1848.203131] [] ? do_path_lookup+0x5b/0xa0 [ 1848.203131] [] ? user_path_at+0x59/0xa0 [ 1848.203131] [] ? __do_page_fault+0x264/0x4f0 [ 1848.203131] [] ? vfs_fstatat+0x46/0x80 [ 1848.203131] [] ? vfs_lstat+0x1e/0x20 [ 1848.203131] [] ? sys32_lstat64+0x24/0x40 [ 1848.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 1848.203131] [] ? ia32_tracesys+0x48/0x8c [ 1848.203131] [] ? ia32_sysret+0x0/0x5 [ 1848.203131] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 1848.203131] Call Trace: [ 1848.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 1848.203131] [] ? iput+0x30/0x70 [ 1848.203131] [] ? dentry_iput+0xb8/0x140 [ 1848.203131] [] ? d_kill+0x3c/0x70 [ 1848.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 1848.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 1848.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 1848.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 1848.203131] [] ? ub_dcache_charge+0x96/0x120 [ 1848.203131] [] ? d_alloc+0x45/0x280 [ 1848.203131] [] ? do_lookup+0x18d/0x210 [ 1848.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 1848.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 1848.203131] [] ? path_walk+0x6a/0xe0 [ 1848.203131] [] ? do_path_lookup+0x5b/0xa0 [ 1848.203131] [] ? user_path_at+0x59/0xa0 [ 1848.203131] [] ? __do_page_fault+0x264/0x4f0 [ 1848.203131] [] ? vfs_fstatat+0x46/0x80 [ 1848.203131] [] ? vfs_lstat+0x1e/0x20 [ 1848.203131] [] ? sys32_lstat64+0x24/0x40 [ 1848.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 1848.203131] [] ? ia32_tracesys+0x48/0x8c [ 1848.203131] [] ? ia32_sysret+0x0/0x5 [ 1848.265007] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 1848.265011] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1848.265129] CPU 3 [ 1848.265131] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1848.265192] [ 1848.265192] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1848.265192] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 1848.265192] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 1848.265192] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 1848.265192] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1848.265192] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 1848.265192] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 1848.265192] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 1848.265192] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 1848.265192] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1848.265192] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 1848.265192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1848.265192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1848.265192] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 1848.265192] Stack: [ 1848.265192] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 1848.265192] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 1848.265192] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 1848.265192] Call Trace: [ 1848.265192] [] ? new_inode+0x3b/0xc0 [ 1848.265192] [] ? create_write_pipe+0x35/0x1f0 [ 1848.265192] [] ? do_sigaction+0x1bb/0x200 [ 1848.265192] [] ? do_pipe_flags+0x57/0x130 [ 1848.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 1848.265192] [] ? sys_pipe2+0x24/0x80 [ 1848.265192] [] ? sys_pipe+0x10/0x20 [ 1848.265192] [] ? system_call_fastpath+0x16/0x1b [ 1848.265192] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 1848.265192] Call Trace: [ 1848.265192] [] ? new_inode+0x3b/0xc0 [ 1848.265192] [] ? create_write_pipe+0x35/0x1f0 [ 1848.265192] [] ? do_sigaction+0x1bb/0x200 [ 1848.265192] [] ? do_pipe_flags+0x57/0x130 [ 1848.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 1848.265192] [] ? sys_pipe2+0x24/0x80 [ 1848.265192] [] ? sys_pipe+0x10/0x20 [ 1848.265192] [] ? system_call_fastpath+0x16/0x1b [ 1848.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1848.076003] CPU 0 [ 1848.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1848.076003] [ 1848.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1848.076003] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 1848.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 1848.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 1848.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1848.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 1848.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 1848.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 1848.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 1848.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1848.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 1848.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1848.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1848.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 1848.076003] Stack: [ 1848.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 1848.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 1848.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 1848.076003] Call Trace: [ 1848.076003] [] ? new_inode+0x3b/0xc0 [ 1848.076003] [] ? create_write_pipe+0x35/0x1f0 [ 1848.076003] [] ? do_sigaction+0x1bb/0x200 [ 1848.076003] [] ? do_pipe_flags+0x57/0x130 [ 1848.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 1848.076003] [] ? sys_pipe2+0x24/0x80 [ 1848.076003] [] ? sys_pipe+0x10/0x20 [ 1848.076003] [] ? system_call_fastpath+0x16/0x1b [ 1848.076003] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 1848.076003] Call Trace: [ 1848.076003] [] ? new_inode+0x3b/0xc0 [ 1848.076003] [] ? create_write_pipe+0x35/0x1f0 [ 1848.076003] [] ? do_sigaction+0x1bb/0x200 [ 1848.076003] [] ? do_pipe_flags+0x57/0x130 [ 1848.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 1848.076003] [] ? sys_pipe2+0x24/0x80 [ 1848.076003] [] ? sys_pipe+0x10/0x20 [ 1848.076003] [] ? system_call_fastpath+0x16/0x1b [ 1932.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 1932.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 1932.139005] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 1932.139007] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1932.139069] CPU 1 [ 1932.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1932.139069] [ 1932.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1932.139069] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 1932.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 1932.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 1932.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1932.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 1932.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 1932.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 1932.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 1932.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1932.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 1932.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1932.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1932.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 1932.139069] Stack: [ 1932.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 1932.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 1932.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 1932.139069] Call Trace: [ 1932.139069] [] ? new_inode+0x3b/0xc0 [ 1932.139069] [] ? create_write_pipe+0x35/0x1f0 [ 1932.139069] [] ? do_sigaction+0x1bb/0x200 [ 1932.139069] [] ? do_pipe_flags+0x57/0x130 [ 1932.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 1932.139069] [] ? sys_pipe2+0x24/0x80 [ 1932.139069] [] ? sys_pipe+0x10/0x20 [ 1932.139069] [] ? system_call_fastpath+0x16/0x1b [ 1932.139069] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 1932.139069] Call Trace: [ 1932.139069] [] ? new_inode+0x3b/0xc0 [ 1932.139069] [] ? create_write_pipe+0x35/0x1f0 [ 1932.139069] [] ? do_sigaction+0x1bb/0x200 [ 1932.139069] [] ? do_pipe_flags+0x57/0x130 [ 1932.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 1932.139069] [] ? sys_pipe2+0x24/0x80 [ 1932.139069] [] ? sys_pipe+0x10/0x20 [ 1932.139069] [] ? system_call_fastpath+0x16/0x1b [ 1932.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 1932.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1932.203096] CPU 2 [ 1932.203097] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1932.203131] [ 1932.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1932.203131] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 1932.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 1932.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 1932.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 1932.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 1932.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 1932.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 1932.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 1932.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 1932.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 1932.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1932.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1932.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 1932.203131] Stack: [ 1932.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 1932.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 1932.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 1932.203131] Call Trace: [ 1932.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 1932.203131] [] ? iput+0x30/0x70 [ 1932.203131] [] ? dentry_iput+0xb8/0x140 [ 1932.203131] [] ? d_kill+0x3c/0x70 [ 1932.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 1932.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 1932.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 1932.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 1932.203131] [] ? ub_dcache_charge+0x96/0x120 [ 1932.203131] [] ? d_alloc+0x45/0x280 [ 1932.203131] [] ? do_lookup+0x18d/0x210 [ 1932.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 1932.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 1932.203131] [] ? path_walk+0x6a/0xe0 [ 1932.203131] [] ? do_path_lookup+0x5b/0xa0 [ 1932.203131] [] ? user_path_at+0x59/0xa0 [ 1932.203131] [] ? __do_page_fault+0x264/0x4f0 [ 1932.203131] [] ? vfs_fstatat+0x46/0x80 [ 1932.203131] [] ? vfs_lstat+0x1e/0x20 [ 1932.203131] [] ? sys32_lstat64+0x24/0x40 [ 1932.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 1932.203131] [] ? ia32_tracesys+0x48/0x8c [ 1932.203131] [] ? ia32_sysret+0x0/0x5 [ 1932.203131] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 1932.203131] Call Trace: [ 1932.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 1932.203131] [] ? iput+0x30/0x70 [ 1932.203131] [] ? dentry_iput+0xb8/0x140 [ 1932.203131] [] ? d_kill+0x3c/0x70 [ 1932.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 1932.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 1932.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 1932.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 1932.203131] [] ? ub_dcache_charge+0x96/0x120 [ 1932.203131] [] ? d_alloc+0x45/0x280 [ 1932.203131] [] ? do_lookup+0x18d/0x210 [ 1932.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 1932.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 1932.203131] [] ? path_walk+0x6a/0xe0 [ 1932.203131] [] ? do_path_lookup+0x5b/0xa0 [ 1932.203131] [] ? user_path_at+0x59/0xa0 [ 1932.203131] [] ? __do_page_fault+0x264/0x4f0 [ 1932.203131] [] ? vfs_fstatat+0x46/0x80 [ 1932.203131] [] ? vfs_lstat+0x1e/0x20 [ 1932.203131] [] ? sys32_lstat64+0x24/0x40 [ 1932.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 1932.203131] [] ? ia32_tracesys+0x48/0x8c [ 1932.203131] [] ? ia32_sysret+0x0/0x5 [ 1932.265008] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 1932.265012] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1932.265123] CPU 3 [ 1932.265125] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1932.265193] [ 1932.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1932.265193] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 1932.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 1932.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 1932.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1932.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 1932.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 1932.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 1932.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 1932.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1932.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 1932.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1932.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1932.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 1932.265193] Stack: [ 1932.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 1932.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 1932.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 1932.265193] Call Trace: [ 1932.265193] [] ? new_inode+0x3b/0xc0 [ 1932.265193] [] ? create_write_pipe+0x35/0x1f0 [ 1932.265193] [] ? do_sigaction+0x1bb/0x200 [ 1932.265193] [] ? do_pipe_flags+0x57/0x130 [ 1932.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 1932.265193] [] ? sys_pipe2+0x24/0x80 [ 1932.265193] [] ? sys_pipe+0x10/0x20 [ 1932.265193] [] ? system_call_fastpath+0x16/0x1b [ 1932.265193] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 1932.265193] Call Trace: [ 1932.265193] [] ? new_inode+0x3b/0xc0 [ 1932.265193] [] ? create_write_pipe+0x35/0x1f0 [ 1932.265193] [] ? do_sigaction+0x1bb/0x200 [ 1932.265193] [] ? do_pipe_flags+0x57/0x130 [ 1932.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 1932.265193] [] ? sys_pipe2+0x24/0x80 [ 1932.265193] [] ? sys_pipe+0x10/0x20 [ 1932.265193] [] ? system_call_fastpath+0x16/0x1b [ 1932.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1932.076003] CPU 0 [ 1932.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 1932.076003] [ 1932.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 1932.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 1932.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 1932.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 1932.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 1932.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 1932.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 1932.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 1932.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 1932.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1932.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 1932.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1932.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1932.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 1932.076003] Stack: [ 1932.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 1932.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 1932.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 1932.076003] Call Trace: [ 1932.076003] [] ? new_inode+0x3b/0xc0 [ 1932.076003] [] ? create_write_pipe+0x35/0x1f0 [ 1932.076003] [] ? do_sigaction+0x1bb/0x200 [ 1932.076003] [] ? do_pipe_flags+0x57/0x130 [ 1932.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 1932.076003] [] ? sys_pipe2+0x24/0x80 [ 1932.076003] [] ? sys_pipe+0x10/0x20 [ 1932.076003] [] ? system_call_fastpath+0x16/0x1b [ 1932.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 1932.076003] Call Trace: [ 1932.076003] [] ? new_inode+0x3b/0xc0 [ 1932.076003] [] ? create_write_pipe+0x35/0x1f0 [ 1932.076003] [] ? do_sigaction+0x1bb/0x200 [ 1932.076003] [] ? do_pipe_flags+0x57/0x130 [ 1932.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 1932.076003] [] ? sys_pipe2+0x24/0x80 [ 1932.076003] [] ? sys_pipe+0x10/0x20 [ 1932.076003] [] ? system_call_fastpath+0x16/0x1b [ 2016.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 2016.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 2016.139005] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 2016.139008] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2016.139069] CPU 1 [ 2016.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2016.139069] [ 2016.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2016.139069] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2016.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 2016.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 2016.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2016.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 2016.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 2016.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 2016.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 2016.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2016.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 2016.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2016.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2016.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 2016.139069] Stack: [ 2016.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 2016.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 2016.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 2016.139069] Call Trace: [ 2016.139069] [] ? new_inode+0x3b/0xc0 [ 2016.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2016.139069] [] ? do_sigaction+0x1bb/0x200 [ 2016.139069] [] ? do_pipe_flags+0x57/0x130 [ 2016.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2016.139069] [] ? sys_pipe2+0x24/0x80 [ 2016.139069] [] ? sys_pipe+0x10/0x20 [ 2016.139069] [] ? system_call_fastpath+0x16/0x1b [ 2016.139069] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2016.139069] Call Trace: [ 2016.139069] [] ? new_inode+0x3b/0xc0 [ 2016.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2016.139069] [] ? do_sigaction+0x1bb/0x200 [ 2016.139069] [] ? do_pipe_flags+0x57/0x130 [ 2016.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2016.139069] [] ? sys_pipe2+0x24/0x80 [ 2016.139069] [] ? sys_pipe+0x10/0x20 [ 2016.139069] [] ? system_call_fastpath+0x16/0x1b [ 2016.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 2016.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2016.203096] CPU 2 [ 2016.203098] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2016.203131] [ 2016.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2016.203131] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2016.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 2016.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 2016.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 2016.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 2016.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 2016.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 2016.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 2016.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 2016.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 2016.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2016.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2016.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 2016.203131] Stack: [ 2016.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 2016.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 2016.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 2016.203131] Call Trace: [ 2016.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2016.203131] [] ? iput+0x30/0x70 [ 2016.203131] [] ? dentry_iput+0xb8/0x140 [ 2016.203131] [] ? d_kill+0x3c/0x70 [ 2016.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2016.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2016.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2016.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2016.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2016.203131] [] ? d_alloc+0x45/0x280 [ 2016.203131] [] ? do_lookup+0x18d/0x210 [ 2016.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2016.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2016.203131] [] ? path_walk+0x6a/0xe0 [ 2016.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2016.203131] [] ? user_path_at+0x59/0xa0 [ 2016.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2016.203131] [] ? vfs_fstatat+0x46/0x80 [ 2016.203131] [] ? vfs_lstat+0x1e/0x20 [ 2016.203131] [] ? sys32_lstat64+0x24/0x40 [ 2016.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2016.203131] [] ? ia32_tracesys+0x48/0x8c [ 2016.203131] [] ? ia32_sysret+0x0/0x5 [ 2016.203131] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2016.203131] Call Trace: [ 2016.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2016.203131] [] ? iput+0x30/0x70 [ 2016.203131] [] ? dentry_iput+0xb8/0x140 [ 2016.203131] [] ? d_kill+0x3c/0x70 [ 2016.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2016.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2016.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2016.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2016.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2016.203131] [] ? d_alloc+0x45/0x280 [ 2016.203131] [] ? do_lookup+0x18d/0x210 [ 2016.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2016.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2016.203131] [] ? path_walk+0x6a/0xe0 [ 2016.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2016.203131] [] ? user_path_at+0x59/0xa0 [ 2016.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2016.203131] [] ? vfs_fstatat+0x46/0x80 [ 2016.203131] [] ? vfs_lstat+0x1e/0x20 [ 2016.203131] [] ? sys32_lstat64+0x24/0x40 [ 2016.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2016.203131] [] ? ia32_tracesys+0x48/0x8c [ 2016.203131] [] ? ia32_sysret+0x0/0x5 [ 2016.265010] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 2016.265014] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2016.265125] CPU 3 [ 2016.265127] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2016.265192] [ 2016.265192] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2016.265192] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2016.265192] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 2016.265192] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 2016.265192] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2016.265192] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 2016.265192] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2016.265192] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 2016.265192] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 2016.265192] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2016.265192] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 2016.265192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2016.265192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2016.265192] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 2016.265192] Stack: [ 2016.265192] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 2016.265192] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 2016.265192] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 2016.265192] Call Trace: [ 2016.265192] [] ? new_inode+0x3b/0xc0 [ 2016.265192] [] ? create_write_pipe+0x35/0x1f0 [ 2016.265192] [] ? do_sigaction+0x1bb/0x200 [ 2016.265192] [] ? do_pipe_flags+0x57/0x130 [ 2016.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 2016.265192] [] ? sys_pipe2+0x24/0x80 [ 2016.265192] [] ? sys_pipe+0x10/0x20 [ 2016.265192] [] ? system_call_fastpath+0x16/0x1b [ 2016.265192] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2016.265192] Call Trace: [ 2016.265192] [] ? new_inode+0x3b/0xc0 [ 2016.265192] [] ? create_write_pipe+0x35/0x1f0 [ 2016.265192] [] ? do_sigaction+0x1bb/0x200 [ 2016.265192] [] ? do_pipe_flags+0x57/0x130 [ 2016.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 2016.265192] [] ? sys_pipe2+0x24/0x80 [ 2016.265192] [] ? sys_pipe+0x10/0x20 [ 2016.265192] [] ? system_call_fastpath+0x16/0x1b [ 2016.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2016.076003] CPU 0 [ 2016.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2016.076003] [ 2016.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2016.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2016.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 2016.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 2016.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2016.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 2016.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2016.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 2016.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 2016.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2016.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 2016.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2016.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2016.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 2016.076003] Stack: [ 2016.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 2016.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 2016.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 2016.076003] Call Trace: [ 2016.076003] [] ? new_inode+0x3b/0xc0 [ 2016.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2016.076003] [] ? do_sigaction+0x1bb/0x200 [ 2016.076003] [] ? do_pipe_flags+0x57/0x130 [ 2016.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2016.076003] [] ? sys_pipe2+0x24/0x80 [ 2016.076003] [] ? sys_pipe+0x10/0x20 [ 2016.076003] [] ? system_call_fastpath+0x16/0x1b [ 2016.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2016.076003] Call Trace: [ 2016.076003] [] ? new_inode+0x3b/0xc0 [ 2016.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2016.076003] [] ? do_sigaction+0x1bb/0x200 [ 2016.076003] [] ? do_pipe_flags+0x57/0x130 [ 2016.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2016.076003] [] ? sys_pipe2+0x24/0x80 [ 2016.076003] [] ? sys_pipe+0x10/0x20 [ 2016.076003] [] ? system_call_fastpath+0x16/0x1b [ 2100.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 2100.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 2100.139005] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 2100.139007] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2100.139069] CPU 1 [ 2100.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2100.139069] [ 2100.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2100.139069] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 2100.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 2100.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 2100.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2100.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 2100.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 2100.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 2100.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 2100.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2100.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 2100.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2100.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2100.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 2100.139069] Stack: [ 2100.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 2100.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 2100.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 2100.139069] Call Trace: [ 2100.139069] [] ? new_inode+0x3b/0xc0 [ 2100.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2100.139069] [] ? do_sigaction+0x1bb/0x200 [ 2100.139069] [] ? do_pipe_flags+0x57/0x130 [ 2100.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2100.139069] [] ? sys_pipe2+0x24/0x80 [ 2100.139069] [] ? sys_pipe+0x10/0x20 [ 2100.139069] [] ? system_call_fastpath+0x16/0x1b [ 2100.139069] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 2100.139069] Call Trace: [ 2100.139069] [] ? new_inode+0x3b/0xc0 [ 2100.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2100.139069] [] ? do_sigaction+0x1bb/0x200 [ 2100.139069] [] ? do_pipe_flags+0x57/0x130 [ 2100.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2100.139069] [] ? sys_pipe2+0x24/0x80 [ 2100.139069] [] ? sys_pipe+0x10/0x20 [ 2100.139069] [] ? system_call_fastpath+0x16/0x1b [ 2100.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 2100.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2100.203096] CPU 2 [ 2100.203098] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2100.203131] [ 2100.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2100.203131] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 2100.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 2100.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 2100.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 2100.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 2100.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 2100.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 2100.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 2100.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 2100.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 2100.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2100.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2100.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 2100.203131] Stack: [ 2100.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 2100.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 2100.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 2100.203131] Call Trace: [ 2100.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2100.203131] [] ? iput+0x30/0x70 [ 2100.203131] [] ? dentry_iput+0xb8/0x140 [ 2100.203131] [] ? d_kill+0x3c/0x70 [ 2100.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2100.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2100.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2100.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2100.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2100.203131] [] ? d_alloc+0x45/0x280 [ 2100.203131] [] ? do_lookup+0x18d/0x210 [ 2100.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2100.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2100.203131] [] ? path_walk+0x6a/0xe0 [ 2100.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2100.203131] [] ? user_path_at+0x59/0xa0 [ 2100.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2100.203131] [] ? vfs_fstatat+0x46/0x80 [ 2100.203131] [] ? vfs_lstat+0x1e/0x20 [ 2100.203131] [] ? sys32_lstat64+0x24/0x40 [ 2100.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2100.203131] [] ? ia32_tracesys+0x48/0x8c [ 2100.203131] [] ? ia32_sysret+0x0/0x5 [ 2100.203131] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 2100.203131] Call Trace: [ 2100.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2100.203131] [] ? iput+0x30/0x70 [ 2100.203131] [] ? dentry_iput+0xb8/0x140 [ 2100.203131] [] ? d_kill+0x3c/0x70 [ 2100.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2100.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2100.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2100.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2100.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2100.203131] [] ? d_alloc+0x45/0x280 [ 2100.203131] [] ? do_lookup+0x18d/0x210 [ 2100.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2100.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2100.203131] [] ? path_walk+0x6a/0xe0 [ 2100.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2100.203131] [] ? user_path_at+0x59/0xa0 [ 2100.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2100.203131] [] ? vfs_fstatat+0x46/0x80 [ 2100.203131] [] ? vfs_lstat+0x1e/0x20 [ 2100.203131] [] ? sys32_lstat64+0x24/0x40 [ 2100.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2100.203131] [] ? ia32_tracesys+0x48/0x8c [ 2100.203131] [] ? ia32_sysret+0x0/0x5 [ 2100.265010] BUG: soft lockup - CPU#3 stuck for 68s! [cron:29387] [ 2100.265014] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2100.265127] CPU 3 [ 2100.265128] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2100.265193] [ 2100.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2100.265193] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2100.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 2100.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 2100.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2100.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 2100.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2100.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 2100.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 2100.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2100.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 2100.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2100.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2100.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 2100.265193] Stack: [ 2100.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 2100.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 2100.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 2100.265193] Call Trace: [ 2100.265193] [] ? new_inode+0x3b/0xc0 [ 2100.265193] [] ? create_write_pipe+0x35/0x1f0 [ 2100.265193] [] ? do_sigaction+0x1bb/0x200 [ 2100.265193] [] ? do_pipe_flags+0x57/0x130 [ 2100.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 2100.265193] [] ? sys_pipe2+0x24/0x80 [ 2100.265193] [] ? sys_pipe+0x10/0x20 [ 2100.265193] [] ? system_call_fastpath+0x16/0x1b [ 2100.265193] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2100.265193] Call Trace: [ 2100.265193] [] ? new_inode+0x3b/0xc0 [ 2100.265193] [] ? create_write_pipe+0x35/0x1f0 [ 2100.265193] [] ? do_sigaction+0x1bb/0x200 [ 2100.265193] [] ? do_pipe_flags+0x57/0x130 [ 2100.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 2100.265193] [] ? sys_pipe2+0x24/0x80 [ 2100.265193] [] ? sys_pipe+0x10/0x20 [ 2100.265193] [] ? system_call_fastpath+0x16/0x1b [ 2100.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2100.076003] CPU 0 [ 2100.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2100.076003] [ 2100.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2100.076003] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 2100.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 2100.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 2100.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2100.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 2100.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2100.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 2100.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 2100.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2100.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 2100.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2100.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2100.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 2100.076003] Stack: [ 2100.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 2100.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 2100.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 2100.076003] Call Trace: [ 2100.076003] [] ? new_inode+0x3b/0xc0 [ 2100.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2100.076003] [] ? do_sigaction+0x1bb/0x200 [ 2100.076003] [] ? do_pipe_flags+0x57/0x130 [ 2100.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2100.076003] [] ? sys_pipe2+0x24/0x80 [ 2100.076003] [] ? sys_pipe+0x10/0x20 [ 2100.076003] [] ? system_call_fastpath+0x16/0x1b [ 2100.076003] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 2100.076003] Call Trace: [ 2100.076003] [] ? new_inode+0x3b/0xc0 [ 2100.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2100.076003] [] ? do_sigaction+0x1bb/0x200 [ 2100.076003] [] ? do_pipe_flags+0x57/0x130 [ 2100.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2100.076003] [] ? sys_pipe2+0x24/0x80 [ 2100.076003] [] ? sys_pipe+0x10/0x20 [ 2100.076003] [] ? system_call_fastpath+0x16/0x1b [ 2184.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 2184.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 2184.139006] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 2184.139009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2184.139069] CPU 1 [ 2184.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2184.139069] [ 2184.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2184.139069] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2184.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 2184.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 2184.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2184.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 2184.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 2184.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 2184.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 2184.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2184.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 2184.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2184.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2184.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 2184.139069] Stack: [ 2184.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 2184.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 2184.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 2184.139069] Call Trace: [ 2184.139069] [] ? new_inode+0x3b/0xc0 [ 2184.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2184.139069] [] ? do_sigaction+0x1bb/0x200 [ 2184.139069] [] ? do_pipe_flags+0x57/0x130 [ 2184.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2184.139069] [] ? sys_pipe2+0x24/0x80 [ 2184.139069] [] ? sys_pipe+0x10/0x20 [ 2184.139069] [] ? system_call_fastpath+0x16/0x1b [ 2184.139069] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2184.139069] Call Trace: [ 2184.139069] [] ? new_inode+0x3b/0xc0 [ 2184.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2184.139069] [] ? do_sigaction+0x1bb/0x200 [ 2184.139069] [] ? do_pipe_flags+0x57/0x130 [ 2184.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2184.139069] [] ? sys_pipe2+0x24/0x80 [ 2184.139069] [] ? sys_pipe+0x10/0x20 [ 2184.139069] [] ? system_call_fastpath+0x16/0x1b [ 2184.203006] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 2184.203008] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2184.203095] CPU 2 [ 2184.203096] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2184.203129] [ 2184.203129] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2184.203129] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2184.203129] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 2184.203129] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 2184.203129] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 2184.203129] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 2184.203129] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 2184.203129] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 2184.203129] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 2184.203129] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 2184.203129] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 2184.203129] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2184.203129] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2184.203129] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 2184.203129] Stack: [ 2184.203129] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 2184.203129] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 2184.203129] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 2184.203129] Call Trace: [ 2184.203129] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2184.203129] [] ? iput+0x30/0x70 [ 2184.203129] [] ? dentry_iput+0xb8/0x140 [ 2184.203129] [] ? d_kill+0x3c/0x70 [ 2184.203129] [] ? prune_one_dentry+0xb1/0xd0 [ 2184.203129] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2184.203129] [] ? shrink_dcache_ub+0x29/0x70 [ 2184.203129] [] ? ub_dcache_shrink+0x3e/0x50 [ 2184.203129] [] ? ub_dcache_charge+0x96/0x120 [ 2184.203129] [] ? d_alloc+0x45/0x280 [ 2184.203129] [] ? do_lookup+0x18d/0x210 [ 2184.203129] [] ? __link_path_walk+0x89e/0x10c0 [ 2184.203129] [] ? vma_prio_tree_insert+0x30/0x50 [ 2184.203129] [] ? path_walk+0x6a/0xe0 [ 2184.203129] [] ? do_path_lookup+0x5b/0xa0 [ 2184.203129] [] ? user_path_at+0x59/0xa0 [ 2184.203129] [] ? __do_page_fault+0x264/0x4f0 [ 2184.203129] [] ? vfs_fstatat+0x46/0x80 [ 2184.203129] [] ? vfs_lstat+0x1e/0x20 [ 2184.203129] [] ? sys32_lstat64+0x24/0x40 [ 2184.203129] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2184.203129] [] ? ia32_tracesys+0x48/0x8c [ 2184.203129] [] ? ia32_sysret+0x0/0x5 [ 2184.203129] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2184.203129] Call Trace: [ 2184.203129] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2184.203129] [] ? iput+0x30/0x70 [ 2184.203129] [] ? dentry_iput+0xb8/0x140 [ 2184.203129] [] ? d_kill+0x3c/0x70 [ 2184.203129] [] ? prune_one_dentry+0xb1/0xd0 [ 2184.203129] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2184.203129] [] ? shrink_dcache_ub+0x29/0x70 [ 2184.203129] [] ? ub_dcache_shrink+0x3e/0x50 [ 2184.203129] [] ? ub_dcache_charge+0x96/0x120 [ 2184.203129] [] ? d_alloc+0x45/0x280 [ 2184.203129] [] ? do_lookup+0x18d/0x210 [ 2184.203129] [] ? __link_path_walk+0x89e/0x10c0 [ 2184.203129] [] ? vma_prio_tree_insert+0x30/0x50 [ 2184.203129] [] ? path_walk+0x6a/0xe0 [ 2184.203129] [] ? do_path_lookup+0x5b/0xa0 [ 2184.203129] [] ? user_path_at+0x59/0xa0 [ 2184.203129] [] ? __do_page_fault+0x264/0x4f0 [ 2184.203129] [] ? vfs_fstatat+0x46/0x80 [ 2184.203129] [] ? vfs_lstat+0x1e/0x20 [ 2184.203129] [] ? sys32_lstat64+0x24/0x40 [ 2184.203129] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2184.203129] [] ? ia32_tracesys+0x48/0x8c [ 2184.203129] [] ? ia32_sysret+0x0/0x5 [ 2184.265012] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 2184.265016] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2184.265128] CPU 3 [ 2184.265129] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2184.265193] [ 2184.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2184.265193] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 2184.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 2184.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 2184.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2184.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 2184.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2184.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 2184.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 2184.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2184.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 2184.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2184.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2184.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 2184.265193] Stack: [ 2184.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 2184.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 2184.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 2184.265193] Call Trace: [ 2184.265193] [] ? new_inode+0x3b/0xc0 [ 2184.265193] [] ? create_write_pipe+0x35/0x1f0 [ 2184.265193] [] ? do_sigaction+0x1bb/0x200 [ 2184.265193] [] ? do_pipe_flags+0x57/0x130 [ 2184.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 2184.265193] [] ? sys_pipe2+0x24/0x80 [ 2184.265193] [] ? sys_pipe+0x10/0x20 [ 2184.265193] [] ? system_call_fastpath+0x16/0x1b [ 2184.265193] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 2184.265193] Call Trace: [ 2184.265193] [] ? new_inode+0x3b/0xc0 [ 2184.265193] [] ? create_write_pipe+0x35/0x1f0 [ 2184.265193] [] ? do_sigaction+0x1bb/0x200 [ 2184.265193] [] ? do_pipe_flags+0x57/0x130 [ 2184.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 2184.265193] [] ? sys_pipe2+0x24/0x80 [ 2184.265193] [] ? sys_pipe+0x10/0x20 [ 2184.265193] [] ? system_call_fastpath+0x16/0x1b [ 2184.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2184.076003] CPU 0 [ 2184.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2184.076003] [ 2184.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2184.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2184.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 2184.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 2184.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2184.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 2184.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2184.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 2184.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 2184.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2184.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 2184.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2184.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2184.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 2184.076003] Stack: [ 2184.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 2184.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 2184.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 2184.076003] Call Trace: [ 2184.076003] [] ? new_inode+0x3b/0xc0 [ 2184.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2184.076003] [] ? do_sigaction+0x1bb/0x200 [ 2184.076003] [] ? do_pipe_flags+0x57/0x130 [ 2184.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2184.076003] [] ? sys_pipe2+0x24/0x80 [ 2184.076003] [] ? sys_pipe+0x10/0x20 [ 2184.076003] [] ? system_call_fastpath+0x16/0x1b [ 2184.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2184.076003] Call Trace: [ 2184.076003] [] ? new_inode+0x3b/0xc0 [ 2184.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2184.076003] [] ? do_sigaction+0x1bb/0x200 [ 2184.076003] [] ? do_pipe_flags+0x57/0x130 [ 2184.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2184.076003] [] ? sys_pipe2+0x24/0x80 [ 2184.076003] [] ? sys_pipe+0x10/0x20 [ 2184.076003] [] ? system_call_fastpath+0x16/0x1b [ 2268.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 2268.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 2268.139006] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 2268.139009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2268.139069] CPU 1 [ 2268.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2268.139069] [ 2268.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2268.139069] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 2268.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 2268.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 2268.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2268.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 2268.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 2268.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 2268.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 2268.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2268.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 2268.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2268.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2268.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 2268.139069] Stack: [ 2268.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 2268.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 2268.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 2268.139069] Call Trace: [ 2268.139069] [] ? new_inode+0x3b/0xc0 [ 2268.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2268.139069] [] ? do_sigaction+0x1bb/0x200 [ 2268.139069] [] ? do_pipe_flags+0x57/0x130 [ 2268.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2268.139069] [] ? sys_pipe2+0x24/0x80 [ 2268.139069] [] ? sys_pipe+0x10/0x20 [ 2268.139069] [] ? system_call_fastpath+0x16/0x1b [ 2268.139069] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 2268.139069] Call Trace: [ 2268.139069] [] ? new_inode+0x3b/0xc0 [ 2268.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2268.139069] [] ? do_sigaction+0x1bb/0x200 [ 2268.139069] [] ? do_pipe_flags+0x57/0x130 [ 2268.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2268.139069] [] ? sys_pipe2+0x24/0x80 [ 2268.139069] [] ? sys_pipe+0x10/0x20 [ 2268.139069] [] ? system_call_fastpath+0x16/0x1b [ 2268.203006] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 2268.203008] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2268.203095] CPU 2 [ 2268.203096] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2268.203131] [ 2268.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2268.203131] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 2268.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 2268.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 2268.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 2268.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 2268.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 2268.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 2268.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 2268.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 2268.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 2268.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2268.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2268.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 2268.203131] Stack: [ 2268.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 2268.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 2268.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 2268.203131] Call Trace: [ 2268.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2268.203131] [] ? iput+0x30/0x70 [ 2268.203131] [] ? dentry_iput+0xb8/0x140 [ 2268.203131] [] ? d_kill+0x3c/0x70 [ 2268.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2268.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2268.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2268.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2268.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2268.203131] [] ? d_alloc+0x45/0x280 [ 2268.203131] [] ? do_lookup+0x18d/0x210 [ 2268.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2268.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2268.203131] [] ? path_walk+0x6a/0xe0 [ 2268.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2268.203131] [] ? user_path_at+0x59/0xa0 [ 2268.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2268.203131] [] ? vfs_fstatat+0x46/0x80 [ 2268.203131] [] ? vfs_lstat+0x1e/0x20 [ 2268.203131] [] ? sys32_lstat64+0x24/0x40 [ 2268.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2268.203131] [] ? ia32_tracesys+0x48/0x8c [ 2268.203131] [] ? ia32_sysret+0x0/0x5 [ 2268.203131] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 2268.203131] Call Trace: [ 2268.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2268.203131] [] ? iput+0x30/0x70 [ 2268.203131] [] ? dentry_iput+0xb8/0x140 [ 2268.203131] [] ? d_kill+0x3c/0x70 [ 2268.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2268.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2268.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2268.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2268.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2268.203131] [] ? d_alloc+0x45/0x280 [ 2268.203131] [] ? do_lookup+0x18d/0x210 [ 2268.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2268.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2268.203131] [] ? path_walk+0x6a/0xe0 [ 2268.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2268.203131] [] ? user_path_at+0x59/0xa0 [ 2268.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2268.203131] [] ? vfs_fstatat+0x46/0x80 [ 2268.203131] [] ? vfs_lstat+0x1e/0x20 [ 2268.203131] [] ? sys32_lstat64+0x24/0x40 [ 2268.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2268.203131] [] ? ia32_tracesys+0x48/0x8c [ 2268.203131] [] ? ia32_sysret+0x0/0x5 [ 2268.265009] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 2268.265013] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2268.265126] CPU 3 [ 2268.265128] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2268.265193] [ 2268.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2268.265193] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2268.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 2268.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 2268.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2268.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 2268.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2268.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 2268.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 2268.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2268.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 2268.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2268.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2268.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 2268.265193] Stack: [ 2268.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 2268.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 2268.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 2268.265193] Call Trace: [ 2268.265193] [] ? new_inode+0x3b/0xc0 [ 2268.265193] [] ? create_write_pipe+0x35/0x1f0 [ 2268.265193] [] ? do_sigaction+0x1bb/0x200 [ 2268.265193] [] ? do_pipe_flags+0x57/0x130 [ 2268.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 2268.265193] [] ? sys_pipe2+0x24/0x80 [ 2268.265193] [] ? sys_pipe+0x10/0x20 [ 2268.265193] [] ? system_call_fastpath+0x16/0x1b [ 2268.265193] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2268.265193] Call Trace: [ 2268.265193] [] ? new_inode+0x3b/0xc0 [ 2268.265193] [] ? create_write_pipe+0x35/0x1f0 [ 2268.265193] [] ? do_sigaction+0x1bb/0x200 [ 2268.265193] [] ? do_pipe_flags+0x57/0x130 [ 2268.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 2268.265193] [] ? sys_pipe2+0x24/0x80 [ 2268.265193] [] ? sys_pipe+0x10/0x20 [ 2268.265193] [] ? system_call_fastpath+0x16/0x1b [ 2268.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2268.076003] CPU 0 [ 2268.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2268.076003] [ 2268.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2268.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2268.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 2268.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 2268.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2268.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 2268.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2268.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 2268.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 2268.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2268.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 2268.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2268.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2268.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 2268.076003] Stack: [ 2268.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 2268.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 2268.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 2268.076003] Call Trace: [ 2268.076003] [] ? new_inode+0x3b/0xc0 [ 2268.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2268.076003] [] ? do_sigaction+0x1bb/0x200 [ 2268.076003] [] ? do_pipe_flags+0x57/0x130 [ 2268.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2268.076003] [] ? sys_pipe2+0x24/0x80 [ 2268.076003] [] ? sys_pipe+0x10/0x20 [ 2268.076003] [] ? system_call_fastpath+0x16/0x1b [ 2268.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2268.076003] Call Trace: [ 2268.076003] [] ? new_inode+0x3b/0xc0 [ 2268.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2268.076003] [] ? do_sigaction+0x1bb/0x200 [ 2268.076003] [] ? do_pipe_flags+0x57/0x130 [ 2268.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2268.076003] [] ? sys_pipe2+0x24/0x80 [ 2268.076003] [] ? sys_pipe+0x10/0x20 [ 2268.076003] [] ? system_call_fastpath+0x16/0x1b [ 2352.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 2352.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 2352.139006] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 2352.139009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2352.139069] CPU 1 [ 2352.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2352.139069] [ 2352.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2352.139069] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 2352.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 2352.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 2352.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2352.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 2352.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 2352.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 2352.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 2352.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2352.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 2352.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2352.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2352.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 2352.139069] Stack: [ 2352.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 2352.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 2352.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 2352.139069] Call Trace: [ 2352.139069] [] ? new_inode+0x3b/0xc0 [ 2352.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2352.139069] [] ? do_sigaction+0x1bb/0x200 [ 2352.139069] [] ? do_pipe_flags+0x57/0x130 [ 2352.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2352.139069] [] ? sys_pipe2+0x24/0x80 [ 2352.139069] [] ? sys_pipe+0x10/0x20 [ 2352.139069] [] ? system_call_fastpath+0x16/0x1b [ 2352.139069] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 2352.139069] Call Trace: [ 2352.139069] [] ? new_inode+0x3b/0xc0 [ 2352.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2352.139069] [] ? do_sigaction+0x1bb/0x200 [ 2352.139069] [] ? do_pipe_flags+0x57/0x130 [ 2352.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2352.139069] [] ? sys_pipe2+0x24/0x80 [ 2352.139069] [] ? sys_pipe+0x10/0x20 [ 2352.139069] [] ? system_call_fastpath+0x16/0x1b [ 2352.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 2352.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2352.203096] CPU 2 [ 2352.203097] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2352.203131] [ 2352.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2352.203131] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 2352.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 2352.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 2352.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 2352.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 2352.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 2352.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 2352.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 2352.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 2352.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 2352.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2352.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2352.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 2352.203131] Stack: [ 2352.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 2352.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 2352.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 2352.203131] Call Trace: [ 2352.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2352.203131] [] ? iput+0x30/0x70 [ 2352.203131] [] ? dentry_iput+0xb8/0x140 [ 2352.203131] [] ? d_kill+0x3c/0x70 [ 2352.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2352.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2352.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2352.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2352.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2352.203131] [] ? d_alloc+0x45/0x280 [ 2352.203131] [] ? do_lookup+0x18d/0x210 [ 2352.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2352.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2352.203131] [] ? path_walk+0x6a/0xe0 [ 2352.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2352.203131] [] ? user_path_at+0x59/0xa0 [ 2352.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2352.203131] [] ? vfs_fstatat+0x46/0x80 [ 2352.203131] [] ? vfs_lstat+0x1e/0x20 [ 2352.203131] [] ? sys32_lstat64+0x24/0x40 [ 2352.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2352.203131] [] ? ia32_tracesys+0x48/0x8c [ 2352.203131] [] ? ia32_sysret+0x0/0x5 [ 2352.203131] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 2352.203131] Call Trace: [ 2352.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2352.203131] [] ? iput+0x30/0x70 [ 2352.203131] [] ? dentry_iput+0xb8/0x140 [ 2352.203131] [] ? d_kill+0x3c/0x70 [ 2352.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2352.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2352.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2352.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2352.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2352.203131] [] ? d_alloc+0x45/0x280 [ 2352.203131] [] ? do_lookup+0x18d/0x210 [ 2352.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2352.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2352.203131] [] ? path_walk+0x6a/0xe0 [ 2352.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2352.203131] [] ? user_path_at+0x59/0xa0 [ 2352.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2352.203131] [] ? vfs_fstatat+0x46/0x80 [ 2352.203131] [] ? vfs_lstat+0x1e/0x20 [ 2352.203131] [] ? sys32_lstat64+0x24/0x40 [ 2352.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2352.203131] [] ? ia32_tracesys+0x48/0x8c [ 2352.203131] [] ? ia32_sysret+0x0/0x5 [ 2352.265012] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 2352.265016] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2352.265128] CPU 3 [ 2352.265130] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2352.265193] [ 2352.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2352.265193] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2352.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 2352.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 2352.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2352.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 2352.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2352.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 2352.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 2352.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2352.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 2352.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2352.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2352.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 2352.265193] Stack: [ 2352.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 2352.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 2352.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 2352.265193] Call Trace: [ 2352.265193] [] ? new_inode+0x3b/0xc0 [ 2352.265193] [] ? create_write_pipe+0x35/0x1f0 [ 2352.265193] [] ? do_sigaction+0x1bb/0x200 [ 2352.265193] [] ? do_pipe_flags+0x57/0x130 [ 2352.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 2352.265193] [] ? sys_pipe2+0x24/0x80 [ 2352.265193] [] ? sys_pipe+0x10/0x20 [ 2352.265193] [] ? system_call_fastpath+0x16/0x1b [ 2352.265193] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2352.265193] Call Trace: [ 2352.265193] [] ? new_inode+0x3b/0xc0 [ 2352.265193] [] ? create_write_pipe+0x35/0x1f0 [ 2352.265193] [] ? do_sigaction+0x1bb/0x200 [ 2352.265193] [] ? do_pipe_flags+0x57/0x130 [ 2352.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 2352.265193] [] ? sys_pipe2+0x24/0x80 [ 2352.265193] [] ? sys_pipe+0x10/0x20 [ 2352.265193] [] ? system_call_fastpath+0x16/0x1b [ 2352.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2352.076003] CPU 0 [ 2352.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2352.076003] [ 2352.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2352.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2352.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 2352.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 2352.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2352.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 2352.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2352.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 2352.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 2352.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2352.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 2352.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2352.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2352.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 2352.076003] Stack: [ 2352.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 2352.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 2352.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 2352.076003] Call Trace: [ 2352.076003] [] ? new_inode+0x3b/0xc0 [ 2352.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2352.076003] [] ? do_sigaction+0x1bb/0x200 [ 2352.076003] [] ? do_pipe_flags+0x57/0x130 [ 2352.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2352.076003] [] ? sys_pipe2+0x24/0x80 [ 2352.076003] [] ? sys_pipe+0x10/0x20 [ 2352.076003] [] ? system_call_fastpath+0x16/0x1b [ 2352.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2352.076003] Call Trace: [ 2352.076003] [] ? new_inode+0x3b/0xc0 [ 2352.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2352.076003] [] ? do_sigaction+0x1bb/0x200 [ 2352.076003] [] ? do_pipe_flags+0x57/0x130 [ 2352.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2352.076003] [] ? sys_pipe2+0x24/0x80 [ 2352.076003] [] ? sys_pipe+0x10/0x20 [ 2352.076003] [] ? system_call_fastpath+0x16/0x1b [ 2436.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 2436.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 2436.139006] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 2436.139009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2436.139069] CPU 1 [ 2436.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2436.139069] [ 2436.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2436.139069] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 2436.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 2436.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 2436.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2436.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 2436.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 2436.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 2436.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 2436.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2436.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 2436.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2436.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2436.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 2436.139069] Stack: [ 2436.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 2436.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 2436.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 2436.139069] Call Trace: [ 2436.139069] [] ? new_inode+0x3b/0xc0 [ 2436.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2436.139069] [] ? do_sigaction+0x1bb/0x200 [ 2436.139069] [] ? do_pipe_flags+0x57/0x130 [ 2436.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2436.139069] [] ? sys_pipe2+0x24/0x80 [ 2436.139069] [] ? sys_pipe+0x10/0x20 [ 2436.139069] [] ? system_call_fastpath+0x16/0x1b [ 2436.139069] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 2436.139069] Call Trace: [ 2436.139069] [] ? new_inode+0x3b/0xc0 [ 2436.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2436.139069] [] ? do_sigaction+0x1bb/0x200 [ 2436.139069] [] ? do_pipe_flags+0x57/0x130 [ 2436.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2436.139069] [] ? sys_pipe2+0x24/0x80 [ 2436.139069] [] ? sys_pipe+0x10/0x20 [ 2436.139069] [] ? system_call_fastpath+0x16/0x1b [ 2436.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 2436.203010] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2436.203097] CPU 2 [ 2436.203098] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2436.203131] [ 2436.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2436.203131] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 2436.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 2436.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 2436.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 2436.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 2436.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 2436.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 2436.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 2436.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 2436.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 2436.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2436.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2436.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 2436.203131] Stack: [ 2436.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 2436.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 2436.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 2436.203131] Call Trace: [ 2436.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2436.203131] [] ? iput+0x30/0x70 [ 2436.203131] [] ? dentry_iput+0xb8/0x140 [ 2436.203131] [] ? d_kill+0x3c/0x70 [ 2436.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2436.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2436.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2436.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2436.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2436.203131] [] ? d_alloc+0x45/0x280 [ 2436.203131] [] ? do_lookup+0x18d/0x210 [ 2436.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2436.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2436.203131] [] ? path_walk+0x6a/0xe0 [ 2436.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2436.203131] [] ? user_path_at+0x59/0xa0 [ 2436.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2436.203131] [] ? vfs_fstatat+0x46/0x80 [ 2436.203131] [] ? vfs_lstat+0x1e/0x20 [ 2436.203131] [] ? sys32_lstat64+0x24/0x40 [ 2436.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2436.203131] [] ? ia32_tracesys+0x48/0x8c [ 2436.203131] [] ? ia32_sysret+0x0/0x5 [ 2436.203131] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 2436.203131] Call Trace: [ 2436.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2436.203131] [] ? iput+0x30/0x70 [ 2436.203131] [] ? dentry_iput+0xb8/0x140 [ 2436.203131] [] ? d_kill+0x3c/0x70 [ 2436.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2436.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2436.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2436.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2436.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2436.203131] [] ? d_alloc+0x45/0x280 [ 2436.203131] [] ? do_lookup+0x18d/0x210 [ 2436.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2436.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2436.203131] [] ? path_walk+0x6a/0xe0 [ 2436.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2436.203131] [] ? user_path_at+0x59/0xa0 [ 2436.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2436.203131] [] ? vfs_fstatat+0x46/0x80 [ 2436.203131] [] ? vfs_lstat+0x1e/0x20 [ 2436.203131] [] ? sys32_lstat64+0x24/0x40 [ 2436.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2436.203131] [] ? ia32_tracesys+0x48/0x8c [ 2436.203131] [] ? ia32_sysret+0x0/0x5 [ 2436.265009] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 2436.265013] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2436.265124] CPU 3 [ 2436.265125] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2436.265192] [ 2436.265192] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2436.265192] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2436.265192] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 2436.265192] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 2436.265192] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2436.265192] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 2436.265192] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2436.265192] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 2436.265192] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 2436.265192] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2436.265192] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 2436.265192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2436.265192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2436.265192] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 2436.265192] Stack: [ 2436.265192] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 2436.265192] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 2436.265192] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 2436.265192] Call Trace: [ 2436.265192] [] ? new_inode+0x3b/0xc0 [ 2436.265192] [] ? create_write_pipe+0x35/0x1f0 [ 2436.265192] [] ? do_sigaction+0x1bb/0x200 [ 2436.265192] [] ? do_pipe_flags+0x57/0x130 [ 2436.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 2436.265192] [] ? sys_pipe2+0x24/0x80 [ 2436.265192] [] ? sys_pipe+0x10/0x20 [ 2436.265192] [] ? system_call_fastpath+0x16/0x1b [ 2436.265192] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2436.265192] Call Trace: [ 2436.265192] [] ? new_inode+0x3b/0xc0 [ 2436.265192] [] ? create_write_pipe+0x35/0x1f0 [ 2436.265192] [] ? do_sigaction+0x1bb/0x200 [ 2436.265192] [] ? do_pipe_flags+0x57/0x130 [ 2436.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 2436.265192] [] ? sys_pipe2+0x24/0x80 [ 2436.265192] [] ? sys_pipe+0x10/0x20 [ 2436.265192] [] ? system_call_fastpath+0x16/0x1b [ 2436.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2436.076003] CPU 0 [ 2436.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2436.076003] [ 2436.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2436.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2436.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 2436.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 2436.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2436.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 2436.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2436.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 2436.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 2436.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2436.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 2436.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2436.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2436.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 2436.076003] Stack: [ 2436.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 2436.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 2436.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 2436.076003] Call Trace: [ 2436.076003] [] ? new_inode+0x3b/0xc0 [ 2436.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2436.076003] [] ? do_sigaction+0x1bb/0x200 [ 2436.076003] [] ? do_pipe_flags+0x57/0x130 [ 2436.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2436.076003] [] ? sys_pipe2+0x24/0x80 [ 2436.076003] [] ? sys_pipe+0x10/0x20 [ 2436.076003] [] ? system_call_fastpath+0x16/0x1b [ 2436.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2436.076003] Call Trace: [ 2436.076003] [] ? new_inode+0x3b/0xc0 [ 2436.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2436.076003] [] ? do_sigaction+0x1bb/0x200 [ 2436.076003] [] ? do_pipe_flags+0x57/0x130 [ 2436.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2436.076003] [] ? sys_pipe2+0x24/0x80 [ 2436.076003] [] ? sys_pipe+0x10/0x20 [ 2436.076003] [] ? system_call_fastpath+0x16/0x1b [ 2520.076003] BUG: soft lockup - CPU#0 stuck for 68s! [cron:29389] [ 2520.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 2520.139005] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 2520.139008] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2520.139070] CPU 1 [ 2520.139070] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2520.139070] [ 2520.139070] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2520.139070] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2520.139070] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 2520.139070] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 2520.139070] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2520.139070] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 2520.139070] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 2520.139070] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 2520.139070] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 2520.139070] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2520.139070] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 2520.139070] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2520.139070] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2520.139070] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 2520.139070] Stack: [ 2520.139070] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 2520.139070] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 2520.139070] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 2520.139070] Call Trace: [ 2520.139070] [] ? new_inode+0x3b/0xc0 [ 2520.139070] [] ? create_write_pipe+0x35/0x1f0 [ 2520.139070] [] ? do_sigaction+0x1bb/0x200 [ 2520.139070] [] ? do_pipe_flags+0x57/0x130 [ 2520.139070] [] ? audit_syscall_entry+0x286/0x2b0 [ 2520.139070] [] ? sys_pipe2+0x24/0x80 [ 2520.139070] [] ? sys_pipe+0x10/0x20 [ 2520.139070] [] ? system_call_fastpath+0x16/0x1b [ 2520.139070] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2520.139070] Call Trace: [ 2520.139070] [] ? new_inode+0x3b/0xc0 [ 2520.139070] [] ? create_write_pipe+0x35/0x1f0 [ 2520.139070] [] ? do_sigaction+0x1bb/0x200 [ 2520.139070] [] ? do_pipe_flags+0x57/0x130 [ 2520.139070] [] ? audit_syscall_entry+0x286/0x2b0 [ 2520.139070] [] ? sys_pipe2+0x24/0x80 [ 2520.139070] [] ? sys_pipe+0x10/0x20 [ 2520.139070] [] ? system_call_fastpath+0x16/0x1b [ 2520.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 2520.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2520.203096] CPU 2 [ 2520.203097] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2520.203131] [ 2520.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2520.203131] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 2520.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 2520.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 2520.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 2520.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 2520.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 2520.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 2520.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 2520.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 2520.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 2520.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2520.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2520.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 2520.203131] Stack: [ 2520.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 2520.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 2520.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 2520.203131] Call Trace: [ 2520.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2520.203131] [] ? iput+0x30/0x70 [ 2520.203131] [] ? dentry_iput+0xb8/0x140 [ 2520.203131] [] ? d_kill+0x3c/0x70 [ 2520.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2520.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2520.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2520.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2520.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2520.203131] [] ? d_alloc+0x45/0x280 [ 2520.203131] [] ? do_lookup+0x18d/0x210 [ 2520.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2520.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2520.203131] [] ? path_walk+0x6a/0xe0 [ 2520.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2520.203131] [] ? user_path_at+0x59/0xa0 [ 2520.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2520.203131] [] ? vfs_fstatat+0x46/0x80 [ 2520.203131] [] ? vfs_lstat+0x1e/0x20 [ 2520.203131] [] ? sys32_lstat64+0x24/0x40 [ 2520.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2520.203131] [] ? ia32_tracesys+0x48/0x8c [ 2520.203131] [] ? ia32_sysret+0x0/0x5 [ 2520.203131] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 2520.203131] Call Trace: [ 2520.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2520.203131] [] ? iput+0x30/0x70 [ 2520.203131] [] ? dentry_iput+0xb8/0x140 [ 2520.203131] [] ? d_kill+0x3c/0x70 [ 2520.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2520.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2520.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2520.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2520.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2520.203131] [] ? d_alloc+0x45/0x280 [ 2520.203131] [] ? do_lookup+0x18d/0x210 [ 2520.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2520.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2520.203131] [] ? path_walk+0x6a/0xe0 [ 2520.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2520.203131] [] ? user_path_at+0x59/0xa0 [ 2520.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2520.203131] [] ? vfs_fstatat+0x46/0x80 [ 2520.203131] [] ? vfs_lstat+0x1e/0x20 [ 2520.203131] [] ? sys32_lstat64+0x24/0x40 [ 2520.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2520.203131] [] ? ia32_tracesys+0x48/0x8c [ 2520.203131] [] ? ia32_sysret+0x0/0x5 [ 2520.265011] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 2520.265015] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2520.265126] CPU 3 [ 2520.265128] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2520.265193] [ 2520.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2520.265193] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2520.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 2520.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 2520.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2520.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 2520.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2520.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 2520.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 2520.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2520.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 2520.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2520.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2520.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 2520.265193] Stack: [ 2520.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 2520.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 2520.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 2520.265193] Call Trace: [ 2520.265193] [] ? new_inode+0x3b/0xc0 [ 2520.265193] [] ? create_write_pipe+0x35/0x1f0 [ 2520.265193] [] ? do_sigaction+0x1bb/0x200 [ 2520.265193] [] ? do_pipe_flags+0x57/0x130 [ 2520.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 2520.265193] [] ? sys_pipe2+0x24/0x80 [ 2520.265193] [] ? sys_pipe+0x10/0x20 [ 2520.265193] [] ? system_call_fastpath+0x16/0x1b [ 2520.265193] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2520.265193] Call Trace: [ 2520.265193] [] ? new_inode+0x3b/0xc0 [ 2520.265193] [] ? create_write_pipe+0x35/0x1f0 [ 2520.265193] [] ? do_sigaction+0x1bb/0x200 [ 2520.265193] [] ? do_pipe_flags+0x57/0x130 [ 2520.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 2520.265193] [] ? sys_pipe2+0x24/0x80 [ 2520.265193] [] ? sys_pipe+0x10/0x20 [ 2520.265193] [] ? system_call_fastpath+0x16/0x1b [ 2520.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2520.076003] CPU 0 [ 2520.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2520.076003] [ 2520.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2520.076003] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 2520.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 2520.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 2520.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2520.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 2520.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2520.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 2520.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 2520.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2520.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 2520.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2520.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2520.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 2520.076003] Stack: [ 2520.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 2520.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 2520.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 2520.076003] Call Trace: [ 2520.076003] [] ? new_inode+0x3b/0xc0 [ 2520.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2520.076003] [] ? do_sigaction+0x1bb/0x200 [ 2520.076003] [] ? do_pipe_flags+0x57/0x130 [ 2520.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2520.076003] [] ? sys_pipe2+0x24/0x80 [ 2520.076003] [] ? sys_pipe+0x10/0x20 [ 2520.076003] [] ? system_call_fastpath+0x16/0x1b [ 2520.076003] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 2520.076003] Call Trace: [ 2520.076003] [] ? new_inode+0x3b/0xc0 [ 2520.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2520.076003] [] ? do_sigaction+0x1bb/0x200 [ 2520.076003] [] ? do_pipe_flags+0x57/0x130 [ 2520.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2520.076003] [] ? sys_pipe2+0x24/0x80 [ 2520.076003] [] ? sys_pipe+0x10/0x20 [ 2520.076003] [] ? system_call_fastpath+0x16/0x1b [ 2604.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 2604.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 2604.139006] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 2604.139009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2604.139069] CPU 1 [ 2604.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2604.139069] [ 2604.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2604.139069] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2604.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 2604.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 2604.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2604.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 2604.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 2604.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 2604.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 2604.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2604.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 2604.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2604.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2604.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 2604.139069] Stack: [ 2604.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 2604.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 2604.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 2604.139069] Call Trace: [ 2604.139069] [] ? new_inode+0x3b/0xc0 [ 2604.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2604.139069] [] ? do_sigaction+0x1bb/0x200 [ 2604.139069] [] ? do_pipe_flags+0x57/0x130 [ 2604.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2604.139069] [] ? sys_pipe2+0x24/0x80 [ 2604.139069] [] ? sys_pipe+0x10/0x20 [ 2604.139069] [] ? system_call_fastpath+0x16/0x1b [ 2604.139069] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2604.139069] Call Trace: [ 2604.139069] [] ? new_inode+0x3b/0xc0 [ 2604.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2604.139069] [] ? do_sigaction+0x1bb/0x200 [ 2604.139069] [] ? do_pipe_flags+0x57/0x130 [ 2604.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2604.139069] [] ? sys_pipe2+0x24/0x80 [ 2604.139069] [] ? sys_pipe+0x10/0x20 [ 2604.139069] [] ? system_call_fastpath+0x16/0x1b [ 2604.203006] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 2604.203008] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2604.203095] CPU 2 [ 2604.203096] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2604.203129] [ 2604.203129] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2604.203129] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2604.203129] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 2604.203129] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 2604.203129] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 2604.203129] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 2604.203129] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 2604.203129] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 2604.203129] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 2604.203129] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 2604.203129] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 2604.203129] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2604.203129] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2604.203129] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 2604.203129] Stack: [ 2604.203129] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 2604.203129] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 2604.203129] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 2604.203129] Call Trace: [ 2604.203129] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2604.203129] [] ? iput+0x30/0x70 [ 2604.203129] [] ? dentry_iput+0xb8/0x140 [ 2604.203129] [] ? d_kill+0x3c/0x70 [ 2604.203129] [] ? prune_one_dentry+0xb1/0xd0 [ 2604.203129] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2604.203129] [] ? shrink_dcache_ub+0x29/0x70 [ 2604.203129] [] ? ub_dcache_shrink+0x3e/0x50 [ 2604.203129] [] ? ub_dcache_charge+0x96/0x120 [ 2604.203129] [] ? d_alloc+0x45/0x280 [ 2604.203129] [] ? do_lookup+0x18d/0x210 [ 2604.203129] [] ? __link_path_walk+0x89e/0x10c0 [ 2604.203129] [] ? vma_prio_tree_insert+0x30/0x50 [ 2604.203129] [] ? path_walk+0x6a/0xe0 [ 2604.203129] [] ? do_path_lookup+0x5b/0xa0 [ 2604.203129] [] ? user_path_at+0x59/0xa0 [ 2604.203129] [] ? __do_page_fault+0x264/0x4f0 [ 2604.203129] [] ? vfs_fstatat+0x46/0x80 [ 2604.203129] [] ? vfs_lstat+0x1e/0x20 [ 2604.203129] [] ? sys32_lstat64+0x24/0x40 [ 2604.203129] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2604.203129] [] ? ia32_tracesys+0x48/0x8c [ 2604.203129] [] ? ia32_sysret+0x0/0x5 [ 2604.203129] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2604.203129] Call Trace: [ 2604.203129] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2604.203129] [] ? iput+0x30/0x70 [ 2604.203129] [] ? dentry_iput+0xb8/0x140 [ 2604.203129] [] ? d_kill+0x3c/0x70 [ 2604.203129] [] ? prune_one_dentry+0xb1/0xd0 [ 2604.203129] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2604.203129] [] ? shrink_dcache_ub+0x29/0x70 [ 2604.203129] [] ? ub_dcache_shrink+0x3e/0x50 [ 2604.203129] [] ? ub_dcache_charge+0x96/0x120 [ 2604.203129] [] ? d_alloc+0x45/0x280 [ 2604.203129] [] ? do_lookup+0x18d/0x210 [ 2604.203129] [] ? __link_path_walk+0x89e/0x10c0 [ 2604.203129] [] ? vma_prio_tree_insert+0x30/0x50 [ 2604.203129] [] ? path_walk+0x6a/0xe0 [ 2604.203129] [] ? do_path_lookup+0x5b/0xa0 [ 2604.203129] [] ? user_path_at+0x59/0xa0 [ 2604.203129] [] ? __do_page_fault+0x264/0x4f0 [ 2604.203129] [] ? vfs_fstatat+0x46/0x80 [ 2604.203129] [] ? vfs_lstat+0x1e/0x20 [ 2604.203129] [] ? sys32_lstat64+0x24/0x40 [ 2604.203129] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2604.203129] [] ? ia32_tracesys+0x48/0x8c [ 2604.203129] [] ? ia32_sysret+0x0/0x5 [ 2604.265012] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 2604.265016] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2604.265129] CPU 3 [ 2604.265131] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2604.265192] [ 2604.265192] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2604.265192] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 2604.265192] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 2604.265192] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 2604.265192] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2604.265192] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 2604.265192] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2604.265192] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 2604.265192] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 2604.265192] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2604.265192] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 2604.265192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2604.265192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2604.265192] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 2604.265192] Stack: [ 2604.265192] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 2604.265192] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 2604.265192] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 2604.265192] Call Trace: [ 2604.265192] [] ? new_inode+0x3b/0xc0 [ 2604.265192] [] ? create_write_pipe+0x35/0x1f0 [ 2604.265192] [] ? do_sigaction+0x1bb/0x200 [ 2604.265192] [] ? do_pipe_flags+0x57/0x130 [ 2604.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 2604.265192] [] ? sys_pipe2+0x24/0x80 [ 2604.265192] [] ? sys_pipe+0x10/0x20 [ 2604.265192] [] ? system_call_fastpath+0x16/0x1b [ 2604.265192] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 2604.265192] Call Trace: [ 2604.265192] [] ? new_inode+0x3b/0xc0 [ 2604.265192] [] ? create_write_pipe+0x35/0x1f0 [ 2604.265192] [] ? do_sigaction+0x1bb/0x200 [ 2604.265192] [] ? do_pipe_flags+0x57/0x130 [ 2604.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 2604.265192] [] ? sys_pipe2+0x24/0x80 [ 2604.265192] [] ? sys_pipe+0x10/0x20 [ 2604.265192] [] ? system_call_fastpath+0x16/0x1b [ 2604.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2604.076003] CPU 0 [ 2604.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2604.076003] [ 2604.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2604.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2604.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 2604.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 2604.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2604.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 2604.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2604.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 2604.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 2604.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2604.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 2604.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2604.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2604.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 2604.076003] Stack: [ 2604.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 2604.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 2604.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 2604.076003] Call Trace: [ 2604.076003] [] ? new_inode+0x3b/0xc0 [ 2604.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2604.076003] [] ? do_sigaction+0x1bb/0x200 [ 2604.076003] [] ? do_pipe_flags+0x57/0x130 [ 2604.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2604.076003] [] ? sys_pipe2+0x24/0x80 [ 2604.076003] [] ? sys_pipe+0x10/0x20 [ 2604.076003] [] ? system_call_fastpath+0x16/0x1b [ 2604.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2604.076003] Call Trace: [ 2604.076003] [] ? new_inode+0x3b/0xc0 [ 2604.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2604.076003] [] ? do_sigaction+0x1bb/0x200 [ 2604.076003] [] ? do_pipe_flags+0x57/0x130 [ 2604.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2604.076003] [] ? sys_pipe2+0x24/0x80 [ 2604.076003] [] ? sys_pipe+0x10/0x20 [ 2604.076003] [] ? system_call_fastpath+0x16/0x1b [ 2688.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 2688.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 2688.139005] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 2688.139007] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2688.139070] CPU 1 [ 2688.139070] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2688.139070] [ 2688.139070] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2688.139070] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2688.139070] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 2688.139070] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 2688.139070] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2688.139070] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 2688.139070] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 2688.139070] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 2688.139070] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 2688.139070] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2688.139070] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 2688.139070] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2688.139070] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2688.139070] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 2688.139070] Stack: [ 2688.139070] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 2688.139070] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 2688.139070] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 2688.139070] Call Trace: [ 2688.139070] [] ? new_inode+0x3b/0xc0 [ 2688.139070] [] ? create_write_pipe+0x35/0x1f0 [ 2688.139070] [] ? do_sigaction+0x1bb/0x200 [ 2688.139070] [] ? do_pipe_flags+0x57/0x130 [ 2688.139070] [] ? audit_syscall_entry+0x286/0x2b0 [ 2688.139070] [] ? sys_pipe2+0x24/0x80 [ 2688.139070] [] ? sys_pipe+0x10/0x20 [ 2688.139070] [] ? system_call_fastpath+0x16/0x1b [ 2688.139070] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2688.139070] Call Trace: [ 2688.139070] [] ? new_inode+0x3b/0xc0 [ 2688.139070] [] ? create_write_pipe+0x35/0x1f0 [ 2688.139070] [] ? do_sigaction+0x1bb/0x200 [ 2688.139070] [] ? do_pipe_flags+0x57/0x130 [ 2688.139070] [] ? audit_syscall_entry+0x286/0x2b0 [ 2688.139070] [] ? sys_pipe2+0x24/0x80 [ 2688.139070] [] ? sys_pipe+0x10/0x20 [ 2688.139070] [] ? system_call_fastpath+0x16/0x1b [ 2688.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 2688.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2688.203096] CPU 2 [ 2688.203098] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2688.203131] [ 2688.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2688.203131] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 2688.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 2688.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 2688.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 2688.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 2688.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 2688.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 2688.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 2688.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 2688.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 2688.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2688.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2688.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 2688.203131] Stack: [ 2688.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 2688.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 2688.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 2688.203131] Call Trace: [ 2688.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2688.203131] [] ? iput+0x30/0x70 [ 2688.203131] [] ? dentry_iput+0xb8/0x140 [ 2688.203131] [] ? d_kill+0x3c/0x70 [ 2688.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2688.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2688.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2688.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2688.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2688.203131] [] ? d_alloc+0x45/0x280 [ 2688.203131] [] ? do_lookup+0x18d/0x210 [ 2688.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2688.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2688.203131] [] ? path_walk+0x6a/0xe0 [ 2688.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2688.203131] [] ? user_path_at+0x59/0xa0 [ 2688.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2688.203131] [] ? vfs_fstatat+0x46/0x80 [ 2688.203131] [] ? vfs_lstat+0x1e/0x20 [ 2688.203131] [] ? sys32_lstat64+0x24/0x40 [ 2688.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2688.203131] [] ? ia32_tracesys+0x48/0x8c [ 2688.203131] [] ? ia32_sysret+0x0/0x5 [ 2688.203131] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 2688.203131] Call Trace: [ 2688.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2688.203131] [] ? iput+0x30/0x70 [ 2688.203131] [] ? dentry_iput+0xb8/0x140 [ 2688.203131] [] ? d_kill+0x3c/0x70 [ 2688.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2688.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2688.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2688.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2688.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2688.203131] [] ? d_alloc+0x45/0x280 [ 2688.203131] [] ? do_lookup+0x18d/0x210 [ 2688.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2688.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2688.203131] [] ? path_walk+0x6a/0xe0 [ 2688.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2688.203131] [] ? user_path_at+0x59/0xa0 [ 2688.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2688.203131] [] ? vfs_fstatat+0x46/0x80 [ 2688.203131] [] ? vfs_lstat+0x1e/0x20 [ 2688.203131] [] ? sys32_lstat64+0x24/0x40 [ 2688.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2688.203131] [] ? ia32_tracesys+0x48/0x8c [ 2688.203131] [] ? ia32_sysret+0x0/0x5 [ 2688.265008] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 2688.265011] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2688.265124] CPU 3 [ 2688.265125] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2688.265193] [ 2688.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2688.265193] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 2688.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 2688.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 2688.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2688.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 2688.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2688.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 2688.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 2688.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2688.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 2688.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2688.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2688.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 2688.265193] Stack: [ 2688.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 2688.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 2688.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 2688.265193] Call Trace: [ 2688.265193] [] ? new_inode+0x3b/0xc0 [ 2688.265193] [] ? create_write_pipe+0x35/0x1f0 [ 2688.265193] [] ? do_sigaction+0x1bb/0x200 [ 2688.265193] [] ? do_pipe_flags+0x57/0x130 [ 2688.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 2688.265193] [] ? sys_pipe2+0x24/0x80 [ 2688.265193] [] ? sys_pipe+0x10/0x20 [ 2688.265193] [] ? system_call_fastpath+0x16/0x1b [ 2688.265193] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 2688.265193] Call Trace: [ 2688.265193] [] ? new_inode+0x3b/0xc0 [ 2688.265193] [] ? create_write_pipe+0x35/0x1f0 [ 2688.265193] [] ? do_sigaction+0x1bb/0x200 [ 2688.265193] [] ? do_pipe_flags+0x57/0x130 [ 2688.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 2688.265193] [] ? sys_pipe2+0x24/0x80 [ 2688.265193] [] ? sys_pipe+0x10/0x20 [ 2688.265193] [] ? system_call_fastpath+0x16/0x1b [ 2688.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2688.076003] CPU 0 [ 2688.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2688.076003] [ 2688.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2688.076003] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 2688.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 2688.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 2688.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2688.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 2688.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2688.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 2688.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 2688.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2688.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 2688.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2688.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2688.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 2688.076003] Stack: [ 2688.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 2688.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 2688.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 2688.076003] Call Trace: [ 2688.076003] [] ? new_inode+0x3b/0xc0 [ 2688.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2688.076003] [] ? do_sigaction+0x1bb/0x200 [ 2688.076003] [] ? do_pipe_flags+0x57/0x130 [ 2688.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2688.076003] [] ? sys_pipe2+0x24/0x80 [ 2688.076003] [] ? sys_pipe+0x10/0x20 [ 2688.076003] [] ? system_call_fastpath+0x16/0x1b [ 2688.076003] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 2688.076003] Call Trace: [ 2688.076003] [] ? new_inode+0x3b/0xc0 [ 2688.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2688.076003] [] ? do_sigaction+0x1bb/0x200 [ 2688.076003] [] ? do_pipe_flags+0x57/0x130 [ 2688.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2688.076003] [] ? sys_pipe2+0x24/0x80 [ 2688.076003] [] ? sys_pipe+0x10/0x20 [ 2688.076003] [] ? system_call_fastpath+0x16/0x1b [ 2772.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 2772.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 2772.139006] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 2772.139008] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2772.139069] CPU 1 [ 2772.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2772.139069] [ 2772.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2772.139069] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 2772.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 2772.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 2772.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2772.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 2772.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 2772.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 2772.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 2772.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2772.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 2772.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2772.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2772.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 2772.139069] Stack: [ 2772.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 2772.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 2772.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 2772.139069] Call Trace: [ 2772.139069] [] ? new_inode+0x3b/0xc0 [ 2772.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2772.139069] [] ? do_sigaction+0x1bb/0x200 [ 2772.139069] [] ? do_pipe_flags+0x57/0x130 [ 2772.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2772.139069] [] ? sys_pipe2+0x24/0x80 [ 2772.139069] [] ? sys_pipe+0x10/0x20 [ 2772.139069] [] ? system_call_fastpath+0x16/0x1b [ 2772.139069] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 2772.139069] Call Trace: [ 2772.139069] [] ? new_inode+0x3b/0xc0 [ 2772.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2772.139069] [] ? do_sigaction+0x1bb/0x200 [ 2772.139069] [] ? do_pipe_flags+0x57/0x130 [ 2772.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2772.139069] [] ? sys_pipe2+0x24/0x80 [ 2772.139069] [] ? sys_pipe+0x10/0x20 [ 2772.139069] [] ? system_call_fastpath+0x16/0x1b [ 2772.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 2772.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2772.203096] CPU 2 [ 2772.203097] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2772.203131] [ 2772.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2772.203131] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2772.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 2772.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 2772.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 2772.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 2772.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 2772.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 2772.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 2772.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 2772.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 2772.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2772.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2772.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 2772.203131] Stack: [ 2772.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 2772.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 2772.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 2772.203131] Call Trace: [ 2772.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2772.203131] [] ? iput+0x30/0x70 [ 2772.203131] [] ? dentry_iput+0xb8/0x140 [ 2772.203131] [] ? d_kill+0x3c/0x70 [ 2772.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2772.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2772.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2772.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2772.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2772.203131] [] ? d_alloc+0x45/0x280 [ 2772.203131] [] ? do_lookup+0x18d/0x210 [ 2772.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2772.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2772.203131] [] ? path_walk+0x6a/0xe0 [ 2772.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2772.203131] [] ? user_path_at+0x59/0xa0 [ 2772.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2772.203131] [] ? vfs_fstatat+0x46/0x80 [ 2772.203131] [] ? vfs_lstat+0x1e/0x20 [ 2772.203131] [] ? sys32_lstat64+0x24/0x40 [ 2772.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2772.203131] [] ? ia32_tracesys+0x48/0x8c [ 2772.203131] [] ? ia32_sysret+0x0/0x5 [ 2772.203131] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2772.203131] Call Trace: [ 2772.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2772.203131] [] ? iput+0x30/0x70 [ 2772.203131] [] ? dentry_iput+0xb8/0x140 [ 2772.203131] [] ? d_kill+0x3c/0x70 [ 2772.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2772.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2772.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2772.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2772.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2772.203131] [] ? d_alloc+0x45/0x280 [ 2772.203131] [] ? do_lookup+0x18d/0x210 [ 2772.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2772.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2772.203131] [] ? path_walk+0x6a/0xe0 [ 2772.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2772.203131] [] ? user_path_at+0x59/0xa0 [ 2772.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2772.203131] [] ? vfs_fstatat+0x46/0x80 [ 2772.203131] [] ? vfs_lstat+0x1e/0x20 [ 2772.203131] [] ? sys32_lstat64+0x24/0x40 [ 2772.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2772.203131] [] ? ia32_tracesys+0x48/0x8c [ 2772.203131] [] ? ia32_sysret+0x0/0x5 [ 2772.265008] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 2772.265012] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2772.265124] CPU 3 [ 2772.265125] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2772.265193] [ 2772.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2772.265193] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2772.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 2772.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 2772.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2772.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 2772.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2772.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 2772.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 2772.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2772.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 2772.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2772.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2772.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 2772.265193] Stack: [ 2772.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 2772.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 2772.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 2772.265193] Call Trace: [ 2772.265193] [] ? new_inode+0x3b/0xc0 [ 2772.265193] [] ? create_write_pipe+0x35/0x1f0 [ 2772.265193] [] ? do_sigaction+0x1bb/0x200 [ 2772.265193] [] ? do_pipe_flags+0x57/0x130 [ 2772.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 2772.265193] [] ? sys_pipe2+0x24/0x80 [ 2772.265193] [] ? sys_pipe+0x10/0x20 [ 2772.265193] [] ? system_call_fastpath+0x16/0x1b [ 2772.265193] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2772.265193] Call Trace: [ 2772.265193] [] ? new_inode+0x3b/0xc0 [ 2772.265193] [] ? create_write_pipe+0x35/0x1f0 [ 2772.265193] [] ? do_sigaction+0x1bb/0x200 [ 2772.265193] [] ? do_pipe_flags+0x57/0x130 [ 2772.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 2772.265193] [] ? sys_pipe2+0x24/0x80 [ 2772.265193] [] ? sys_pipe+0x10/0x20 [ 2772.265193] [] ? system_call_fastpath+0x16/0x1b [ 2772.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2772.076003] CPU 0 [ 2772.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2772.076003] [ 2772.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2772.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2772.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 2772.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 2772.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2772.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 2772.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2772.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 2772.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 2772.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2772.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 2772.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2772.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2772.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 2772.076003] Stack: [ 2772.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 2772.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 2772.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 2772.076003] Call Trace: [ 2772.076003] [] ? new_inode+0x3b/0xc0 [ 2772.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2772.076003] [] ? do_sigaction+0x1bb/0x200 [ 2772.076003] [] ? do_pipe_flags+0x57/0x130 [ 2772.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2772.076003] [] ? sys_pipe2+0x24/0x80 [ 2772.076003] [] ? sys_pipe+0x10/0x20 [ 2772.076003] [] ? system_call_fastpath+0x16/0x1b [ 2772.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2772.076003] Call Trace: [ 2772.076003] [] ? new_inode+0x3b/0xc0 [ 2772.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2772.076003] [] ? do_sigaction+0x1bb/0x200 [ 2772.076003] [] ? do_pipe_flags+0x57/0x130 [ 2772.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2772.076003] [] ? sys_pipe2+0x24/0x80 [ 2772.076003] [] ? sys_pipe+0x10/0x20 [ 2772.076003] [] ? system_call_fastpath+0x16/0x1b [ 2856.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 2856.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 2856.139006] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 2856.139009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2856.139069] CPU 1 [ 2856.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2856.139069] [ 2856.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2856.139069] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2856.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 2856.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 2856.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2856.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 2856.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 2856.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 2856.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 2856.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2856.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 2856.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2856.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2856.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 2856.139069] Stack: [ 2856.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 2856.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 2856.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 2856.139069] Call Trace: [ 2856.139069] [] ? new_inode+0x3b/0xc0 [ 2856.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2856.139069] [] ? do_sigaction+0x1bb/0x200 [ 2856.139069] [] ? do_pipe_flags+0x57/0x130 [ 2856.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2856.139069] [] ? sys_pipe2+0x24/0x80 [ 2856.139069] [] ? sys_pipe+0x10/0x20 [ 2856.139069] [] ? system_call_fastpath+0x16/0x1b [ 2856.139069] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2856.139069] Call Trace: [ 2856.139069] [] ? new_inode+0x3b/0xc0 [ 2856.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2856.139069] [] ? do_sigaction+0x1bb/0x200 [ 2856.139069] [] ? do_pipe_flags+0x57/0x130 [ 2856.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2856.139069] [] ? sys_pipe2+0x24/0x80 [ 2856.139069] [] ? sys_pipe+0x10/0x20 [ 2856.139069] [] ? system_call_fastpath+0x16/0x1b [ 2856.203006] BUG: soft lockup - CPU#2 stuck for 68s! [cvs:29276] [ 2856.203008] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2856.203096] CPU 2 [ 2856.203097] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2856.203129] [ 2856.203129] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2856.203129] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 2856.203129] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 2856.203129] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 2856.203129] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 2856.203129] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 2856.203129] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 2856.203129] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 2856.203129] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 2856.203129] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 2856.203129] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 2856.203129] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2856.203129] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2856.203129] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 2856.203129] Stack: [ 2856.203129] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 2856.203129] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 2856.203129] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 2856.203129] Call Trace: [ 2856.203129] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2856.203129] [] ? iput+0x30/0x70 [ 2856.203129] [] ? dentry_iput+0xb8/0x140 [ 2856.203129] [] ? d_kill+0x3c/0x70 [ 2856.203129] [] ? prune_one_dentry+0xb1/0xd0 [ 2856.203129] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2856.203129] [] ? shrink_dcache_ub+0x29/0x70 [ 2856.203129] [] ? ub_dcache_shrink+0x3e/0x50 [ 2856.203129] [] ? ub_dcache_charge+0x96/0x120 [ 2856.203129] [] ? d_alloc+0x45/0x280 [ 2856.203129] [] ? do_lookup+0x18d/0x210 [ 2856.203129] [] ? __link_path_walk+0x89e/0x10c0 [ 2856.203129] [] ? vma_prio_tree_insert+0x30/0x50 [ 2856.203129] [] ? path_walk+0x6a/0xe0 [ 2856.203129] [] ? do_path_lookup+0x5b/0xa0 [ 2856.203129] [] ? user_path_at+0x59/0xa0 [ 2856.203129] [] ? __do_page_fault+0x264/0x4f0 [ 2856.203129] [] ? vfs_fstatat+0x46/0x80 [ 2856.203129] [] ? vfs_lstat+0x1e/0x20 [ 2856.203129] [] ? sys32_lstat64+0x24/0x40 [ 2856.203129] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2856.203129] [] ? ia32_tracesys+0x48/0x8c [ 2856.203129] [] ? ia32_sysret+0x0/0x5 [ 2856.203129] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 2856.203129] Call Trace: [ 2856.203129] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2856.203129] [] ? iput+0x30/0x70 [ 2856.203129] [] ? dentry_iput+0xb8/0x140 [ 2856.203129] [] ? d_kill+0x3c/0x70 [ 2856.203129] [] ? prune_one_dentry+0xb1/0xd0 [ 2856.203129] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2856.203129] [] ? shrink_dcache_ub+0x29/0x70 [ 2856.203129] [] ? ub_dcache_shrink+0x3e/0x50 [ 2856.203129] [] ? ub_dcache_charge+0x96/0x120 [ 2856.203129] [] ? d_alloc+0x45/0x280 [ 2856.203129] [] ? do_lookup+0x18d/0x210 [ 2856.203129] [] ? __link_path_walk+0x89e/0x10c0 [ 2856.203129] [] ? vma_prio_tree_insert+0x30/0x50 [ 2856.203129] [] ? path_walk+0x6a/0xe0 [ 2856.203129] [] ? do_path_lookup+0x5b/0xa0 [ 2856.203129] [] ? user_path_at+0x59/0xa0 [ 2856.203129] [] ? __do_page_fault+0x264/0x4f0 [ 2856.203129] [] ? vfs_fstatat+0x46/0x80 [ 2856.203129] [] ? vfs_lstat+0x1e/0x20 [ 2856.203129] [] ? sys32_lstat64+0x24/0x40 [ 2856.203129] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2856.203129] [] ? ia32_tracesys+0x48/0x8c [ 2856.203129] [] ? ia32_sysret+0x0/0x5 [ 2856.265011] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 2856.265016] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2856.265126] CPU 3 [ 2856.265127] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2856.265192] [ 2856.265192] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2856.265192] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 2856.265192] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 2856.265192] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 2856.265192] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2856.265192] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 2856.265192] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2856.265192] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 2856.265192] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 2856.265192] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2856.265192] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 2856.265192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2856.265192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2856.265192] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 2856.265192] Stack: [ 2856.265192] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 2856.265192] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 2856.265192] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 2856.265192] Call Trace: [ 2856.265192] [] ? new_inode+0x3b/0xc0 [ 2856.265192] [] ? create_write_pipe+0x35/0x1f0 [ 2856.265192] [] ? do_sigaction+0x1bb/0x200 [ 2856.265192] [] ? do_pipe_flags+0x57/0x130 [ 2856.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 2856.265192] [] ? sys_pipe2+0x24/0x80 [ 2856.265192] [] ? sys_pipe+0x10/0x20 [ 2856.265192] [] ? system_call_fastpath+0x16/0x1b [ 2856.265192] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 2856.265192] Call Trace: [ 2856.265192] [] ? new_inode+0x3b/0xc0 [ 2856.265192] [] ? create_write_pipe+0x35/0x1f0 [ 2856.265192] [] ? do_sigaction+0x1bb/0x200 [ 2856.265192] [] ? do_pipe_flags+0x57/0x130 [ 2856.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 2856.265192] [] ? sys_pipe2+0x24/0x80 [ 2856.265192] [] ? sys_pipe+0x10/0x20 [ 2856.265192] [] ? system_call_fastpath+0x16/0x1b [ 2856.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2856.076003] CPU 0 [ 2856.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2856.076003] [ 2856.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2856.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 2856.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 2856.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 2856.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2856.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 2856.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2856.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 2856.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 2856.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2856.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 2856.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2856.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2856.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 2856.076003] Stack: [ 2856.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 2856.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 2856.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 2856.076003] Call Trace: [ 2856.076003] [] ? new_inode+0x3b/0xc0 [ 2856.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2856.076003] [] ? do_sigaction+0x1bb/0x200 [ 2856.076003] [] ? do_pipe_flags+0x57/0x130 [ 2856.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2856.076003] [] ? sys_pipe2+0x24/0x80 [ 2856.076003] [] ? sys_pipe+0x10/0x20 [ 2856.076003] [] ? system_call_fastpath+0x16/0x1b [ 2856.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 2856.076003] Call Trace: [ 2856.076003] [] ? new_inode+0x3b/0xc0 [ 2856.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2856.076003] [] ? do_sigaction+0x1bb/0x200 [ 2856.076003] [] ? do_pipe_flags+0x57/0x130 [ 2856.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2856.076003] [] ? sys_pipe2+0x24/0x80 [ 2856.076003] [] ? sys_pipe+0x10/0x20 [ 2856.076003] [] ? system_call_fastpath+0x16/0x1b [ 2940.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 2940.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 2940.139005] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 2940.139008] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2940.139069] CPU 1 [ 2940.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2940.139069] [ 2940.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2940.139069] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 2940.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 2940.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 2940.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2940.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 2940.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 2940.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 2940.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 2940.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2940.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 2940.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2940.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2940.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 2940.139069] Stack: [ 2940.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 2940.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 2940.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 2940.139069] Call Trace: [ 2940.139069] [] ? new_inode+0x3b/0xc0 [ 2940.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2940.139069] [] ? do_sigaction+0x1bb/0x200 [ 2940.139069] [] ? do_pipe_flags+0x57/0x130 [ 2940.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2940.139069] [] ? sys_pipe2+0x24/0x80 [ 2940.139069] [] ? sys_pipe+0x10/0x20 [ 2940.139069] [] ? system_call_fastpath+0x16/0x1b [ 2940.139069] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 2940.139069] Call Trace: [ 2940.139069] [] ? new_inode+0x3b/0xc0 [ 2940.139069] [] ? create_write_pipe+0x35/0x1f0 [ 2940.139069] [] ? do_sigaction+0x1bb/0x200 [ 2940.139069] [] ? do_pipe_flags+0x57/0x130 [ 2940.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 2940.139069] [] ? sys_pipe2+0x24/0x80 [ 2940.139069] [] ? sys_pipe+0x10/0x20 [ 2940.139069] [] ? system_call_fastpath+0x16/0x1b [ 2940.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 2940.203010] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2940.203097] CPU 2 [ 2940.203098] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2940.203131] [ 2940.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2940.203131] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 2940.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 2940.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 2940.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 2940.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 2940.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 2940.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 2940.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 2940.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 2940.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 2940.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2940.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2940.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 2940.203131] Stack: [ 2940.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 2940.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 2940.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 2940.203131] Call Trace: [ 2940.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2940.203131] [] ? iput+0x30/0x70 [ 2940.203131] [] ? dentry_iput+0xb8/0x140 [ 2940.203131] [] ? d_kill+0x3c/0x70 [ 2940.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2940.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2940.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2940.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2940.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2940.203131] [] ? d_alloc+0x45/0x280 [ 2940.203131] [] ? do_lookup+0x18d/0x210 [ 2940.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2940.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2940.203131] [] ? path_walk+0x6a/0xe0 [ 2940.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2940.203131] [] ? user_path_at+0x59/0xa0 [ 2940.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2940.203131] [] ? vfs_fstatat+0x46/0x80 [ 2940.203131] [] ? vfs_lstat+0x1e/0x20 [ 2940.203131] [] ? sys32_lstat64+0x24/0x40 [ 2940.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2940.203131] [] ? ia32_tracesys+0x48/0x8c [ 2940.203131] [] ? ia32_sysret+0x0/0x5 [ 2940.203131] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 2940.203131] Call Trace: [ 2940.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 2940.203131] [] ? iput+0x30/0x70 [ 2940.203131] [] ? dentry_iput+0xb8/0x140 [ 2940.203131] [] ? d_kill+0x3c/0x70 [ 2940.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 2940.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 2940.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 2940.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 2940.203131] [] ? ub_dcache_charge+0x96/0x120 [ 2940.203131] [] ? d_alloc+0x45/0x280 [ 2940.203131] [] ? do_lookup+0x18d/0x210 [ 2940.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 2940.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 2940.203131] [] ? path_walk+0x6a/0xe0 [ 2940.203131] [] ? do_path_lookup+0x5b/0xa0 [ 2940.203131] [] ? user_path_at+0x59/0xa0 [ 2940.203131] [] ? __do_page_fault+0x264/0x4f0 [ 2940.203131] [] ? vfs_fstatat+0x46/0x80 [ 2940.203131] [] ? vfs_lstat+0x1e/0x20 [ 2940.203131] [] ? sys32_lstat64+0x24/0x40 [ 2940.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 2940.203131] [] ? ia32_tracesys+0x48/0x8c [ 2940.203131] [] ? ia32_sysret+0x0/0x5 [ 2940.265008] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 2940.265012] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2940.265125] CPU 3 [ 2940.265126] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2940.265192] [ 2940.265192] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2940.265192] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 2940.265192] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 2940.265192] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 2940.265192] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2940.265192] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 2940.265192] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2940.265192] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 2940.265192] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 2940.265192] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2940.265192] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 2940.265192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2940.265192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2940.265192] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 2940.265192] Stack: [ 2940.265192] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 2940.265192] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 2940.265192] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 2940.265192] Call Trace: [ 2940.265192] [] ? new_inode+0x3b/0xc0 [ 2940.265192] [] ? create_write_pipe+0x35/0x1f0 [ 2940.265192] [] ? do_sigaction+0x1bb/0x200 [ 2940.265192] [] ? do_pipe_flags+0x57/0x130 [ 2940.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 2940.265192] [] ? sys_pipe2+0x24/0x80 [ 2940.265192] [] ? sys_pipe+0x10/0x20 [ 2940.265192] [] ? system_call_fastpath+0x16/0x1b [ 2940.265192] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 2940.265192] Call Trace: [ 2940.265192] [] ? new_inode+0x3b/0xc0 [ 2940.265192] [] ? create_write_pipe+0x35/0x1f0 [ 2940.265192] [] ? do_sigaction+0x1bb/0x200 [ 2940.265192] [] ? do_pipe_flags+0x57/0x130 [ 2940.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 2940.265192] [] ? sys_pipe2+0x24/0x80 [ 2940.265192] [] ? sys_pipe+0x10/0x20 [ 2940.265192] [] ? system_call_fastpath+0x16/0x1b [ 2940.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2940.076003] CPU 0 [ 2940.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 2940.076003] [ 2940.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 2940.076003] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 2940.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 2940.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 2940.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 2940.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 2940.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 2940.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 2940.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 2940.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2940.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 2940.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2940.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2940.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 2940.076003] Stack: [ 2940.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 2940.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 2940.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 2940.076003] Call Trace: [ 2940.076003] [] ? new_inode+0x3b/0xc0 [ 2940.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2940.076003] [] ? do_sigaction+0x1bb/0x200 [ 2940.076003] [] ? do_pipe_flags+0x57/0x130 [ 2940.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2940.076003] [] ? sys_pipe2+0x24/0x80 [ 2940.076003] [] ? sys_pipe+0x10/0x20 [ 2940.076003] [] ? system_call_fastpath+0x16/0x1b [ 2940.076003] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 2940.076003] Call Trace: [ 2940.076003] [] ? new_inode+0x3b/0xc0 [ 2940.076003] [] ? create_write_pipe+0x35/0x1f0 [ 2940.076003] [] ? do_sigaction+0x1bb/0x200 [ 2940.076003] [] ? do_pipe_flags+0x57/0x130 [ 2940.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 2940.076003] [] ? sys_pipe2+0x24/0x80 [ 2940.076003] [] ? sys_pipe+0x10/0x20 [ 2940.076003] [] ? system_call_fastpath+0x16/0x1b [ 3024.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 3024.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 3024.139005] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 3024.139007] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3024.139069] CPU 1 [ 3024.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3024.139069] [ 3024.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3024.139069] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3024.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 3024.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 3024.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3024.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 3024.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 3024.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 3024.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 3024.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3024.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 3024.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3024.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3024.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 3024.139069] Stack: [ 3024.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 3024.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 3024.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 3024.139069] Call Trace: [ 3024.139069] [] ? new_inode+0x3b/0xc0 [ 3024.139069] [] ? create_write_pipe+0x35/0x1f0 [ 3024.139069] [] ? do_sigaction+0x1bb/0x200 [ 3024.139069] [] ? do_pipe_flags+0x57/0x130 [ 3024.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 3024.139069] [] ? sys_pipe2+0x24/0x80 [ 3024.139069] [] ? sys_pipe+0x10/0x20 [ 3024.139069] [] ? system_call_fastpath+0x16/0x1b [ 3024.139069] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3024.139069] Call Trace: [ 3024.139069] [] ? new_inode+0x3b/0xc0 [ 3024.139069] [] ? create_write_pipe+0x35/0x1f0 [ 3024.139069] [] ? do_sigaction+0x1bb/0x200 [ 3024.139069] [] ? do_pipe_flags+0x57/0x130 [ 3024.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 3024.139069] [] ? sys_pipe2+0x24/0x80 [ 3024.139069] [] ? sys_pipe+0x10/0x20 [ 3024.139069] [] ? system_call_fastpath+0x16/0x1b [ 3024.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 3024.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3024.203096] CPU 2 [ 3024.203097] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3024.203131] [ 3024.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3024.203131] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3024.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 3024.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 3024.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 3024.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 3024.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 3024.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 3024.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 3024.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 3024.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 3024.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3024.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3024.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 3024.203131] Stack: [ 3024.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 3024.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 3024.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 3024.203131] Call Trace: [ 3024.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3024.203131] [] ? iput+0x30/0x70 [ 3024.203131] [] ? dentry_iput+0xb8/0x140 [ 3024.203131] [] ? d_kill+0x3c/0x70 [ 3024.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3024.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3024.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3024.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3024.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3024.203131] [] ? d_alloc+0x45/0x280 [ 3024.203131] [] ? do_lookup+0x18d/0x210 [ 3024.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3024.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3024.203131] [] ? path_walk+0x6a/0xe0 [ 3024.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3024.203131] [] ? user_path_at+0x59/0xa0 [ 3024.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3024.203131] [] ? vfs_fstatat+0x46/0x80 [ 3024.203131] [] ? vfs_lstat+0x1e/0x20 [ 3024.203131] [] ? sys32_lstat64+0x24/0x40 [ 3024.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3024.203131] [] ? ia32_tracesys+0x48/0x8c [ 3024.203131] [] ? ia32_sysret+0x0/0x5 [ 3024.203131] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3024.203131] Call Trace: [ 3024.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3024.203131] [] ? iput+0x30/0x70 [ 3024.203131] [] ? dentry_iput+0xb8/0x140 [ 3024.203131] [] ? d_kill+0x3c/0x70 [ 3024.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3024.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3024.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3024.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3024.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3024.203131] [] ? d_alloc+0x45/0x280 [ 3024.203131] [] ? do_lookup+0x18d/0x210 [ 3024.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3024.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3024.203131] [] ? path_walk+0x6a/0xe0 [ 3024.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3024.203131] [] ? user_path_at+0x59/0xa0 [ 3024.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3024.203131] [] ? vfs_fstatat+0x46/0x80 [ 3024.203131] [] ? vfs_lstat+0x1e/0x20 [ 3024.203131] [] ? sys32_lstat64+0x24/0x40 [ 3024.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3024.203131] [] ? ia32_tracesys+0x48/0x8c [ 3024.203131] [] ? ia32_sysret+0x0/0x5 [ 3024.265008] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 3024.265013] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3024.265130] CPU 3 [ 3024.265131] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3024.265193] [ 3024.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3024.265193] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3024.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 3024.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 3024.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3024.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 3024.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3024.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 3024.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 3024.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3024.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 3024.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3024.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3024.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 3024.265193] Stack: [ 3024.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 3024.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 3024.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 3024.265193] Call Trace: [ 3024.265193] [] ? new_inode+0x3b/0xc0 [ 3024.265193] [] ? create_write_pipe+0x35/0x1f0 [ 3024.265193] [] ? do_sigaction+0x1bb/0x200 [ 3024.265193] [] ? do_pipe_flags+0x57/0x130 [ 3024.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 3024.265193] [] ? sys_pipe2+0x24/0x80 [ 3024.265193] [] ? sys_pipe+0x10/0x20 [ 3024.265193] [] ? system_call_fastpath+0x16/0x1b [ 3024.265193] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3024.265193] Call Trace: [ 3024.265193] [] ? new_inode+0x3b/0xc0 [ 3024.265193] [] ? create_write_pipe+0x35/0x1f0 [ 3024.265193] [] ? do_sigaction+0x1bb/0x200 [ 3024.265193] [] ? do_pipe_flags+0x57/0x130 [ 3024.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 3024.265193] [] ? sys_pipe2+0x24/0x80 [ 3024.265193] [] ? sys_pipe+0x10/0x20 [ 3024.265193] [] ? system_call_fastpath+0x16/0x1b [ 3024.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3024.076003] CPU 0 [ 3024.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3024.076003] [ 3024.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3024.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3024.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 3024.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 3024.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3024.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 3024.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3024.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 3024.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 3024.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3024.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 3024.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3024.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3024.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 3024.076003] Stack: [ 3024.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 3024.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 3024.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 3024.076003] Call Trace: [ 3024.076003] [] ? new_inode+0x3b/0xc0 [ 3024.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3024.076003] [] ? do_sigaction+0x1bb/0x200 [ 3024.076003] [] ? do_pipe_flags+0x57/0x130 [ 3024.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3024.076003] [] ? sys_pipe2+0x24/0x80 [ 3024.076003] [] ? sys_pipe+0x10/0x20 [ 3024.076003] [] ? system_call_fastpath+0x16/0x1b [ 3024.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3024.076003] Call Trace: [ 3024.076003] [] ? new_inode+0x3b/0xc0 [ 3024.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3024.076003] [] ? do_sigaction+0x1bb/0x200 [ 3024.076003] [] ? do_pipe_flags+0x57/0x130 [ 3024.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3024.076003] [] ? sys_pipe2+0x24/0x80 [ 3024.076003] [] ? sys_pipe+0x10/0x20 [ 3024.076003] [] ? system_call_fastpath+0x16/0x1b [ 3108.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 3108.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 3108.139006] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 3108.139009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3108.139069] CPU 1 [ 3108.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3108.139069] [ 3108.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3108.139069] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 3108.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 3108.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 3108.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3108.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 3108.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 3108.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 3108.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 3108.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3108.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 3108.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3108.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3108.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 3108.139069] Stack: [ 3108.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 3108.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 3108.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 3108.139069] Call Trace: [ 3108.139069] [] ? new_inode+0x3b/0xc0 [ 3108.139069] [] ? create_write_pipe+0x35/0x1f0 [ 3108.139069] [] ? do_sigaction+0x1bb/0x200 [ 3108.139069] [] ? do_pipe_flags+0x57/0x130 [ 3108.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 3108.139069] [] ? sys_pipe2+0x24/0x80 [ 3108.139069] [] ? sys_pipe+0x10/0x20 [ 3108.139069] [] ? system_call_fastpath+0x16/0x1b [ 3108.139069] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 3108.139069] Call Trace: [ 3108.139069] [] ? new_inode+0x3b/0xc0 [ 3108.139069] [] ? create_write_pipe+0x35/0x1f0 [ 3108.139069] [] ? do_sigaction+0x1bb/0x200 [ 3108.139069] [] ? do_pipe_flags+0x57/0x130 [ 3108.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 3108.139069] [] ? sys_pipe2+0x24/0x80 [ 3108.139069] [] ? sys_pipe+0x10/0x20 [ 3108.139069] [] ? system_call_fastpath+0x16/0x1b [ 3108.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 3108.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3108.203095] CPU 2 [ 3108.203097] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3108.203131] [ 3108.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3108.203131] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3108.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 3108.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 3108.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 3108.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 3108.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 3108.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 3108.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 3108.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 3108.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 3108.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3108.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3108.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 3108.203131] Stack: [ 3108.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 3108.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 3108.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 3108.203131] Call Trace: [ 3108.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3108.203131] [] ? iput+0x30/0x70 [ 3108.203131] [] ? dentry_iput+0xb8/0x140 [ 3108.203131] [] ? d_kill+0x3c/0x70 [ 3108.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3108.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3108.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3108.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3108.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3108.203131] [] ? d_alloc+0x45/0x280 [ 3108.203131] [] ? do_lookup+0x18d/0x210 [ 3108.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3108.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3108.203131] [] ? path_walk+0x6a/0xe0 [ 3108.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3108.203131] [] ? user_path_at+0x59/0xa0 [ 3108.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3108.203131] [] ? vfs_fstatat+0x46/0x80 [ 3108.203131] [] ? vfs_lstat+0x1e/0x20 [ 3108.203131] [] ? sys32_lstat64+0x24/0x40 [ 3108.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3108.203131] [] ? ia32_tracesys+0x48/0x8c [ 3108.203131] [] ? ia32_sysret+0x0/0x5 [ 3108.203131] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3108.203131] Call Trace: [ 3108.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3108.203131] [] ? iput+0x30/0x70 [ 3108.203131] [] ? dentry_iput+0xb8/0x140 [ 3108.203131] [] ? d_kill+0x3c/0x70 [ 3108.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3108.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3108.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3108.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3108.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3108.203131] [] ? d_alloc+0x45/0x280 [ 3108.203131] [] ? do_lookup+0x18d/0x210 [ 3108.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3108.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3108.203131] [] ? path_walk+0x6a/0xe0 [ 3108.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3108.203131] [] ? user_path_at+0x59/0xa0 [ 3108.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3108.203131] [] ? vfs_fstatat+0x46/0x80 [ 3108.203131] [] ? vfs_lstat+0x1e/0x20 [ 3108.203131] [] ? sys32_lstat64+0x24/0x40 [ 3108.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3108.203131] [] ? ia32_tracesys+0x48/0x8c [ 3108.203131] [] ? ia32_sysret+0x0/0x5 [ 3108.265012] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 3108.265016] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3108.265128] CPU 3 [ 3108.265129] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3108.265193] [ 3108.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3108.265193] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 3108.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 3108.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 3108.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3108.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 3108.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3108.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 3108.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 3108.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3108.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 3108.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3108.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3108.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 3108.265193] Stack: [ 3108.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 3108.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 3108.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 3108.265193] Call Trace: [ 3108.265193] [] ? new_inode+0x3b/0xc0 [ 3108.265193] [] ? create_write_pipe+0x35/0x1f0 [ 3108.265193] [] ? do_sigaction+0x1bb/0x200 [ 3108.265193] [] ? do_pipe_flags+0x57/0x130 [ 3108.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 3108.265193] [] ? sys_pipe2+0x24/0x80 [ 3108.265193] [] ? sys_pipe+0x10/0x20 [ 3108.265193] [] ? system_call_fastpath+0x16/0x1b [ 3108.265193] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 3108.265193] Call Trace: [ 3108.265193] [] ? new_inode+0x3b/0xc0 [ 3108.265193] [] ? create_write_pipe+0x35/0x1f0 [ 3108.265193] [] ? do_sigaction+0x1bb/0x200 [ 3108.265193] [] ? do_pipe_flags+0x57/0x130 [ 3108.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 3108.265193] [] ? sys_pipe2+0x24/0x80 [ 3108.265193] [] ? sys_pipe+0x10/0x20 [ 3108.265193] [] ? system_call_fastpath+0x16/0x1b [ 3108.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3108.076003] CPU 0 [ 3108.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3108.076003] [ 3108.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3108.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3108.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 3108.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 3108.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3108.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 3108.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3108.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 3108.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 3108.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3108.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 3108.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3108.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3108.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 3108.076003] Stack: [ 3108.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 3108.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 3108.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 3108.076003] Call Trace: [ 3108.076003] [] ? new_inode+0x3b/0xc0 [ 3108.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3108.076003] [] ? do_sigaction+0x1bb/0x200 [ 3108.076003] [] ? do_pipe_flags+0x57/0x130 [ 3108.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3108.076003] [] ? sys_pipe2+0x24/0x80 [ 3108.076003] [] ? sys_pipe+0x10/0x20 [ 3108.076003] [] ? system_call_fastpath+0x16/0x1b [ 3108.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3108.076003] Call Trace: [ 3108.076003] [] ? new_inode+0x3b/0xc0 [ 3108.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3108.076003] [] ? do_sigaction+0x1bb/0x200 [ 3108.076003] [] ? do_pipe_flags+0x57/0x130 [ 3108.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3108.076003] [] ? sys_pipe2+0x24/0x80 [ 3108.076003] [] ? sys_pipe+0x10/0x20 [ 3108.076003] [] ? system_call_fastpath+0x16/0x1b [ 3192.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 3192.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 3192.139005] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 3192.139007] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3192.139069] CPU 1 [ 3192.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3192.139069] [ 3192.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3192.139069] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3192.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 3192.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 3192.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3192.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 3192.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 3192.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 3192.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 3192.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3192.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 3192.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3192.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3192.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 3192.139069] Stack: [ 3192.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 3192.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 3192.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 3192.139069] Call Trace: [ 3192.139069] [] ? new_inode+0x3b/0xc0 [ 3192.139069] [] ? create_write_pipe+0x35/0x1f0 [ 3192.139069] [] ? do_sigaction+0x1bb/0x200 [ 3192.139069] [] ? do_pipe_flags+0x57/0x130 [ 3192.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 3192.139069] [] ? sys_pipe2+0x24/0x80 [ 3192.139069] [] ? sys_pipe+0x10/0x20 [ 3192.139069] [] ? system_call_fastpath+0x16/0x1b [ 3192.139069] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3192.139069] Call Trace: [ 3192.139069] [] ? new_inode+0x3b/0xc0 [ 3192.139069] [] ? create_write_pipe+0x35/0x1f0 [ 3192.139069] [] ? do_sigaction+0x1bb/0x200 [ 3192.139069] [] ? do_pipe_flags+0x57/0x130 [ 3192.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 3192.139069] [] ? sys_pipe2+0x24/0x80 [ 3192.139069] [] ? sys_pipe+0x10/0x20 [ 3192.139069] [] ? system_call_fastpath+0x16/0x1b [ 3192.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 3192.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3192.203096] CPU 2 [ 3192.203097] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3192.203131] [ 3192.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3192.203131] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 3192.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 3192.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 3192.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 3192.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 3192.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 3192.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 3192.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 3192.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 3192.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 3192.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3192.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3192.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 3192.203131] Stack: [ 3192.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 3192.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 3192.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 3192.203131] Call Trace: [ 3192.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3192.203131] [] ? iput+0x30/0x70 [ 3192.203131] [] ? dentry_iput+0xb8/0x140 [ 3192.203131] [] ? d_kill+0x3c/0x70 [ 3192.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3192.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3192.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3192.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3192.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3192.203131] [] ? d_alloc+0x45/0x280 [ 3192.203131] [] ? do_lookup+0x18d/0x210 [ 3192.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3192.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3192.203131] [] ? path_walk+0x6a/0xe0 [ 3192.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3192.203131] [] ? user_path_at+0x59/0xa0 [ 3192.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3192.203131] [] ? vfs_fstatat+0x46/0x80 [ 3192.203131] [] ? vfs_lstat+0x1e/0x20 [ 3192.203131] [] ? sys32_lstat64+0x24/0x40 [ 3192.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3192.203131] [] ? ia32_tracesys+0x48/0x8c [ 3192.203131] [] ? ia32_sysret+0x0/0x5 [ 3192.203131] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 3192.203131] Call Trace: [ 3192.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3192.203131] [] ? iput+0x30/0x70 [ 3192.203131] [] ? dentry_iput+0xb8/0x140 [ 3192.203131] [] ? d_kill+0x3c/0x70 [ 3192.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3192.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3192.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3192.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3192.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3192.203131] [] ? d_alloc+0x45/0x280 [ 3192.203131] [] ? do_lookup+0x18d/0x210 [ 3192.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3192.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3192.203131] [] ? path_walk+0x6a/0xe0 [ 3192.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3192.203131] [] ? user_path_at+0x59/0xa0 [ 3192.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3192.203131] [] ? vfs_fstatat+0x46/0x80 [ 3192.203131] [] ? vfs_lstat+0x1e/0x20 [ 3192.203131] [] ? sys32_lstat64+0x24/0x40 [ 3192.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3192.203131] [] ? ia32_tracesys+0x48/0x8c [ 3192.203131] [] ? ia32_sysret+0x0/0x5 [ 3192.265007] BUG: soft lockup - CPU#3 stuck for 68s! [cron:29387] [ 3192.265010] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3192.265123] CPU 3 [ 3192.265125] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3192.265192] [ 3192.265192] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3192.265192] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 3192.265192] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 3192.265192] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 3192.265192] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3192.265192] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 3192.265192] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3192.265192] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 3192.265192] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 3192.265192] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3192.265192] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 3192.265192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3192.265192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3192.265192] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 3192.265192] Stack: [ 3192.265192] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 3192.265192] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 3192.265192] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 3192.265192] Call Trace: [ 3192.265192] [] ? new_inode+0x3b/0xc0 [ 3192.265192] [] ? create_write_pipe+0x35/0x1f0 [ 3192.265192] [] ? do_sigaction+0x1bb/0x200 [ 3192.265192] [] ? do_pipe_flags+0x57/0x130 [ 3192.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 3192.265192] [] ? sys_pipe2+0x24/0x80 [ 3192.265192] [] ? sys_pipe+0x10/0x20 [ 3192.265192] [] ? system_call_fastpath+0x16/0x1b [ 3192.265192] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 3192.265192] Call Trace: [ 3192.265192] [] ? new_inode+0x3b/0xc0 [ 3192.265192] [] ? create_write_pipe+0x35/0x1f0 [ 3192.265192] [] ? do_sigaction+0x1bb/0x200 [ 3192.265192] [] ? do_pipe_flags+0x57/0x130 [ 3192.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 3192.265192] [] ? sys_pipe2+0x24/0x80 [ 3192.265192] [] ? sys_pipe+0x10/0x20 [ 3192.265192] [] ? system_call_fastpath+0x16/0x1b [ 3192.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3192.076003] CPU 0 [ 3192.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3192.076003] [ 3192.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3192.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3192.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 3192.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 3192.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3192.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 3192.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3192.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 3192.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 3192.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3192.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 3192.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3192.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3192.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 3192.076003] Stack: [ 3192.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 3192.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 3192.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 3192.076003] Call Trace: [ 3192.076003] [] ? new_inode+0x3b/0xc0 [ 3192.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3192.076003] [] ? do_sigaction+0x1bb/0x200 [ 3192.076003] [] ? do_pipe_flags+0x57/0x130 [ 3192.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3192.076003] [] ? sys_pipe2+0x24/0x80 [ 3192.076003] [] ? sys_pipe+0x10/0x20 [ 3192.076003] [] ? system_call_fastpath+0x16/0x1b [ 3192.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3192.076003] Call Trace: [ 3192.076003] [] ? new_inode+0x3b/0xc0 [ 3192.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3192.076003] [] ? do_sigaction+0x1bb/0x200 [ 3192.076003] [] ? do_pipe_flags+0x57/0x130 [ 3192.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3192.076003] [] ? sys_pipe2+0x24/0x80 [ 3192.076003] [] ? sys_pipe+0x10/0x20 [ 3192.076003] [] ? system_call_fastpath+0x16/0x1b [ 3276.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 3276.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 3276.139005] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 3276.139007] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3276.139070] CPU 1 [ 3276.139070] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3276.139070] [ 3276.139070] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3276.139070] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3276.139070] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 3276.139070] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 3276.139070] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3276.139070] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 3276.139070] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 3276.139070] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 3276.139070] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 3276.139070] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3276.139070] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 3276.139070] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3276.139070] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3276.139070] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 3276.139070] Stack: [ 3276.139070] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 3276.139070] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 3276.139070] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 3276.139070] Call Trace: [ 3276.139070] [] ? new_inode+0x3b/0xc0 [ 3276.139070] [] ? create_write_pipe+0x35/0x1f0 [ 3276.139070] [] ? do_sigaction+0x1bb/0x200 [ 3276.139070] [] ? do_pipe_flags+0x57/0x130 [ 3276.139070] [] ? audit_syscall_entry+0x286/0x2b0 [ 3276.139070] [] ? sys_pipe2+0x24/0x80 [ 3276.139070] [] ? sys_pipe+0x10/0x20 [ 3276.139070] [] ? system_call_fastpath+0x16/0x1b [ 3276.139070] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3276.139070] Call Trace: [ 3276.139070] [] ? new_inode+0x3b/0xc0 [ 3276.139070] [] ? create_write_pipe+0x35/0x1f0 [ 3276.139070] [] ? do_sigaction+0x1bb/0x200 [ 3276.139070] [] ? do_pipe_flags+0x57/0x130 [ 3276.139070] [] ? audit_syscall_entry+0x286/0x2b0 [ 3276.139070] [] ? sys_pipe2+0x24/0x80 [ 3276.139070] [] ? sys_pipe+0x10/0x20 [ 3276.139070] [] ? system_call_fastpath+0x16/0x1b [ 3276.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 3276.203010] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3276.203096] CPU 2 [ 3276.203097] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3276.203131] [ 3276.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3276.203131] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 3276.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 3276.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 3276.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 3276.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 3276.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 3276.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 3276.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 3276.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 3276.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 3276.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3276.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3276.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 3276.203131] Stack: [ 3276.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 3276.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 3276.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 3276.203131] Call Trace: [ 3276.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3276.203131] [] ? iput+0x30/0x70 [ 3276.203131] [] ? dentry_iput+0xb8/0x140 [ 3276.203131] [] ? d_kill+0x3c/0x70 [ 3276.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3276.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3276.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3276.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3276.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3276.203131] [] ? d_alloc+0x45/0x280 [ 3276.203131] [] ? do_lookup+0x18d/0x210 [ 3276.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3276.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3276.203131] [] ? path_walk+0x6a/0xe0 [ 3276.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3276.203131] [] ? user_path_at+0x59/0xa0 [ 3276.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3276.203131] [] ? vfs_fstatat+0x46/0x80 [ 3276.203131] [] ? vfs_lstat+0x1e/0x20 [ 3276.203131] [] ? sys32_lstat64+0x24/0x40 [ 3276.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3276.203131] [] ? ia32_tracesys+0x48/0x8c [ 3276.203131] [] ? ia32_sysret+0x0/0x5 [ 3276.203131] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 3276.203131] Call Trace: [ 3276.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3276.203131] [] ? iput+0x30/0x70 [ 3276.203131] [] ? dentry_iput+0xb8/0x140 [ 3276.203131] [] ? d_kill+0x3c/0x70 [ 3276.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3276.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3276.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3276.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3276.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3276.203131] [] ? d_alloc+0x45/0x280 [ 3276.203131] [] ? do_lookup+0x18d/0x210 [ 3276.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3276.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3276.203131] [] ? path_walk+0x6a/0xe0 [ 3276.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3276.203131] [] ? user_path_at+0x59/0xa0 [ 3276.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3276.203131] [] ? vfs_fstatat+0x46/0x80 [ 3276.203131] [] ? vfs_lstat+0x1e/0x20 [ 3276.203131] [] ? sys32_lstat64+0x24/0x40 [ 3276.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3276.203131] [] ? ia32_tracesys+0x48/0x8c [ 3276.203131] [] ? ia32_sysret+0x0/0x5 [ 3276.265006] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 3276.265010] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3276.265122] CPU 3 [ 3276.265124] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3276.265192] [ 3276.265192] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3276.265192] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3276.265192] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 3276.265192] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 3276.265192] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3276.265192] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 3276.265192] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3276.265192] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 3276.265192] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 3276.265192] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3276.265192] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 3276.265192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3276.265192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3276.265192] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 3276.265192] Stack: [ 3276.265192] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 3276.265192] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 3276.265192] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 3276.265192] Call Trace: [ 3276.265192] [] ? new_inode+0x3b/0xc0 [ 3276.265192] [] ? create_write_pipe+0x35/0x1f0 [ 3276.265192] [] ? do_sigaction+0x1bb/0x200 [ 3276.265192] [] ? do_pipe_flags+0x57/0x130 [ 3276.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 3276.265192] [] ? sys_pipe2+0x24/0x80 [ 3276.265192] [] ? sys_pipe+0x10/0x20 [ 3276.265192] [] ? system_call_fastpath+0x16/0x1b [ 3276.265192] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3276.265192] Call Trace: [ 3276.265192] [] ? new_inode+0x3b/0xc0 [ 3276.265192] [] ? create_write_pipe+0x35/0x1f0 [ 3276.265192] [] ? do_sigaction+0x1bb/0x200 [ 3276.265192] [] ? do_pipe_flags+0x57/0x130 [ 3276.265192] [] ? audit_syscall_entry+0x286/0x2b0 [ 3276.265192] [] ? sys_pipe2+0x24/0x80 [ 3276.265192] [] ? sys_pipe+0x10/0x20 [ 3276.265192] [] ? system_call_fastpath+0x16/0x1b [ 3276.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3276.076003] CPU 0 [ 3276.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3276.076003] [ 3276.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3276.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3276.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 3276.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 3276.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3276.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 3276.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3276.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 3276.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 3276.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3276.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 3276.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3276.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3276.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 3276.076003] Stack: [ 3276.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 3276.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 3276.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 3276.076003] Call Trace: [ 3276.076003] [] ? new_inode+0x3b/0xc0 [ 3276.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3276.076003] [] ? do_sigaction+0x1bb/0x200 [ 3276.076003] [] ? do_pipe_flags+0x57/0x130 [ 3276.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3276.076003] [] ? sys_pipe2+0x24/0x80 [ 3276.076003] [] ? sys_pipe+0x10/0x20 [ 3276.076003] [] ? system_call_fastpath+0x16/0x1b [ 3276.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3276.076003] Call Trace: [ 3276.076003] [] ? new_inode+0x3b/0xc0 [ 3276.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3276.076003] [] ? do_sigaction+0x1bb/0x200 [ 3276.076003] [] ? do_pipe_flags+0x57/0x130 [ 3276.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3276.076003] [] ? sys_pipe2+0x24/0x80 [ 3276.076003] [] ? sys_pipe+0x10/0x20 [ 3276.076003] [] ? system_call_fastpath+0x16/0x1b [ 3360.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 3360.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 3360.139005] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 3360.139008] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3360.139069] CPU 1 [ 3360.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3360.139069] [ 3360.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3360.139069] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3360.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 3360.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 3360.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3360.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 3360.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 3360.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 3360.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 3360.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3360.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 3360.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3360.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3360.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 3360.139069] Stack: [ 3360.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 3360.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 3360.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 3360.139069] Call Trace: [ 3360.139069] [] ? new_inode+0x3b/0xc0 [ 3360.139069] [] ? create_write_pipe+0x35/0x1f0 [ 3360.139069] [] ? do_sigaction+0x1bb/0x200 [ 3360.139069] [] ? do_pipe_flags+0x57/0x130 [ 3360.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 3360.139069] [] ? sys_pipe2+0x24/0x80 [ 3360.139069] [] ? sys_pipe+0x10/0x20 [ 3360.139069] [] ? system_call_fastpath+0x16/0x1b [ 3360.139069] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3360.139069] Call Trace: [ 3360.139069] [] ? new_inode+0x3b/0xc0 [ 3360.139069] [] ? create_write_pipe+0x35/0x1f0 [ 3360.139069] [] ? do_sigaction+0x1bb/0x200 [ 3360.139069] [] ? do_pipe_flags+0x57/0x130 [ 3360.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 3360.139069] [] ? sys_pipe2+0x24/0x80 [ 3360.139069] [] ? sys_pipe+0x10/0x20 [ 3360.139069] [] ? system_call_fastpath+0x16/0x1b [ 3360.203008] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 3360.203010] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3360.203096] CPU 2 [ 3360.203097] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3360.203131] [ 3360.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3360.203131] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3360.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 3360.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 3360.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 3360.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 3360.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 3360.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 3360.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 3360.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 3360.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 3360.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3360.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3360.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 3360.203131] Stack: [ 3360.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 3360.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 3360.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 3360.203131] Call Trace: [ 3360.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3360.203131] [] ? iput+0x30/0x70 [ 3360.203131] [] ? dentry_iput+0xb8/0x140 [ 3360.203131] [] ? d_kill+0x3c/0x70 [ 3360.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3360.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3360.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3360.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3360.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3360.203131] [] ? d_alloc+0x45/0x280 [ 3360.203131] [] ? do_lookup+0x18d/0x210 [ 3360.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3360.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3360.203131] [] ? path_walk+0x6a/0xe0 [ 3360.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3360.203131] [] ? user_path_at+0x59/0xa0 [ 3360.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3360.203131] [] ? vfs_fstatat+0x46/0x80 [ 3360.203131] [] ? vfs_lstat+0x1e/0x20 [ 3360.203131] [] ? sys32_lstat64+0x24/0x40 [ 3360.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3360.203131] [] ? ia32_tracesys+0x48/0x8c [ 3360.203131] [] ? ia32_sysret+0x0/0x5 [ 3360.203131] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3360.203131] Call Trace: [ 3360.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3360.203131] [] ? iput+0x30/0x70 [ 3360.203131] [] ? dentry_iput+0xb8/0x140 [ 3360.203131] [] ? d_kill+0x3c/0x70 [ 3360.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3360.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3360.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3360.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3360.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3360.203131] [] ? d_alloc+0x45/0x280 [ 3360.203131] [] ? do_lookup+0x18d/0x210 [ 3360.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3360.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3360.203131] [] ? path_walk+0x6a/0xe0 [ 3360.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3360.203131] [] ? user_path_at+0x59/0xa0 [ 3360.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3360.203131] [] ? vfs_fstatat+0x46/0x80 [ 3360.203131] [] ? vfs_lstat+0x1e/0x20 [ 3360.203131] [] ? sys32_lstat64+0x24/0x40 [ 3360.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3360.203131] [] ? ia32_tracesys+0x48/0x8c [ 3360.203131] [] ? ia32_sysret+0x0/0x5 [ 3360.265009] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 3360.265013] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3360.265134] CPU 3 [ 3360.265136] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3360.265193] [ 3360.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3360.265193] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3360.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 3360.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 3360.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3360.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 3360.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3360.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 3360.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 3360.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3360.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 3360.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3360.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3360.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 3360.265193] Stack: [ 3360.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 3360.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 3360.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 3360.265193] Call Trace: [ 3360.265193] [] ? new_inode+0x3b/0xc0 [ 3360.265193] [] ? create_write_pipe+0x35/0x1f0 [ 3360.265193] [] ? do_sigaction+0x1bb/0x200 [ 3360.265193] [] ? do_pipe_flags+0x57/0x130 [ 3360.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 3360.265193] [] ? sys_pipe2+0x24/0x80 [ 3360.265193] [] ? sys_pipe+0x10/0x20 [ 3360.265193] [] ? system_call_fastpath+0x16/0x1b [ 3360.265193] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3360.265193] Call Trace: [ 3360.265193] [] ? new_inode+0x3b/0xc0 [ 3360.265193] [] ? create_write_pipe+0x35/0x1f0 [ 3360.265193] [] ? do_sigaction+0x1bb/0x200 [ 3360.265193] [] ? do_pipe_flags+0x57/0x130 [ 3360.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 3360.265193] [] ? sys_pipe2+0x24/0x80 [ 3360.265193] [] ? sys_pipe+0x10/0x20 [ 3360.265193] [] ? system_call_fastpath+0x16/0x1b [ 3360.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3360.076003] CPU 0 [ 3360.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3360.076003] [ 3360.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3360.076003] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 3360.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 3360.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 3360.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3360.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 3360.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3360.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 3360.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 3360.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3360.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 3360.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3360.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3360.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 3360.076003] Stack: [ 3360.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 3360.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 3360.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 3360.076003] Call Trace: [ 3360.076003] [] ? new_inode+0x3b/0xc0 [ 3360.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3360.076003] [] ? do_sigaction+0x1bb/0x200 [ 3360.076003] [] ? do_pipe_flags+0x57/0x130 [ 3360.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3360.076003] [] ? sys_pipe2+0x24/0x80 [ 3360.076003] [] ? sys_pipe+0x10/0x20 [ 3360.076003] [] ? system_call_fastpath+0x16/0x1b [ 3360.076003] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 3360.076003] Call Trace: [ 3360.076003] [] ? new_inode+0x3b/0xc0 [ 3360.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3360.076003] [] ? do_sigaction+0x1bb/0x200 [ 3360.076003] [] ? do_pipe_flags+0x57/0x130 [ 3360.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3360.076003] [] ? sys_pipe2+0x24/0x80 [ 3360.076003] [] ? sys_pipe+0x10/0x20 [ 3360.076003] [] ? system_call_fastpath+0x16/0x1b [ 3444.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 3444.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 3444.139005] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 3444.139008] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3444.139069] CPU 1 [ 3444.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3444.139069] [ 3444.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3444.139069] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 3444.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 3444.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 3444.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3444.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 3444.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 3444.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 3444.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 3444.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3444.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 3444.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3444.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3444.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 3444.139069] Stack: [ 3444.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 3444.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 3444.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 3444.139069] Call Trace: [ 3444.139069] [] ? new_inode+0x3b/0xc0 [ 3444.139069] [] ? create_write_pipe+0x35/0x1f0 [ 3444.139069] [] ? do_sigaction+0x1bb/0x200 [ 3444.139069] [] ? do_pipe_flags+0x57/0x130 [ 3444.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 3444.139069] [] ? sys_pipe2+0x24/0x80 [ 3444.139069] [] ? sys_pipe+0x10/0x20 [ 3444.139069] [] ? system_call_fastpath+0x16/0x1b [ 3444.139069] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 3444.139069] Call Trace: [ 3444.139069] [] ? new_inode+0x3b/0xc0 [ 3444.139069] [] ? create_write_pipe+0x35/0x1f0 [ 3444.139069] [] ? do_sigaction+0x1bb/0x200 [ 3444.139069] [] ? do_pipe_flags+0x57/0x130 [ 3444.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 3444.139069] [] ? sys_pipe2+0x24/0x80 [ 3444.139069] [] ? sys_pipe+0x10/0x20 [ 3444.139069] [] ? system_call_fastpath+0x16/0x1b [ 3444.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 3444.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3444.203095] CPU 2 [ 3444.203097] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3444.203131] [ 3444.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3444.203131] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3444.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 3444.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 3444.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 3444.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 3444.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 3444.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 3444.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 3444.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 3444.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 3444.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3444.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3444.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 3444.203131] Stack: [ 3444.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 3444.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 3444.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 3444.203131] Call Trace: [ 3444.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3444.203131] [] ? iput+0x30/0x70 [ 3444.203131] [] ? dentry_iput+0xb8/0x140 [ 3444.203131] [] ? d_kill+0x3c/0x70 [ 3444.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3444.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3444.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3444.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3444.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3444.203131] [] ? d_alloc+0x45/0x280 [ 3444.203131] [] ? do_lookup+0x18d/0x210 [ 3444.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3444.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3444.203131] [] ? path_walk+0x6a/0xe0 [ 3444.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3444.203131] [] ? user_path_at+0x59/0xa0 [ 3444.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3444.203131] [] ? vfs_fstatat+0x46/0x80 [ 3444.203131] [] ? vfs_lstat+0x1e/0x20 [ 3444.203131] [] ? sys32_lstat64+0x24/0x40 [ 3444.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3444.203131] [] ? ia32_tracesys+0x48/0x8c [ 3444.203131] [] ? ia32_sysret+0x0/0x5 [ 3444.203131] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3444.203131] Call Trace: [ 3444.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3444.203131] [] ? iput+0x30/0x70 [ 3444.203131] [] ? dentry_iput+0xb8/0x140 [ 3444.203131] [] ? d_kill+0x3c/0x70 [ 3444.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3444.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3444.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3444.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3444.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3444.203131] [] ? d_alloc+0x45/0x280 [ 3444.203131] [] ? do_lookup+0x18d/0x210 [ 3444.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3444.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3444.203131] [] ? path_walk+0x6a/0xe0 [ 3444.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3444.203131] [] ? user_path_at+0x59/0xa0 [ 3444.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3444.203131] [] ? vfs_fstatat+0x46/0x80 [ 3444.203131] [] ? vfs_lstat+0x1e/0x20 [ 3444.203131] [] ? sys32_lstat64+0x24/0x40 [ 3444.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3444.203131] [] ? ia32_tracesys+0x48/0x8c [ 3444.203131] [] ? ia32_sysret+0x0/0x5 [ 3444.265011] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 3444.265015] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3444.265125] CPU 3 [ 3444.265127] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3444.265193] [ 3444.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3444.265193] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 3444.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 3444.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 3444.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3444.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 3444.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3444.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 3444.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 3444.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3444.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 3444.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3444.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3444.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 3444.265193] Stack: [ 3444.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 3444.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 3444.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 3444.265193] Call Trace: [ 3444.265193] [] ? new_inode+0x3b/0xc0 [ 3444.265193] [] ? create_write_pipe+0x35/0x1f0 [ 3444.265193] [] ? do_sigaction+0x1bb/0x200 [ 3444.265193] [] ? do_pipe_flags+0x57/0x130 [ 3444.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 3444.265193] [] ? sys_pipe2+0x24/0x80 [ 3444.265193] [] ? sys_pipe+0x10/0x20 [ 3444.265193] [] ? system_call_fastpath+0x16/0x1b [ 3444.265193] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 3444.265193] Call Trace: [ 3444.265193] [] ? new_inode+0x3b/0xc0 [ 3444.265193] [] ? create_write_pipe+0x35/0x1f0 [ 3444.265193] [] ? do_sigaction+0x1bb/0x200 [ 3444.265193] [] ? do_pipe_flags+0x57/0x130 [ 3444.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 3444.265193] [] ? sys_pipe2+0x24/0x80 [ 3444.265193] [] ? sys_pipe+0x10/0x20 [ 3444.265193] [] ? system_call_fastpath+0x16/0x1b [ 3444.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3444.076003] CPU 0 [ 3444.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3444.076003] [ 3444.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3444.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3444.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 3444.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 3444.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3444.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 3444.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3444.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 3444.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 3444.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3444.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 3444.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3444.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3444.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 3444.076003] Stack: [ 3444.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 3444.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 3444.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 3444.076003] Call Trace: [ 3444.076003] [] ? new_inode+0x3b/0xc0 [ 3444.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3444.076003] [] ? do_sigaction+0x1bb/0x200 [ 3444.076003] [] ? do_pipe_flags+0x57/0x130 [ 3444.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3444.076003] [] ? sys_pipe2+0x24/0x80 [ 3444.076003] [] ? sys_pipe+0x10/0x20 [ 3444.076003] [] ? system_call_fastpath+0x16/0x1b [ 3444.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3444.076003] Call Trace: [ 3444.076003] [] ? new_inode+0x3b/0xc0 [ 3444.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3444.076003] [] ? do_sigaction+0x1bb/0x200 [ 3444.076003] [] ? do_pipe_flags+0x57/0x130 [ 3444.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3444.076003] [] ? sys_pipe2+0x24/0x80 [ 3444.076003] [] ? sys_pipe+0x10/0x20 [ 3444.076003] [] ? system_call_fastpath+0x16/0x1b [ 3528.076003] BUG: soft lockup - CPU#0 stuck for 67s! [cron:29389] [ 3528.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 3528.139005] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 3528.139008] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3528.139069] CPU 1 [ 3528.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3528.139069] [ 3528.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3528.139069] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3528.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 3528.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 3528.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3528.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 3528.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 3528.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 3528.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 3528.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3528.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 3528.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3528.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3528.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 3528.139069] Stack: [ 3528.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 3528.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 3528.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 3528.139069] Call Trace: [ 3528.139069] [] ? new_inode+0x3b/0xc0 [ 3528.139069] [] ? create_write_pipe+0x35/0x1f0 [ 3528.139069] [] ? do_sigaction+0x1bb/0x200 [ 3528.139069] [] ? do_pipe_flags+0x57/0x130 [ 3528.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 3528.139069] [] ? sys_pipe2+0x24/0x80 [ 3528.139069] [] ? sys_pipe+0x10/0x20 [ 3528.139069] [] ? system_call_fastpath+0x16/0x1b [ 3528.139069] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3528.139069] Call Trace: [ 3528.139069] [] ? new_inode+0x3b/0xc0 [ 3528.139069] [] ? create_write_pipe+0x35/0x1f0 [ 3528.139069] [] ? do_sigaction+0x1bb/0x200 [ 3528.139069] [] ? do_pipe_flags+0x57/0x130 [ 3528.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 3528.139069] [] ? sys_pipe2+0x24/0x80 [ 3528.139069] [] ? sys_pipe+0x10/0x20 [ 3528.139069] [] ? system_call_fastpath+0x16/0x1b [ 3528.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 3528.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3528.203096] CPU 2 [ 3528.203097] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3528.203131] [ 3528.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3528.203131] RIP: 0010:[] [] _spin_lock+0x31/0x40 [ 3528.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 3528.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 3528.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 3528.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 3528.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 3528.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 3528.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 3528.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 3528.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 3528.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3528.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3528.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 3528.203131] Stack: [ 3528.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 3528.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 3528.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 3528.203131] Call Trace: [ 3528.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3528.203131] [] ? iput+0x30/0x70 [ 3528.203131] [] ? dentry_iput+0xb8/0x140 [ 3528.203131] [] ? d_kill+0x3c/0x70 [ 3528.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3528.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3528.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3528.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3528.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3528.203131] [] ? d_alloc+0x45/0x280 [ 3528.203131] [] ? do_lookup+0x18d/0x210 [ 3528.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3528.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3528.203131] [] ? path_walk+0x6a/0xe0 [ 3528.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3528.203131] [] ? user_path_at+0x59/0xa0 [ 3528.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3528.203131] [] ? vfs_fstatat+0x46/0x80 [ 3528.203131] [] ? vfs_lstat+0x1e/0x20 [ 3528.203131] [] ? sys32_lstat64+0x24/0x40 [ 3528.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3528.203131] [] ? ia32_tracesys+0x48/0x8c [ 3528.203131] [] ? ia32_sysret+0x0/0x5 [ 3528.203131] Code: 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 0f b7 17 f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 e5 0f 1f [ 3528.203131] Call Trace: [ 3528.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3528.203131] [] ? iput+0x30/0x70 [ 3528.203131] [] ? dentry_iput+0xb8/0x140 [ 3528.203131] [] ? d_kill+0x3c/0x70 [ 3528.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3528.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3528.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3528.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3528.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3528.203131] [] ? d_alloc+0x45/0x280 [ 3528.203131] [] ? do_lookup+0x18d/0x210 [ 3528.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3528.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3528.203131] [] ? path_walk+0x6a/0xe0 [ 3528.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3528.203131] [] ? user_path_at+0x59/0xa0 [ 3528.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3528.203131] [] ? vfs_fstatat+0x46/0x80 [ 3528.203131] [] ? vfs_lstat+0x1e/0x20 [ 3528.203131] [] ? sys32_lstat64+0x24/0x40 [ 3528.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3528.203131] [] ? ia32_tracesys+0x48/0x8c [ 3528.203131] [] ? ia32_sysret+0x0/0x5 [ 3528.265012] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 3528.265017] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3528.265129] CPU 3 [ 3528.265130] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3528.265193] [ 3528.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3528.265193] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3528.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 3528.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 3528.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3528.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 3528.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3528.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 3528.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 3528.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3528.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 3528.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3528.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3528.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 3528.265193] Stack: [ 3528.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 3528.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 3528.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 3528.265193] Call Trace: [ 3528.265193] [] ? new_inode+0x3b/0xc0 [ 3528.265193] [] ? create_write_pipe+0x35/0x1f0 [ 3528.265193] [] ? do_sigaction+0x1bb/0x200 [ 3528.265193] [] ? do_pipe_flags+0x57/0x130 [ 3528.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 3528.265193] [] ? sys_pipe2+0x24/0x80 [ 3528.265193] [] ? sys_pipe+0x10/0x20 [ 3528.265193] [] ? system_call_fastpath+0x16/0x1b [ 3528.265193] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3528.265193] Call Trace: [ 3528.265193] [] ? new_inode+0x3b/0xc0 [ 3528.265193] [] ? create_write_pipe+0x35/0x1f0 [ 3528.265193] [] ? do_sigaction+0x1bb/0x200 [ 3528.265193] [] ? do_pipe_flags+0x57/0x130 [ 3528.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 3528.265193] [] ? sys_pipe2+0x24/0x80 [ 3528.265193] [] ? sys_pipe+0x10/0x20 [ 3528.265193] [] ? system_call_fastpath+0x16/0x1b [ 3528.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3528.076003] CPU 0 [ 3528.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3528.076003] [ 3528.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3528.076003] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3528.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 3528.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 3528.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3528.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 3528.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3528.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 3528.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 3528.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3528.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 3528.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3528.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3528.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 3528.076003] Stack: [ 3528.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 3528.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 3528.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 3528.076003] Call Trace: [ 3528.076003] [] ? new_inode+0x3b/0xc0 [ 3528.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3528.076003] [] ? do_sigaction+0x1bb/0x200 [ 3528.076003] [] ? do_pipe_flags+0x57/0x130 [ 3528.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3528.076003] [] ? sys_pipe2+0x24/0x80 [ 3528.076003] [] ? sys_pipe+0x10/0x20 [ 3528.076003] [] ? system_call_fastpath+0x16/0x1b [ 3528.076003] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3528.076003] Call Trace: [ 3528.076003] [] ? new_inode+0x3b/0xc0 [ 3528.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3528.076003] [] ? do_sigaction+0x1bb/0x200 [ 3528.076003] [] ? do_pipe_flags+0x57/0x130 [ 3528.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3528.076003] [] ? sys_pipe2+0x24/0x80 [ 3528.076003] [] ? sys_pipe+0x10/0x20 [ 3528.076003] [] ? system_call_fastpath+0x16/0x1b [ 3612.076003] BUG: soft lockup - CPU#0 stuck for 68s! [cron:29389] [ 3612.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy [ 3612.139006] BUG: soft lockup - CPU#1 stuck for 67s! [cron:29388] [ 3612.139009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3612.139069] CPU 1 [ 3612.139069] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3612.139069] [ 3612.139069] Pid: 29388, comm: cron veid: 1035 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3612.139069] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 3612.139069] RSP: 0018:ffff88027210fe68 EFLAGS: 00000297 [ 3612.139069] RAX: 0000000000005a4d RBX: ffff88027210fe68 RCX: 0000000000000000 [ 3612.139069] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3612.139069] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000002 [ 3612.139069] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000008 [ 3612.139069] R13: ffff88027210e000 R14: ffff880427c27280 R15: 00007f0eabc38000 [ 3612.139069] FS: 00007f0eabc35770(0000) GS:ffff880028280000(0000) knlGS:0000000006452bb0 [ 3612.139069] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3612.139069] CR2: 00007f0eab16c370 CR3: 0000000423946000 CR4: 00000000000006e0 [ 3612.139069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3612.139069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3612.139069] Process cron (pid: 29388, veid: 1035, threadinfo ffff88027210e000, task ffff8800d33e2880) [ 3612.139069] Stack: [ 3612.139069] ffff88027210fe98 ffffffff811bbc3b ffff88027210fee8 0000000000000000 [ 3612.139069] <0> 0000000000000000 00007f0eabc356a0 ffff88027210fee8 ffffffff811a93c5 [ 3612.139069] <0> 0000000000000000 ffffffff817ae66e ffff88027210ff08 ffffffff8108711b [ 3612.139069] Call Trace: [ 3612.139069] [] ? new_inode+0x3b/0xc0 [ 3612.139069] [] ? create_write_pipe+0x35/0x1f0 [ 3612.139069] [] ? do_sigaction+0x1bb/0x200 [ 3612.139069] [] ? do_pipe_flags+0x57/0x130 [ 3612.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 3612.139069] [] ? sys_pipe2+0x24/0x80 [ 3612.139069] [] ? sys_pipe+0x10/0x20 [ 3612.139069] [] ? system_call_fastpath+0x16/0x1b [ 3612.139069] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 3612.139069] Call Trace: [ 3612.139069] [] ? new_inode+0x3b/0xc0 [ 3612.139069] [] ? create_write_pipe+0x35/0x1f0 [ 3612.139069] [] ? do_sigaction+0x1bb/0x200 [ 3612.139069] [] ? do_pipe_flags+0x57/0x130 [ 3612.139069] [] ? audit_syscall_entry+0x286/0x2b0 [ 3612.139069] [] ? sys_pipe2+0x24/0x80 [ 3612.139069] [] ? sys_pipe+0x10/0x20 [ 3612.139069] [] ? system_call_fastpath+0x16/0x1b [ 3612.203007] BUG: soft lockup - CPU#2 stuck for 67s! [cvs:29276] [ 3612.203009] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3612.203095] CPU 2 [ 3612.203096] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3612.203131] [ 3612.203131] Pid: 29276, comm: cvs veid: 224 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3612.203131] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 3612.203131] RSP: 0000:ffff88027202da78 EFLAGS: 00000297 [ 3612.203131] RAX: 0000000000005a4a RBX: ffff88027202da78 RCX: 0000000000000001 [ 3612.203131] RDX: 0000000000005a49 RSI: ffffffff823d2a60 RDI: ffffffff823d2a60 [ 3612.203131] RBP: ffffffff8100bd8e R08: ffff8803a9e71510 R09: 0000000000000005 [ 3612.203131] R10: ffff88041faee040 R11: 0000000009913c70 R12: ffffffff8128d74c [ 3612.203131] R13: ffff88027202da28 R14: ffffffff81507241 R15: ffff88027202daa8 [ 3612.203131] FS: 00007f27274ef6d0(0000) GS:ffff880230400000(0000) knlGS:00000000b3645b90 [ 3612.203131] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b [ 3612.203131] CR2: 00000000b75c6004 CR3: 0000000272294000 CR4: 00000000000006e0 [ 3612.203131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3612.203131] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3612.203131] Process cvs (pid: 29276, veid: 224, threadinfo ffff88027202c000, task ffff8800d3088200) [ 3612.203131] Stack: [ 3612.203131] ffff88027202daa8 ffffffff8127dd55 000000000000038f ffffffff823d2a60 [ 3612.203131] <0> ffff88041faee2a0 ffff880411cc25b0 ffff88027202dac8 ffffffff811ba1f0 [ 3612.203131] <0> ffff88027202db08 ffff8803a9e714b0 ffff88027202dae8 ffffffff811b6578 [ 3612.203131] Call Trace: [ 3612.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3612.203131] [] ? iput+0x30/0x70 [ 3612.203131] [] ? dentry_iput+0xb8/0x140 [ 3612.203131] [] ? d_kill+0x3c/0x70 [ 3612.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3612.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3612.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3612.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3612.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3612.203131] [] ? d_alloc+0x45/0x280 [ 3612.203131] [] ? do_lookup+0x18d/0x210 [ 3612.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3612.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3612.203131] [] ? path_walk+0x6a/0xe0 [ 3612.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3612.203131] [] ? user_path_at+0x59/0xa0 [ 3612.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3612.203131] [] ? vfs_fstatat+0x46/0x80 [ 3612.203131] [] ? vfs_lstat+0x1e/0x20 [ 3612.203131] [] ? sys32_lstat64+0x24/0x40 [ 3612.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3612.203131] [] ? ia32_tracesys+0x48/0x8c [ 3612.203131] [] ? ia32_sysret+0x0/0x5 [ 3612.203131] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 3612.203131] Call Trace: [ 3612.203131] [] ? _atomic_dec_and_lock+0x55/0x90 [ 3612.203131] [] ? iput+0x30/0x70 [ 3612.203131] [] ? dentry_iput+0xb8/0x140 [ 3612.203131] [] ? d_kill+0x3c/0x70 [ 3612.203131] [] ? prune_one_dentry+0xb1/0xd0 [ 3612.203131] [] ? __shrink_dcache_ub+0x1b9/0x1f0 [ 3612.203131] [] ? shrink_dcache_ub+0x29/0x70 [ 3612.203131] [] ? ub_dcache_shrink+0x3e/0x50 [ 3612.203131] [] ? ub_dcache_charge+0x96/0x120 [ 3612.203131] [] ? d_alloc+0x45/0x280 [ 3612.203131] [] ? do_lookup+0x18d/0x210 [ 3612.203131] [] ? __link_path_walk+0x89e/0x10c0 [ 3612.203131] [] ? vma_prio_tree_insert+0x30/0x50 [ 3612.203131] [] ? path_walk+0x6a/0xe0 [ 3612.203131] [] ? do_path_lookup+0x5b/0xa0 [ 3612.203131] [] ? user_path_at+0x59/0xa0 [ 3612.203131] [] ? __do_page_fault+0x264/0x4f0 [ 3612.203131] [] ? vfs_fstatat+0x46/0x80 [ 3612.203131] [] ? vfs_lstat+0x1e/0x20 [ 3612.203131] [] ? sys32_lstat64+0x24/0x40 [ 3612.203131] [] ? syscall_trace_enter+0x1c8/0x1f0 [ 3612.203131] [] ? ia32_tracesys+0x48/0x8c [ 3612.203131] [] ? ia32_sysret+0x0/0x5 [ 3612.265010] BUG: soft lockup - CPU#3 stuck for 67s! [cron:29387] [ 3612.265014] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3612.265126] CPU 3 [ 3612.265128] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3612.265193] [ 3612.265193] Pid: 29387, comm: cron veid: 1048 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3612.265193] RIP: 0010:[] [] _spin_lock+0x2e/0x40 [ 3612.265193] RSP: 0018:ffff880279ecfe68 EFLAGS: 00000293 [ 3612.265193] RAX: 0000000000005a4b RBX: ffff880279ecfe68 RCX: 0000000000000000 [ 3612.265193] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3612.265193] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 0000000000000007 [ 3612.265193] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3612.265193] R13: ffff880279ece000 R14: ffff880427c27280 R15: 00007f1de02e1000 [ 3612.265193] FS: 00007f1de02de770(0000) GS:ffff880230480000(0000) knlGS:00000000b76ffa40 [ 3612.265193] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3612.265193] CR2: 00007f1ddf815370 CR3: 00000002722f2000 CR4: 00000000000006e0 [ 3612.265193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3612.265193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3612.265193] Process cron (pid: 29387, veid: 1048, threadinfo ffff880279ece000, task ffff8800d33e3350) [ 3612.265193] Stack: [ 3612.265193] ffff880279ecfe98 ffffffff811bbc3b ffff880279ecfee8 0000000000000000 [ 3612.265193] <0> 0000000000000000 00007f1de02de6a0 ffff880279ecfee8 ffffffff811a93c5 [ 3612.265193] <0> 0000000000000000 ffffffff817ae66e ffff880279ecff08 ffffffff8108711b [ 3612.265193] Call Trace: [ 3612.265193] [] ? new_inode+0x3b/0xc0 [ 3612.265193] [] ? create_write_pipe+0x35/0x1f0 [ 3612.265193] [] ? do_sigaction+0x1bb/0x200 [ 3612.265193] [] ? do_pipe_flags+0x57/0x130 [ 3612.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 3612.265193] [] ? sys_pipe2+0x24/0x80 [ 3612.265193] [] ? sys_pipe+0x10/0x20 [ 3612.265193] [] ? system_call_fastpath+0x16/0x1b [ 3612.265193] Code: e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 48 89 [ 3612.265193] Call Trace: [ 3612.265193] [] ? new_inode+0x3b/0xc0 [ 3612.265193] [] ? create_write_pipe+0x35/0x1f0 [ 3612.265193] [] ? do_sigaction+0x1bb/0x200 [ 3612.265193] [] ? do_pipe_flags+0x57/0x130 [ 3612.265193] [] ? audit_syscall_entry+0x286/0x2b0 [ 3612.265193] [] ? sys_pipe2+0x24/0x80 [ 3612.265193] [] ? sys_pipe+0x10/0x20 [ 3612.265193] [] ? system_call_fastpath+0x16/0x1b [ 3612.076003] xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3612.076003] CPU 0 [ 3612.076003] Modules linked in: vzethdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache nfs_acl auth_rpcgss sunrpc vzdquota nf_nat_ftp vzevent vznetdev vzmon vzdev xt_sctp xt_realm xt_NOTRACK xt_dccp xt_CONNMARK xt_connmark xt_connbytes xt_CLASSIFY nfnetlink_queue nfnetlink_log xt_recent xt_DSCP nf_nat_irc xt_conntrack ipt_LOG nf_conntrack_irc nf_conntrack_ftp xt_hl xt_tcpmss xt_TCPMSS xt_dscp ipt_REDIRECT ipt_REJECT tun nf_conntrack_netbios_ns nf_conntrack_broadcast iptable_nat nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables xt_string xt_statistic xt_SECMARK xt_quota xt_policy xt_pkttype xt_physdev xt_NFQUEUE xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_CONNSECMARK xt_comment nfnetlink ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables arc4 ecb ppp_mppe ppp_deflate zlib_deflate ppp_async ppp_generic slhc crc_ccitt serio_raw k8temp amd64_edac_mod edac_core edac_mce_amd sg i2c_nforce2 i2c_core ext4 mbcache jbd2 raid1 sd_mod crc_t10dif sr_mod cdrom forcedeth pata_amd ata_generic pata_acpi sata_nv dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan] [ 3612.076003] [ 3612.076003] Pid: 29389, comm: cron veid: 1113 Not tainted 2.6.32-042stab055.7.debug #1 042stab055_7 empty empty/S2927 [ 3612.076003] RIP: 0010:[] [] _spin_lock+0x28/0x40 [ 3612.076003] RSP: 0018:ffff880279eafe68 EFLAGS: 00000293 [ 3612.076003] RAX: 0000000000005a4c RBX: ffff880279eafe68 RCX: 0000000000000000 [ 3612.076003] RDX: 0000000000005a49 RSI: ffffffff817a63a5 RDI: ffffffff823d2a60 [ 3612.076003] RBP: ffffffff8100bd8e R08: 0000000000000000 R09: 000000000000000d [ 3612.076003] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008 [ 3612.076003] R13: ffff880279eae000 R14: ffff880427c27280 R15: 00007fb3a16b6000 [ 3612.076003] FS: 00007fb3a16b27a0(0000) GS:ffff880028200000(0000) knlGS:0000000003de4bb0 [ 3612.076003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3612.076003] CR2: 00007fb3a0d29a5e CR3: 0000000279ff8000 CR4: 00000000000006f0 [ 3612.076003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3612.076003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 3612.076003] Process cron (pid: 29389, veid: 1113, threadinfo ffff880279eae000, task ffff8800d33e5390) [ 3612.076003] Stack: [ 3612.076003] ffff880279eafe98 ffffffff811bbc3b ffff880279eafee8 0000000000000000 [ 3612.076003] <0> 0000000000000000 000000000000006e ffff880279eafee8 ffffffff811a93c5 [ 3612.076003] <0> 0000000000000000 ffffffff817ae66e ffff880279eaff08 ffffffff8108711b [ 3612.076003] Call Trace: [ 3612.076003] [] ? new_inode+0x3b/0xc0 [ 3612.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3612.076003] [] ? do_sigaction+0x1bb/0x200 [ 3612.076003] [] ? do_pipe_flags+0x57/0x130 [ 3612.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3612.076003] [] ? sys_pipe2+0x24/0x80 [ 3612.076003] [] ? sys_pipe+0x10/0x20 [ 3612.076003] [] ? system_call_fastpath+0x16/0x1b [ 3612.076003] Code: ff c9 c3 55 48 89 e5 0f 1f 44 00 00 65 48 8b 04 25 c8 cb 00 00 83 80 44 e0 ff ff 01 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 <39> c2 74 0e f3 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f [ 3612.076003] Call Trace: [ 3612.076003] [] ? new_inode+0x3b/0xc0 [ 3612.076003] [] ? create_write_pipe+0x35/0x1f0 [ 3612.076003] [] ? do_sigaction+0x1bb/0x200 [ 3612.076003] [] ? do_pipe_flags+0x57/0x130 [ 3612.076003] [] ? audit_syscall_entry+0x286/0x2b0 [ 3612.076003] [] ? sys_pipe2+0x24/0x80 [ 3612.076003] [] ? sys_pipe+0x10/0x20 [ 3612.076003] [] ? system_call_fastpath+0x16/0x1b