Uploaded image for project: 'OpenVZ'
  1. OpenVZ
  2. OVZ-6811

Kernel crash on ixgbe_xmit_frame

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Fix Version/s: OpenVZ-legacy
    • Component/s: Containers::Kernel
    • Security Level: Public
    • Environment:
      Hardware: Intel R2208WTTR (Motherboard: S2600WTTR)
      Network interface: Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)

      Software: Scientific Linux 6.8 x86_64
      Kernel: 2.6.32-042stab117.16

      Description

      >Description of problem:
      Random reboot on the server (it may possible to trigger with a specially crafted packet but this not yet confirmed)

      >How reproducible:
      Not yet confirmed to reproducible

      >Steps to Reproduce:
      1.
      2.
      3.

      >Actual results:

      <1>[ 7282.072723] BUG: unable to handle kernel paging request at ffff880972b46c0c
      <1>[ 7282.073140] IP: [<ffffffffa01d7862>] ixgbe_xmit_frame_ring+0x2d2/0xe90 [ixgbe]
      <4>[ 7282.073658] PGD 1a8e063 PUD 0
      <4>[ 7282.074040] Oops: 0000 [#1] SMP
      <4>[ 7282.074421] last sysfs file: /sys/devices/system/cpu/online
      <4>[ 7282.074682] CPU 0
      <4>[ 7282.074753] Modules linked in: vzethdev pio_kaio pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst vzcpt nfs lockd fscache auth_rpcgss nfs_acl sunrpc vziolimit vzdquota xt_owner xt_mac ipt_REDIRECT xt_state xt_length xt_hl xt_tcpmss xt_TCPMSS xt_DSCP xt_dscp xt_limit vzevent xt_recent iptable_nat xt_comment xt_AUDIT ipt_REJECT ipt_addrtype xt_MARK iptable_mangle xt_helper iptable_raw xt_multiport xt_conntrack ipt_ULOG xt_NFLOG nfnetlink_log ipt_LOG nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ts_kmp nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_proto_udplite nf_conntrack_proto_sctp nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink nfnetlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nf_conntrack iptable_filter ip_tables tun vznetdev vzmon vzdev ipv6 ipmi_devintf iTCO_wdt iTCO_vendor_support power_meter acpi_ipmi ipmi_si ipmi_msghandler joydev sb_edac edac_core raid10 lpc_ich mfd_core i2c_i801 i2c_core sg ixgbe dca ptp pps_core mdio ext4 jbd2 mbcache raid1 sd_mod crc_t10dif mpt3sas scsi_transport_sas raid_class xhci_hcd ahci wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
      <4>[ 7282.086520]
      <4>[ 7282.086766] Pid: 0, comm: swapper veid: 0 Not tainted 2.6.32-042stab117.16 #1 042stab117_16 Intel Corporation S2600WTTR/S2600WTTR
      <4>[ 7282.087417] RIP: 0010:[<ffffffffa01d7862>] [<ffffffffa01d7862>] ixgbe_xmit_frame_ring+0x2d2/0xe90 [ixgbe]
      <4>[ 7282.087936] RSP: 0018:ffff88004a203670 EFLAGS: 00010286
      <4>[ 7282.088196] RAX: 0000000000000001 RBX: ffffc9001703b2b0 RCX: ffff880872b46c60
      <4>[ 7282.088461] RDX: ffff880872b46c00 RSI: ffff88086db6d000 RDI: ffff880972b46bff
      <4>[ 7282.088727] RBP: ffff88004a2036f0 R08: ffff880878ca0860 R09: ffff880871c60436
      <4>[ 7282.088992] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88086db6d380
      <4>[ 7282.089258] R13: ffff88033fe9a1c0 R14: ffff88033fe9a1c0 R15: 0000000000000000
      <4>[ 7282.089525] FS: 0000000000000000(0000) GS:ffff88004a200000(0000) knlGS:0000000000000000
      <4>[ 7282.089969] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
      <4>[ 7282.090229] CR2: ffff880972b46c0c CR3: 0000000001a8d000 CR4: 00000000001407f0
      <4>[ 7282.090494] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      <4>[ 7282.090761] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
      <4>[ 7282.091028] Process swapper (pid: 0, veid: 0, threadinfo ffffffff81a00000, task ffffffff81a95020)
      <4>[ 7282.091476] Stack:
      <4>[ 7282.091723] ffff880872b46c52 0000000000000046 0000000000000004 0000000000000246
      <4>[ 7282.092181] <d> 000000005804c2b4 000000003afbfe0c ffff88004a2036c0 ffffffff8147ab7e
      <4>[ 7282.092944] <d> ffff88033fe9a1c0 00ff88087326e6e8 ffff880876352a80 ffff88033fe9a1c0
      <4>[ 7282.093945] Call Trace:
      <4>[ 7282.094194] <IRQ>
      <4>[ 7282.094510] [<ffffffff8147ab7e>] ? __skb_clone+0x2e/0x170
      <4>[ 7282.094776] [<ffffffffa01d8463>] ixgbe_xmit_frame+0x43/0x90 [ixgbe]
      <4>[ 7282.095041] [<ffffffff8148f58a>] dev_hard_start_xmit+0x11a/0x500
      <4>[ 7282.095305] [<ffffffff81485164>] ? skb_tx_hash+0x14/0x20
      <4>[ 7282.095565] [<ffffffff814852c8>] ? __netdev_pick_tx+0x158/0x1b0
      <4>[ 7282.095844] [<ffffffff814ad0ba>] sch_direct_xmit+0x15a/0x1c0
      <4>[ 7282.096106] [<ffffffff8148fbe8>] dev_queue_xmit+0x278/0x370
      <4>[ 7282.096366] [<ffffffff814cc704>] ip_finish_output+0x184/0x340
      <4>[ 7282.096627] [<ffffffff814cc978>] ip_output+0xb8/0xc0
      <4>[ 7282.096885] [<ffffffff814cbbff>] ? __ip_local_out+0x9f/0xb0
      <4>[ 7282.097144] [<ffffffff814cbc35>] ip_local_out+0x25/0x30
      <4>[ 7282.097404] [<ffffffffa0465468>] reject_tg+0x3e8/0x3f0 [ipt_REJECT]
      <4>[ 7282.097667] [<ffffffffa0461132>] ? addrtype_mt_v1+0xb2/0x110 [ipt_addrtype]
      <4>[ 7282.097935] [<ffffffffa035534b>] ipt_do_table+0x3fb/0x6b0 [ip_tables]
      <4>[ 7282.098202] [<ffffffff81068c91>] ? enqueue_boosted_entity+0x41/0xc0
      <4>[ 7282.098466] [<ffffffffa0355215>] ? ipt_do_table+0x2c5/0x6b0 [ip_tables]
      <4>[ 7282.098735] [<ffffffffa035b163>] ipt_local_in_hook+0x23/0x28 [iptable_filter]
      <4>[ 7282.099178] [<ffffffff814baed9>] nf_iterate+0x69/0xb0
      <4>[ 7282.099438] [<ffffffff814c6410>] ? ip_local_deliver_finish+0x0/0x310
      <4>[ 7282.099701] [<ffffffff814bb096>] nf_hook_slow+0x76/0x120
      <4>[ 7282.099961] [<ffffffff814c6410>] ? ip_local_deliver_finish+0x0/0x310
      <4>[ 7282.100227] [<ffffffff814c677a>] ip_local_deliver+0x5a/0xa0
      <4>[ 7282.100487] [<ffffffff814c5c2d>] ip_rcv_finish+0x12d/0x440
      <4>[ 7282.100747] [<ffffffff814c61c4>] ip_rcv+0x284/0x360
      <4>[ 7282.101006] [<ffffffff81489168>] __netif_receive_skb+0x248/0x5f0
      <4>[ 7282.101270] [<ffffffff8148d138>] netif_receive_skb+0x58/0x60
      <4>[ 7282.101531] [<ffffffff8148d240>] napi_skb_finish+0x50/0x70
      <4>[ 7282.101793] [<ffffffff8148ead9>] napi_gro_receive+0x39/0x50
      <4>[ 7282.102058] [<ffffffffa01d2ca7>] ixgbe_clean_rx_irq+0x507/0xc10 [ixgbe]
      <4>[ 7282.102328] [<ffffffffa01d387c>] ixgbe_poll+0x39c/0x790 [ixgbe]
      <4>[ 7282.102594] [<ffffffff81015189>] ? read_tsc+0x9/0x10
      <4>[ 7282.102854] [<ffffffff810bd3df>] ? ktime_get+0x6f/0x100
      <4>[ 7282.103114] [<ffffffff810bd3df>] ? ktime_get+0x6f/0x100
      <4>[ 7282.103375] [<ffffffff8148ec00>] net_rx_action+0x110/0x310
      <4>[ 7282.103636] [<ffffffff8108cb65>] __do_softirq+0x125/0x2a0
      <4>[ 7282.103896] [<ffffffff8100c4cc>] call_softirq+0x1c/0x30
      <4>[ 7282.104156] [<ffffffff810103c5>] do_softirq+0x65/0xa0
      <4>[ 7282.104415] [<ffffffff8108c96d>] irq_exit+0xcd/0xd0
      <4>[ 7282.104674] [<ffffffff8155d385>] do_IRQ+0x75/0xf0
      <4>[ 7282.104931] [<ffffffff8100bb13>] ret_from_intr+0x0/0x11
      <4>[ 7282.105191] <EOI>
      <4>[ 7282.105504] [<ffffffff8130beae>] ? intel_idle+0xfe/0x1b0
      <4>[ 7282.105779] [<ffffffff8130be91>] ? intel_idle+0xe1/0x1b0
      <4>[ 7282.106038] [<ffffffff81015139>] ? sched_clock+0x9/0x10
      <4>[ 7282.106297] [<ffffffff810b888d>] ? sched_clock_cpu+0xcd/0x110
      <4>[ 7282.106559] [<ffffffff81450d2a>] cpuidle_idle_call+0x7a/0xe0
      <4>[ 7282.106821] [<ffffffff8100a046>] cpu_idle+0xb6/0x110
      <4>[ 7282.107081] [<ffffffff81539725>] rest_init+0x85/0x90
      <4>[ 7282.107339] [<ffffffff81c48109>] start_kernel+0x406/0x413
      <4>[ 7282.107599] [<ffffffff81c4733a>] x86_64_start_reservations+0x125/0x129
      <4>[ 7282.107862] [<ffffffff81c47453>] x86_64_start_kernel+0x115/0x124
      <4>[ 7282.108121] Code: dd 0f 84 e2 06 00 00 66 83 ff 08 0f 85 cc 00 00 00 80 79 09 06 0f 85 c2 00 00 00 41 8b be c4 00 00 00 48 01 d7 0f 84 b2 00 00 00 <0f> b6 57 0d f6 c2 01 0f 85 a5 00 00 00 80 e2 02 75 0b 41 3a 44
      <1>[ 7282.112927] RIP [<ffffffffa01d7862>] ixgbe_xmit_frame_ring+0x2d2/0xe90 [ixgbe]
      <4>[ 7282.113439] RSP <ffff88004a203670>
      <4>[ 7282.113691] CR2: ffff880972b46c0c

      >Expected results:

      Continue to work

      >Host OS:

      Scientific Linux 6.8 x86_64

      >Guest OS:

      No guest OS running on the server

      >Additional info (see https://openvz.org/Reporting_OpenVZ_problem):

      Related reports:
      https://www.centos.org/forums/viewtopic.php?t=58879
      https://access.redhat.com/solutions/2374831
      https://patchwork.ozlabs.org/patch/208450/
      https://github.com/MISL-EBU-System-SW/mainline-public/commit/c6f408996c625cb950cad024f90e50519f94713c

        Attachments

          Activity

            People

            Assignee:
            vvs Vasily Averin
            Reporter:
            kp-tech OpenVZ Admin
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: