[SOLVED] Upgraded to 4.14.2-1, libvirtd locks up trying to boot VMs
kode54
Status: Interested
Joined: 19 Nov 2017
Posts: 13
Reply Quote
As reported in the topic, I upgraded from 4.13.16-1 to 4.14.2-1, and now libvirtd won't start. I've since rolled back to the stable Arch kernel, but that doesn't have preempt, and my VM now stutters somewhat.

Do I need to supply inxi output again?

E: Disregard that, it appears I upgraded to a 4.14 kernel too soon. SPL and ZFS won't support 4.14 until a 0.7.4 release happens.
Back to top
ylluminate
Status: Curious
Joined: 02 Jan 2018
Posts: 7
Reply Quote
So what's the situation here? Is there a way to get ZFS(/SPL) running on Liquorix at this point? Really needing this on a specific workstation I'm running this on, and didn't want to get into hacking together a solution until I really understand the situation better.

So the `zfs-dkms` and `zfsutils-linux` are at version `0.6.5.5` right now. You're saying that we need to get to `0.7.4` until this is supported? Is there a back port or can we use the unstable `0.7.5` package yet in a proper way?

I was thinking that perhaps the proper solution may be to use this PPA (https://launchpad.net/~zfs-native/+archive/ubuntu/stable/+packages), but I'm not really seeing the proper way to use this in this context.
Back to top
ylluminate
Status: Curious
Joined: 02 Jan 2018
Posts: 7
Reply Quote
Still no thoughts on this?
Back to top
damentz
Status: Assistant
Joined: 09 Sep 2008
Posts: 1143
Reply Quote
It's probably best to ask these questions on the forum of the actual zfs project.
Back to top
ylluminate
Status: Curious
Joined: 02 Jan 2018
Posts: 7
Reply Quote
:: damentz wrote ::
It's probably best to ask these questions on the forum of the actual zfs project.


I guess the issues is that since it's already "done" at the ZFS level, then it's really a question of packaging right now...
Back to top
kode54
Status: Interested
Joined: 19 Nov 2017
Posts: 13
Reply Quote
And now, on Arch, with the spl-dkms and zfs-dkms 0.7.5 packages installed, the 4.14.18-2 kernel outright locks up the host machine when it starts my virtual machines. Sometimes it manages to make the desktop VM start the Tianocore logo, but otherwise, it just locks up. I've downgraded to 4.14.17-1.

E: Log seems to show the host is still running, but the passthrough of unused VGA and all USB devices, and the apparent disconnect of the network interface, leave me without any way of reaching the host or guests. Maybe this has something to do with the IBPB patches that made it in? I sure hope not.

E2: Now I have traced a bit more. The 4.14.18-2 kernel causes the virtual machine and/or libvirt to hang up, and dump a bug check to the syslog.

It produced a bug check on almost every boot, related to XHCI. It may have had something to do with having a bunch of USB 2.0 devices attached to a 2.0/3.0 bus powered hub.

:: Code ::
Feb 12 23:23:15 umaro kernel: BUG: unable to handle kernel NULL pointer dereference at 0000000000000008
Feb 12 23:23:15 umaro kernel: IP: xhci_mem_cleanup+0x1b1/0x800 [xhci_hcd]
Feb 12 23:23:15 umaro kernel: PGD 0 P4D 0
Feb 12 23:23:15 umaro kernel: Oops: 0002 [#1] PREEMPT SMP PTI
Feb 12 23:23:15 umaro kernel: Modules linked in: vfio_pci vfio_virqfd vfio_iommu_type1 vfio unix_diag xt_CHECKSUM iptable_mangle ipt_MA
SQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c crc
32c_generic ipt_REJECT nf_reject_ipv4 xt_tcpudp tun devlink ebtable_filter ebtables xt_multiport ip6table_filter ip6_tables iptable_fil
ter cmac bnep ipheth af_packet nls_iso8859_1 nls_cp437 vfat fat bridge stp llc sch_fq_codel uvcvideo videobuf2_vmalloc videobuf2_memops
 videobuf2_v4l2 videobuf2_core videodev media snd_usb_audio snd_usbmidi_lib snd_rawmidi snd_seq_device intel_rapl x86_pkg_temp_thermal
intel_powerclamp coretemp iTCO_wdt iTCO_vendor_support kvm_intel zfs(PO) joydev kvm input_leds irqbypass crct10dif_pclmul crc32_pclmul
ghash_clmulni_intel
Feb 12 23:23:15 umaro kernel:  pcbc zunicode(PO) zavl(PO) aesni_intel aes_x86_64 crypto_simd glue_helper icp(PO) btusb btrtl mxm_wmi ev
dev efi_pstore cryptd btbcm amdgpu snd_hda_intel btintel efivars pcspkr snd_hda_codec bluetooth i915 i2c_i801 ecdh_generic ttm snd_hda_
core snd_hwdep snd_pcm lpc_ich drm_kms_helper snd_timer xpad rfkill e1000e intel_gtt hid_generic snd drm soundcore ptp i2c_algo_bit pps
_core i2c_core shpchp ie31200_edac thermal fan rtc_cmos wmi video intel_smartconnect button acpi_cpufreq zcommon(PO) znvpair(PO) spl(O)
 crypto_user efivarfs ip_tables x_tables ipv6 crc_ccitt autofs4 ext4 crc16 mbcache jbd2 fscrypto sr_mod cdrom usbhid sd_mod ahci libahc
i firewire_ohci crc32c_intel libata firewire_core ehci_pci xhci_pci crc_itu_t scsi_mod ehci_hcd xhci_hcd pci_stub
Feb 12 23:23:15 umaro kernel: CPU: 3 PID: 1828 Comm: libvirtd Tainted: P           O    4.14.18-2-lqx #1
Feb 12 23:23:15 umaro kernel: Hardware name: MSI MS-7751/Z77A-GD65 (MS-7751), BIOS V10.11 10/09/2013
Feb 12 23:23:15 umaro kernel: task: ffff8802021a3500 task.stack: ffffc90003de8000
Feb 12 23:23:15 umaro kernel: RIP: 0010:xhci_mem_cleanup+0x1b1/0x800 [xhci_hcd]
Feb 12 23:23:15 umaro kernel: RSP: 0018:ffffc90003debcb8 EFLAGS: 00010202
Feb 12 23:23:15 umaro kernel: RAX: ffff8802070da4d0 RBX: 0000000000000008 RCX: 0000000000000000
Feb 12 23:23:15 umaro kernel: RDX: 0000000000000000 RSI: ffff88020905e8b0 RDI: ffff88020905e8b0
Feb 12 23:23:15 umaro kernel: RBP: ffff88080b2bf0a0 R08: ffff88020905eab8 R09: 0000000000000810
Feb 12 23:23:15 umaro kernel: R10: 00000000000012d0 R11: 0000000000000000 R12: ffff880209058238
Feb 12 23:23:15 umaro kernel: R13: 0000000008000820 R14: ffff8802090583a0 R15: ffff880209058284
Feb 12 23:23:15 umaro kernel: FS:  00007fd18960e700(0000) GS:ffff88082f2c0000(0000) knlGS:0000000000000000
Feb 12 23:23:15 umaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Feb 12 23:23:15 umaro kernel: CR2: 0000000000000008 CR3: 0000000200f60002 CR4: 00000000001626e0
Feb 12 23:23:15 umaro kernel: Call Trace:
Feb 12 23:23:15 umaro kernel:  xhci_stop+0x22a/0x2f0 [xhci_hcd]
Feb 12 23:23:15 umaro kernel:  usb_remove_hcd+0xdd/0x230
Feb 12 23:23:15 umaro kernel:  usb_hcd_pci_remove+0x6a/0x1a0
Feb 12 23:23:15 umaro kernel:  pci_device_remove+0x36/0xb0
Feb 12 23:23:15 umaro kernel:  device_release_driver+0x123/0x1f0
Feb 12 23:23:15 umaro kernel:  unbind_store+0x14b/0x180
Feb 12 23:23:15 umaro kernel:  kernfs_fop_write+0x106/0x190
Feb 12 23:23:15 umaro kernel:  vfs_write.part.5+0x5c/0x270
Feb 12 23:23:15 umaro kernel:  SyS_write+0xc2/0x1a0
Feb 12 23:23:15 umaro kernel:  do_syscall_64+0x79/0x1a0
Feb 12 23:23:15 umaro kernel:  entry_SYSCALL_64_after_hwframe+0x21/0x86
Feb 12 23:23:15 umaro kernel: RIP: 0033:0x7fd191d388eb
Feb 12 23:23:15 umaro kernel: RSP: 002b:00007fd18960d670 EFLAGS: 00000293 ORIG_RAX: 0000000000000001
Feb 12 23:23:15 umaro kernel: RAX: ffffffffffffffda RBX: 00007fd1780135a4 RCX: 00007fd191d388eb
Feb 12 23:23:15 umaro kernel: RDX: 000000000000000c RSI: 00007fd1780135a4 RDI: 0000000000000016
Feb 12 23:23:15 umaro kernel: RBP: 000000000000000c R08: 0000000000000000 R09: 00007fd194c65108
Feb 12 23:23:15 umaro kernel: R10: 0000000000000000 R11: 0000000000000293 R12: 00007fd1780135a4
Feb 12 23:23:15 umaro kernel: R13: 0000000000000000 R14: 0000000000000016 R15: 00007fd194c6535b
Feb 12 23:23:15 umaro kernel: Code: 44 8d 53 ff 45 31 c9 4d 69 d2 b0 02 00 00 4c 01 c8 48 8d 70 28 4c 8d 80 a8 02 00 00 48 89 f7 eb 18
48 8b 07 48 8b 08 48 8b 50 08 <48> 89 51 08 48 89 0a 48 89 00 48 89 40 08 48 8b 06 48 39 c6 75
Feb 12 23:23:15 umaro kernel: RIP: xhci_mem_cleanup+0x1b1/0x800 [xhci_hcd] RSP: ffffc90003debcb8
Feb 12 23:23:15 umaro kernel: CR2: 0000000000000008
Feb 12 23:23:15 umaro kernel: ---[ end trace 12c737b6faba287e ]---


Then it did this:

:: Code ::

Feb 12 23:23:23 umaro kernel: BUG: scheduling while atomic: CPU 0/KVM/2161/0x00000000
Feb 12 23:23:23 umaro kernel: Modules linked in: vfio_pci vfio_virqfd vfio_iommu_type1 vfio unix_diag xt_CHECKSUM iptable_mangle ipt_MA
SQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c crc
32c_generic ipt_REJECT nf_reject_ipv4 xt_tcpudp tun devlink ebtable_filter ebtables xt_multiport ip6table_filter ip6_tables iptable_fil
ter cmac bnep ipheth af_packet nls_iso8859_1 nls_cp437 vfat fat bridge stp llc sch_fq_codel uvcvideo videobuf2_vmalloc videobuf2_memops
 videobuf2_v4l2 videobuf2_core videodev media snd_usb_audio snd_usbmidi_lib snd_rawmidi snd_seq_device intel_rapl x86_pkg_temp_thermal
intel_powerclamp coretemp iTCO_wdt iTCO_vendor_support kvm_intel zfs(PO) joydev kvm input_leds irqbypass crct10dif_pclmul crc32_pclmul
ghash_clmulni_intel
Feb 12 23:23:23 umaro kernel:  pcbc zunicode(PO) zavl(PO) aesni_intel aes_x86_64 crypto_simd glue_helper icp(PO) btusb btrtl mxm_wmi ev
dev efi_pstore cryptd btbcm amdgpu snd_hda_intel btintel efivars pcspkr snd_hda_codec bluetooth i915 i2c_i801 ecdh_generic ttm snd_hda_
core snd_hwdep snd_pcm lpc_ich drm_kms_helper snd_timer xpad rfkill e1000e intel_gtt hid_generic snd drm soundcore ptp i2c_algo_bit pps
_core i2c_core shpchp ie31200_edac thermal fan rtc_cmos wmi video intel_smartconnect button acpi_cpufreq zcommon(PO) znvpair(PO) spl(O)
 crypto_user efivarfs ip_tables x_tables ipv6 crc_ccitt autofs4 ext4 crc16 mbcache jbd2 fscrypto sr_mod cdrom usbhid sd_mod ahci libahc
i firewire_ohci crc32c_intel libata firewire_core ehci_pci xhci_pci crc_itu_t scsi_mod ehci_hcd xhci_hcd pci_stub
Feb 12 23:23:23 umaro kernel: CPU: 3 PID: 2161 Comm: CPU 0/KVM Tainted: P      D    O    4.14.18-2-lqx #1
Feb 12 23:23:23 umaro kernel: Hardware name: MSI MS-7751/Z77A-GD65 (MS-7751), BIOS V10.11 10/09/2013
Feb 12 23:23:23 umaro kernel: Call Trace:
Feb 12 23:23:23 umaro kernel:  dump_stack+0x5c/0x7e
Feb 12 23:23:23 umaro kernel:  __schedule_bug+0x43/0x50
Feb 12 23:23:23 umaro kernel:  __schedule+0x1016/0x14c0
Feb 12 23:23:23 umaro kernel:  ? vmx_vcpu_load+0x18b/0x310 [kvm_intel]
Feb 12 23:23:23 umaro kernel:  ? kvm_apic_set_irq+0x16f/0x320 [kvm]
Feb 12 23:23:23 umaro kernel:  schedule+0x2a/0xc0
Feb 12 23:23:23 umaro kernel:  yield_to+0x19d/0x270
Feb 12 23:23:23 umaro kernel:  kvm_vcpu_on_spin+0x153/0x240 [kvm]
Feb 12 23:23:23 umaro kernel:  kvm_hv_hypercall+0x403/0x450 [kvm]
Feb 12 23:23:23 umaro kernel:  ? finish_task_switch+0x189/0x380
Feb 12 23:23:23 umaro kernel:  ? finish_task_switch+0x189/0x380
Feb 12 23:23:23 umaro kernel:  kvm_emulate_hypercall+0x2f7/0x540 [kvm]
Feb 12 23:23:23 umaro kernel:  ? __schedule+0x75e/0x14c0
Feb 12 23:23:23 umaro kernel:  ? vmx_handle_exit+0x9a/0x1590 [kvm_intel]
Feb 12 23:23:23 umaro kernel:  ? handle_dr+0x1f0/0x1f0 [kvm_intel]
Feb 12 23:23:23 umaro kernel:  vmx_handle_exit+0x9a/0x1590 [kvm_intel]
Feb 12 23:23:23 umaro kernel:  ? __fill_rsb+0x15/0x40
Feb 12 23:23:23 umaro kernel:  ? __fill_rsb+0x25/0x40
Feb 12 23:23:23 umaro kernel:  ? __fill_rsb+0x15/0x40
Feb 12 23:23:23 umaro kernel:  ? vmx_vcpu_run+0x3a8/0x570 [kvm_intel]
Feb 12 23:23:23 umaro kernel:  kvm_arch_vcpu_ioctl_run+0x8e2/0x19f0 [kvm]
Feb 12 23:23:23 umaro kernel:  ? kvm_vcpu_ioctl+0x2d3/0x660 [kvm]
Feb 12 23:23:23 umaro kernel:  kvm_vcpu_ioctl+0x2d3/0x660 [kvm]
Feb 12 23:23:23 umaro kernel:  SyS_ioctl+0xd8/0x8b0
Feb 12 23:23:23 umaro kernel:  ? __do_page_fault+0x285/0x550
Feb 12 23:23:23 umaro kernel:  ? kvm_on_user_return+0x63/0xa0 [kvm]
Feb 12 23:23:23 umaro kernel:  do_syscall_64+0x79/0x1a0
Feb 12 23:23:23 umaro kernel:  entry_SYSCALL_64_after_hwframe+0x21/0x86
Feb 12 23:23:23 umaro kernel: RIP: 0033:0x7f3112585d87
Feb 12 23:23:23 umaro kernel: RSP: 002b:00007f3105117118 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Feb 12 23:23:23 umaro kernel: RAX: ffffffffffffffda RBX: 000000000000ae80 RCX: 00007f3112585d87
Feb 12 23:23:23 umaro kernel: RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000017
Feb 12 23:23:23 umaro kernel: RBP: 00007f3107581ac0 R08: 000055fe693fd970 R09: 0000000000000014
Feb 12 23:23:23 umaro kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
Feb 12 23:23:23 umaro kernel: R13: 00007f31195db000 R14: 0000000000000000 R15: 00007f3107581ac0

Back to top
kode54
Status: Interested
Joined: 19 Nov 2017
Posts: 13
Reply Quote
This Debian bug report seems relevant to the issue at hand, and also seems to suggest that I should wait until Liquorix makes its way to 4.15 before I perform any further updates.

bugs.debian.org/cgi-bin/bugreport.cgi?bug=885166
Back to top
ylluminate
Status: Curious
Joined: 02 Jan 2018
Posts: 7
Reply Quote
It seems that perhaps https://launchpad.net/~jonathonf/+archive/ubuntu/zfs solves the issue...

EDIT: So yes, after some days now of running this, ZFS is great and stable with Liquorix and this ZFS PPA. Things are smooth and performance is great.

< Edited by ylluminate :: Feb 19, 18, 16:35 >

Back to top
kode54
Status: Interested
Joined: 19 Nov 2017
Posts: 13
Reply Quote
I also didn't realize that my distribution had a nearly identical linux-zen package in the official binary repository, which also keeps up with the mainline kernel, somewhat. Makes me wonder what the difference is here.
Back to top
Display posts from previous:   

All times are GMT - 8 Hours