Kernels built since 2024.10.05 unable to start a graphical session in VMware Workstation

[aronowski]

How to file a helpful issue

Qubes OS release

4.2.3

Brief summary

Kernel regressions, which got introduced to all the supported kernel releases (kernel-515, kernel and kernel-latest) as early as of 2024.10.05 result in the inability to boot Qubes OS into a graphical session on VMware Workstation due to either:

• no graphical output at all (not even console messages being visible) when booting in UEFI mode
• infinite loop when booting with nomodeset at the part when LightDM attempts to start
• a reboot before reaching Plymouth.

As far as I'm aware, VMware Workstation is the only hypervisor, where as of today Qubes OS can be installed directly and all its features work out-of-the-box, like HVM qubes, without the need of manual tweaks, like changing their virtualization mode to PV after installation. This issue results in at best just some annoyances when working with Qubes OS in such laboratory setting, and at worst prevents crucial features from operating. For example, testing the upgrade process from Qubes OS 4.1 to 4.2 would be problematic without manual workarounds.

Steps to reproduce

Install Qubes OS 4.2.3 inside VMware Workstation 17.6.1, so the kernel 6.6.48 gets installed. After a successful installation, install the additional kernels from the timeframe 2024.09.14 - 2024.11.02 of their buildtimes:

[user@dom0 ~]$ sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing -y kernel-515-5.15.167 kernel-515-5.15.170 kernel-6.6.54 kernel-latest-6.10.10 kernel-latest-6.11.2

Also, attaching the listing for their buildtimes:

[user@dom0 ~]$ rpm -qa --qf '%{VERSION}: %{BUILDTIME:date}\n' kernel-515 kernel kernel-latest | sort -V
5.15.167: Sat 14 Sep 2024 02:00:00 AM CEST
5.15.170: Sat 02 Nov 2024 01:00:00 AM CET
6.6.48: Tue 03 Sep 2024 02:00:00 AM CEST
6.6.54: Sat 05 Oct 2024 02:00:00 AM CEST
6.10.10: Mon 16 Sep 2024 02:00:00 AM CEST
6.11.2: Sat 05 Oct 2024 02:00:00 AM CEST

Reboot the system and try booting each of the kernels. If there is no output, add nomodeset to the kernel commandline or a virtual serial port, e.g. to output the messages to a file in the host filesystem. Witness the following results, sorted here chronologically:

• (2024.09.03) kernel 6.6.48 - works fine
• (2024.09.14) kernel 5.15.167 - works fine
• (2024.09.16) kernel 6.10.10 - works fine
• (2024.10.05) kernel 6.6.54 - no graphics, not even console messages are visible in UEFI mode
• (2024.10.05) kernel 6.11.2 - console messages visible, but can't reach Plymouth, reboots instead
• (2024.11.02) kernel 5.15.170 - no graphics, not even console messages are visible in UEFI mode

If having a virtual serial port, notice that the login prompt does indeed appear in the serial getty ttyS0, only the graphical session fails to start. Furthermore, there are error messages related to drm configuration:

[    0.000000] Linux version 6.6.54-1.qubes.fc37.x86_64 (mockbuild@88464547b772444f82c444d96f0906c1) (gcc (GCC) 12.3.1 20230508 (Red Hat 12.3.1-1), GNU ld version 2.38-27.fc37) #1 SMP PREEMPT_DYNAMIC Mon Oct  7 20:17:52 GMT 2024
[    0.000000] Command line: placeholder root=/dev/mapper/qubes_dom0-root ro rd.lvm.lv=qubes_dom0/root rd.lvm.lv=qubes_dom0/swap console=ttyS0,115200 6.6.48-1.qubes.fc37.x86_64 x86_64
[    0.000000] Disabled fast string operations
[    0.000000] Released 0 page(s)
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] Xen: [mem 0x0000000000000000-0x000000000007ffff] usable
[    0.000000] Xen: [mem 0x0000000000080000-0x00000000000fffff] reserved
[    0.000000] Xen: [mem 0x0000000000100000-0x00000000bfecffff] usable
[    0.000000] Xen: [mem 0x00000000bfed0000-0x00000000bfefefff] ACPI data
[    0.000000] Xen: [mem 0x00000000bfeff000-0x00000000bfefffff] ACPI NVS
[    0.000000] Xen: [mem 0x00000000bff00000-0x00000000bfffffff] usable
[    0.000000] Xen: [mem 0x00000000f0000000-0x00000000f7ffffff] reserved
[    0.000000] Xen: [mem 0x00000000fec00000-0x00000000fec10fff] reserved
[    0.000000] Xen: [mem 0x00000000fed00000-0x00000000fed00fff] reserved
[    0.000000] Xen: [mem 0x00000000fee00000-0x00000000feefffff] reserved
[    0.000000] Xen: [mem 0x00000000fffe0000-0x00000000ffffffff] reserved
[    0.000000] Xen: [mem 0x0000000100000000-0x00000001400affff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] APIC: Static calls initialized
[    0.000000] SMBIOS 2.7 present.
[    0.000000] DMI: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
[    0.000000] Hypervisor detected: Xen PV
[...]
[   47.286669] vmwgfx 0000:00:0f.0: vgaarb: deactivate vga console
[   47.291025] Console: switching to colour dummy device 80x25
[   47.294017] Already setup the GSI :16
[   47.295916] vmwgfx 0000:00:0f.0: [drm] FIFO at 0x00000000fe000000 size is 8192 kiB
[   47.306151] vmwgfx 0000:00:0f.0: [drm] VRAM at 0x00000000e8000000 size is 131072 kiB
[   47.308927] vmwgfx 0000:00:0f.0: [drm] Running on SVGA version 2.
[   47.311230] vmwgfx 0000:00:0f.0: [drm] Capabilities: rect copy, cursor, cursor bypass, cursor bypass 2, 8bit emulation, alpha cursor, extended fifo, multimon, pitchlock, irq mask, display topology, gmr, traces, gmr2, screen object 2, command buffers, command buffers 2, gbobject, dx, hp cmd queue, no bb restriction, cap2 register, 
[   47.320543] vmwgfx 0000:00:0f.0: [drm] Capabilities2: grow otable, intra surface copy, dx2, gb memsize 2, screendma reg, otable ptdepth2, non ms to ms stretchblt, cursor mob, mshint, cb max size 4mb, dx3, frame type, trace full fb, extra regs, lo staging, 
[   47.327697] vmwgfx 0000:00:0f.0: [drm] *ERROR* vmwgfx seems to be running on an unsupported hypervisor.
[   47.331235] vmwgfx 0000:00:0f.0: [drm] *ERROR* This configuration is likely broken.
[   47.334417] vmwgfx 0000:00:0f.0: [drm] *ERROR* Please switch to a supported graphics device to avoid problems.
[   47.338747] vmwgfx 0000:00:0f.0: [drm] DMA map mode: Caching DMA mappings.
[   47.342701] vmwgfx 0000:00:0f.0: [drm] Legacy memory limits: VRAM = 4096 kB, FIFO = 256 kB, surface = 0 kB
[   47.347176] vmwgfx 0000:00:0f.0: [drm] MOB limits: max mob size = 262144 kB, max mob pages = 65536
[   47.351793] vmwgfx 0000:00:0f.0: [drm] Max GMR ids is 64
[   47.354636] vmwgfx 0000:00:0f.0: [drm] Max number of GMR pages is 65536
[   47.358005] vmwgfx 0000:00:0f.0: [drm] Maximum display memory size is 262144 kiB
[   47.382837] vmwgfx 0000:00:0f.0: [drm] Screen Target display unit initialized
[   47.388299] vmwgfx 0000:00:0f.0: [drm] Fifo max 0x00040000 min 0x00001000 cap 0x0000077f
[   47.398368] vmwgfx 0000:00:0f.0: swiotlb buffer is full (sz: 2097152 bytes), total 32768 (slots), used 0 (slots)
[   47.405216] vmwgfx 0000:00:0f.0: [drm] VSG table map failed!
[   47.407817] [drm:vmw_request_device_late [vmwgfx]] *ERROR* Unable to initialize guest Memory OBjects.
[   47.413743] vmwgfx: probe of 0000:00:0f.0 failed with error -5
[   47.466201] Already setup the GSI :16
[...]
[   74.116768] audit: type=1105 audit(1736692507.643:180): pid=2277 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:session_open grantors=pam_keyinit,pam_limits,pam_unix acct="user" exe="/usr/sbin/runuser" hostname=? addr=? terminal=? res=success'



Qubes release 4.2.3 (R4.2)
Kernel 6.6.54-1.qubes.fc37.x86_64 on an x86_64 (ttyS0)

dom0 login: [   80.445177] xen-blkback: backend/vbd/1/51712: using 2 queues, protocol 1 (x86_64-abi) persistent grants
[   80.474904] xen-blkback: backend/vbd/1/51728: using 2 queues, protocol 1 (x86_64-abi) persistent grants
[...]

Expected behavior

The kernels installed as part of regular dom0 updates allow booting Qubes OS just fine on VMware Workstation.

Actual behavior

The kernels installed as part of regular dom0 updates result in an graphical-session-wise unbootable Qubes OS system on VMware Workstation.

[marmarek]

[ 47.327697] vmwgfx 0000:00:0f.0: [drm] ERROR vmwgfx seems to be running on an unsupported hypervisor.
[ 47.331235] vmwgfx 0000:00:0f.0: [drm] ERROR This configuration is likely broken.
[ 47.334417] vmwgfx 0000:00:0f.0: [drm] ERROR Please switch to a supported graphics device to avoid problems.

Looks like this driver doesn't officially support running inside Xen, I guess it worked before only by coincidence. I doubt there will be anybody willing to debug this issue...

As far as I'm aware, VMware Workstation is the only hypervisor, where as of today Qubes OS can be installed directly and all its features work out-of-the-box, like HVM qubes, without the need of manual tweaks, like changing their virtualization mode to PV after installation.

This is not true. While this is not officially supported, it happens to work in KVM, as long as you enable nested virtualization and virtual IOMMU there (and avoid vitio devices, use fully emulated ones, like e1000e for network).

[andrewdavidwong]

Since installing Qubes in a VM is officially not supported, I don't think this issue is applicable in the Qubes issue tracker, but please let me know if I've misunderstood the situation.

[github-actions]

This issue has been closed as "not applicable." Here are some common examples of cases in which issues are closed as not applicable:

• Help and support requests (please see Help, support, mailing lists, and forum)
• Questions (please see Help, support, mailing lists, and forum)
• Discussion issues (please see Help, support, mailing lists, and forum)
• Bug reports for behavior that is already working as intended
• Enhancement requests to improve things that are already working as intended
• Issues that rest on mistaken assumptions or misunderstandings
• Issues that do not provide enough information
• Issues that are not actionable

We respect the time and effort you have taken to file this issue, and we understand that this outcome may be unsatisfying. Please accept our sincere apologies and know that we greatly value your participation and membership in the Qubes community.

Regarding help and support requests, please note that this issue tracker (qubes-issues) is not intended to serve as a help desk or tech support center. Instead, we've set up other venues where you can ask for help and support, ask questions, and have discussions. By contrast, the issue tracker is more of a technical tool intended to support our developers in their work. We thank you for your understanding.

If anyone reading this believes that this issue was closed in error or that the resolution of "not applicable" is not accurate, please leave a comment below saying so, and we will review this issue again. For more information, see How issues get closed.