Support Wayland

[parkerlreed]

Tested it out on Arch Linux with KDE Plasma 6 running Wayland (And subsequently Steam/Proton within XWayland)

Highlights and control work as expected.

out.mp4

[madewokherd]

Proton runs Xalia inside Wine, so it'd be going through win32 stuff and probably winex11.drv into xwayland.

Where I expect problems is trying to use this with native Linux apps running in Wayland, and possibly also winewayland.drv.

[parkerlreed]

Oh! I didn't realize this had a native mode (I had just assumed it was only for the WINE subsystem)

I'll see if I can get the build working and will report back.

Thank you!

[parkerlreed]

Ok yeah zero luck there as suspected. Binary is running (dotnet 8.0) but seemingly nothing happening. I'll dig into Plasma/AT-SPI/X11 support and see what I can find.

[parkerlreed]

After some quick testing on the Plasma X11 session, buttons sending keyboard events works but the highlight never appears, so navigation is not possible. I tried with the QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1 with the same result either way.

[madewokherd]

The problem I know you'll run into is lack of window coordinates that work across different clients. We currently rely solely on AT-SPI for this (but we're going to need window-system-specific code, as this also doesn't work with GTK3). There's been some work on this: https://lwn.net/Articles/971541/

I'm not sure if it's worth paying much attention to Wayland until "Newton" is stable and we can start using it. (Actually, it may be worth trying to use it on Windows/Wine too. Using process hooks to translate the existing APIs to a "push" architecture would likely be easier than working with comctl32/msaa/uia is turning out to be. But I'd much rather do that in a common way and not invent my own serialization protocol.)

That said, there may be other obstacles (like getting something to render on the screen above other apps) independent of the accessibility technology.

[madewokherd]

Maybe I should repurpose this as a general "Wayland support" issue?

[parkerlreed]

Sure, that would be fine.

[parkerlreed]

Ok so some progress. I have the same limited key event functionality in Wayland. I missed the sharing input dialog the first time I had tried it.

So I can send button events on Plasma Wayland but still have the issue of no element highlighting.

[parkerlreed]

Here's what that looked like

[madewokherd]

It's surprising to me that even that much works. Sending input is implemented via Win32 in Win32WindowingSystem and xtest in X11WindowingSystem. So that implies that we're using X11WindowingSystem even on wayland for some reason.

[parkerlreed]

If you are talking to AT-SPI maybe that is handling the event translation? Is XTEST always used on X11 or is that a fallback?

[madewokherd]

Sending events through the AT-SPI connection isn't implemented, so I think it has to be using xtest.