Sword of the Necromancer missing libraries

[michaelneverwins]

Your system information

• Steam Runtime Version: 1.0 (scout)
• Distribution (e.g. Ubuntu 18.04): Linux Mint 22
• Link to your full system information: https://gist.github.com/michaelneverwins/19fa4e92e4cbb7b957b343c838006eb9
• Have you checked for system updates?: Yes
• What compatibility tool are you using?: default [Steam Linux Runtime 1.0 (scout)]
• What versions are listed in steamapps/common/SteamLinuxRuntime/VERSIONS.txt?

  #Name	Version		Runtime	Runtime_Version	Comment
  depot	0.20240806.0			# Overall version number
  LD_LIBRARY_PATH	-	scout	-	# see ~/.steam/root/ubuntu12_32/steam-runtime/version.txt
  scripts	0.20240806.0			# from steam-runtime-tools
  

  • Contents of ~/.steam/root/ubuntu12_32/steam-runtime/version.txt:

    steam-runtime_0.20241024.105847
    

• What versions are listed in steamapps/common/SteamLinuxRuntime_soldier/VERSIONS.txt?

  #Name	Version		Runtime	Runtime_Version	Comment
  depot	0.20241118.108551			# Overall version number
  pressure-vessel	0.20241118.0	scout		# pressure-vessel-bin.tar.gz
  scripts	0.20241118.0			# from steam-runtime-tools
  soldier	0.20241118.108551	soldier	0.20241118.108551	# soldier_platform_0.20241118.108551/
  

• What versions are listed in steamapps/common/SteamLinuxRuntime_sniper/VERSIONS.txt?

  #Name	Version		Runtime	Runtime_Version	Comment
  depot	0.20241118.108552			# Overall version number
  pressure-vessel	0.20241118.0	scout		# pressure-vessel-bin.tar.gz
  scripts	0.20241118.0			# from steam-runtime-tools
  sniper	0.20241118.108552	sniper	0.20241118.108552	# sniper_platform_0.20241118.108552/
  

Please describe your issue in as much detail as possible:

Apologies if this isn't considered a bug, but Sword of the Necromancer (app ID 1243890) fails to run because it can't find some libraries. The initial complaint is about librtmp.so.1 as shown in my SLR log (slr-app1243890-t20241217T152612.log), but when I provided that library, it then failed to find libhogweed.so.6, and then libnettle.so.8. All three of these libraries are installed on my system, under /lib/x86_64-linux-gnu, and simply copying them into the game's lib folder

cp /lib/x86_64-linux-gnu/librtmp.so.1 ~/.steam/root/steamapps/common/SwordOfTheNecromancer/lib/
cp -L /lib/x86_64-linux-gnu/libhogweed.so.6 ~/.steam/root/steamapps/common/SwordOfTheNecromancer/lib/
cp -L /lib/x86_64-linux-gnu/libnettle.so.8 ~/.steam/root/steamapps/common/SwordOfTheNecromancer/lib/

did get the game to run. I don't know much about Steam Linux Runtime, but I assume it prevents the game's use of system libraries (by design and thus not a bug), which in this case breaks the game because SLR does not supply these libraries (maybe a bug). The only solution without workarounds is probably for these libraries to be added to Steam Linux Runtime, but I do understand that it's probably just not going to happen if this turns out to be the only one of Steam's two-hundred-thousand-something games that requires them.

Note that forcing Steam not to run the game in SLR would not actually help in this case, because the game also requires some libraries which are not provided by Linux Mint but are provided by SLR — such as, for example, libcrypto.so.1.1 and libssl.so.1.1. (This actually seems like a bug in the game itself, considering that it ships with libcrypto.so.1.0.0 and libssl.so.1.0.0 instead, but whatever. Even if simply disabling SLR were a viable workaround, we would still want the game to work with SLR anyway because it's the default now.)

Steps for reproducing this issue:

1. Install Sword of the Necromancer (app ID 1243890).
2. Leave the default compatibility tool, Steam Linux Runtime 1.0 (scout).
3. Try to run the game.

[smcv]

Please report this to the game developer as a bug in the game. Steam has never guaranteed to provide librtmp.so.1, libhogweed.so.6 or libnettle.so.8, and not all host OSs will have them available either.

[smcv]

forcing Steam not to run the game in SLR

Do you mean setting the compatibility tool to "Legacy runtime 1.0"?

That compatibility tool is intended to be a way to run games that make assumptions about the host OS, if those assumptions are still true today. Before November 2024, Steam's default behaviour for native Linux games was the equivalent of what we're now calling the legacy runtime.

However...

would not actually help in this case, because the game also requires some libraries which are not provided by Linux Mint but are provided by SLR

... if the game's assumptions are not true of your host system and also not true of SLR, there isn't necessarily much that we can do about that. If we tested this on lots of systems, I suspect we'd find that it worked as intended (with the legacy runtime) on older versions of Ubuntu, but no longer works on newer Ubuntu, and in particular doesn't work any more on the version of Ubuntu that your Linux Mint system is based on.

The libraries that the original (v1.0) Steam Runtime "officially" provides are listed in the scout ABI: those are the only libraries that a Steam game is meant to rely on, in the absence of configuration to the contrary. This includes librtmp.so.0 but not librtmp.so.1, and OpenSSL 1.0 (libcrypto.so.1.0.0) but not OpenSSL 1.1 (libcrypto.so.1.1).

In an attempt to be nice to game authors who have made assumptions about newer libraries, the Steam Linux Runtime 1.0 container also provides the Steam Runtime 2 'soldier' ABI. That does include OpenSSL 1.1, but doesn't include a version of librtmp at all (we excluded it in an effort to keep the size of the runtime under control).

I suspect this game probably does not actually need libhogweed.so.6 or libnettle.so.8, and you're only finding that you need those because you've dropped in a much newer version of librtmp.so.1 which was compiled against those versions.

If you fetched librtmp.so.1 from Debian 10 or from a contemporary version of Ubuntu, you might find that putting that in the game's directory would work as a less intrusive workaround.

Game developers can opt-in to having their game run in a newer container, Steam Linux Runtime 3.0 (sniper), which provides the Steam Runtime 3 ABI instead of Steam Runtimes 1 and 2 - but that doesn't have librtmp.so.1 either, so I don't think that would help you here.

This actually seems like a bug in the game itself, considering that it ships with libcrypto.so.1.0.0 and libssl.so.1.0.0 instead, but whatever

Yes. The intention is that the game developer should compile their game in a consistent environment - either the scout SDK and continue to target the scout ABI, or the sniper SDK and use the Steamworks partner web interface to tell Steam that they are now targeting the sniper ABI - and bundle or statically link any library dependencies that aren't available in their chosen runtime branch.

If bundled copies of those libraries are present, then I suspect that this particular developer probably started by compiling for the scout SDK (or for an old Ubuntu version like 12.04), but later switched to a newer build environment and making assumptions about the host system that we can't guarantee over time.

I don't know much about Steam Linux Runtime, but I assume it prevents the game's use of system libraries (by design and thus not a bug)

Yes, that's right. What we're trying to do is to make sure that if a game works today, it's very likely that it'll still work a few years from now.

The legacy runtime didn't achieve that, as you've seen: Sword of the Necromancer presumably worked in the past, but no longer does, because it was making assumptions that were not backed up by anyone's compatibility guarantees (either ours or the OS vendor's).

The only solution without workarounds is probably for these libraries to be added to Steam Linux Runtime, but I do understand that it's probably just not going to happen if this turns out to be the only one of Steam's two-hundred-thousand-something games that requires them.

Yes, this. We've added libraries as a workaround for mis-packaged games before (like libcups.so.2), and we could certainly consider doing the same for librtmp.so.1 if it's a common dependency, but we do have to draw a line somewhere: each new library makes the runtime larger for all Steam-on-Linux users, and not everything is legally or technically suitable to be included.

[michaelneverwins]

forcing Steam not to run the game in SLR

Do you mean setting the compatibility tool to "Legacy runtime 1.0"?

Yes and no. I had actually meant forcing Steam not to use any Steam runtime at all (i.e. just running the game with system libraries), because I thought there was some way to do that, but I think I was mistaken. Prior to the update which made "scout" the default, I thought the default was no Steam runtime, but it looks like the "legacy" runtime is the old default which the -compat-force-slr off option would restore.

I didn't bother to try it because I assumed it wouldn't help for this game, since I knew (by trying to run it from the command line) that the game requires other dependencies e.g. libcrypto.so.1.1 which I figured I wouldn't get by forcing SLR off. (If the legacy runtime would provide some dependencies while falling back on my system for others, then maybe I should actually try it. Then again, changing a global Steam setting is a more annoying workaround than what I've already done, and I might just end up using Proton for this game anyway if I can't find a fix for the Linux version's apparently broken controller support.)

I believe Sword of the Necromancer uses the GameMaker engine, and I'd often relied on Steam to get such games to work in the past. Many of the older ones, at least, require libcrypto.so.1.0.0 and libssl.so.1.0.0 which are in the Steam runtime folders (although I don't actually remember whether I had to enable "scout" for them back when the legacy runtime was the default). This game is a bit different in that it requires libcrypto.so.1.1 and libssl.so.1.1, maybe because it's a newer version of the engine or something, and maybe the developer shipped it with libcrypto.so.1.0.0 and libssl.so.1.0.0 not realizing that.

On that note, updating SLR to include librtmp.so.1 etc. might be worth considering if those are requirements of every game built for Linux with the same version of GameMaker that this game uses, but frankly I have no idea if that's the case. I haven't personally seen any others, not that I've gone looking for them.

[smcv]

Looking at the game's depot file listing, it seems to have a bundled copy of libcurl.so.4, which might well be part of the problem - if the game or engine developer has copied this libcurl from some general-purpose distro like Ubuntu circa 2020, it might well have the optional RTMP feature enabled, which adds the dependency on librtmp that is causing you trouble. (Our libcurl in the Steam Runtime does not: we turned that feature off to reduce its dependencies, on the basis that it seems likely to be rarely-used in practice.)

If you undo your workaround of adding librtmp.so.1, and then move the bundled copy of libcurl.so.4 out of the way, does the game run successfully with the Steam Linux Runtime 1.0 (scout) version of libcurl.so.4?

It might not (we don't know what assumptions the game or engine developer has made) but it's worth a try.

I might just end up using Proton for this game anyway if I can't find a fix for the Linux version's apparently broken controller support

For what it's worth, Valve's compatibility testing lists Proton as the recommended way to run this game on Steam Deck, so you'd be in good company there.

I had actually meant forcing Steam not to use any Steam runtime at all (i.e. just running the game with system libraries)

This is not something that Steam-on-Linux has ever offered as a supported code path: even back in 2013 it was reasonably obvious that there's no such thing as a forward-compatible Linux platform ABI, beyond the absolute bare minimum (ancient versions of glibc, libX11 and libGL, and that's about all you get). Technically there is still a way to make this happen, but it's unsupported and in practice will break most games, and a future Steam release might well remove it.

Prior to the update which made "scout" the default, I thought the default was no Steam runtime, but it looks like the "legacy" runtime is the old default which the -compat-force-slr off option would restore.

That's right. The "Legacy runtime 1.0" didn't exist as a separate compatibility tool until recently, but the old default (as restored by -compat-force-slr off) was functionally equivalent to using the legacy runtime for all applicable games, and the new default is equivalent to using "Steam Linux Runtime 1.0 (scout)" for all applicable games.

If the legacy runtime would provide some dependencies while falling back on my system for others

Actually it does, but for this specific game I don't think it's going to help you, because libcrypto.so.1.1 is not among the dependencies that the legacy runtime provides: OpenSSL 1.1 is simultaneously too new to be provided by the legacy runtime, and too old to be provided by your host system.

Then again, changing a global Steam setting is a more annoying workaround than what I've already done

Now that we have "Legacy runtime 1.0" as an independent compatibility tool, you don't need to change a global Steam setting: you can swap between compatibility tools on a per-game basis.

updating SLR to include librtmp.so.1 etc. might be worth considering if those are requirements of every game built for Linux with the same version of GameMaker that this game uses, but frankly I have no idea if that's the case

Yes, that's good reasoning: I don't think we're going to include it for the benefit of a single game, but we're more likely to consider including it if it benefits a whole family of related games.

[michaelneverwins]

If you undo your workaround of adding librtmp.so.1, and then move the bundled copy of libcurl.so.4 out of the way, does the game run successfully with the Steam Linux Runtime 1.0 (scout) version of libcurl.so.4?

After undoing all of my previous tinkering with a fresh install of the game, I removed the included lib/libcurl.so.4 and tried to run the game with %command% 2> ~/stderr.log in the launch options. It failed to run, and I found the following at the end of ~/stderr.log:

./SwordOfTheNecromancer.backup: /home/michael/.steam/debian-installation/steamapps/common/SteamLinuxRuntime/var/steam-runtime/pinned_libs_64/libcurl.so.4: version `CURL_OPENSSL_4' not found (required by ./SwordOfTheNecromancer.backup)

So apparently it doesn't like the Steam runtime's libcurl.so.4, unless I'm totally misunderstanding the error. In any case, I only tried this for the sake of having the answer to your question documented. I've already got the workaround mentioned in the original post if ever I want to try this game's Linux port again in the future. In the meantime...

For what it's worth, Valve's compatibility testing lists Proton as the recommended way to run this game on Steam Deck, so you'd be in good company there.

This does seem like the kind of game I'd prefer to play on my Steam Deck anyway, so I'll probably install it there and let it default to Proton. I do enjoy running Linux versions of games whenever possible (to the point where I'm willing to go through some extra steps if a Linux port is only slightly broken), but I don't know how to fix the controller input for this one (and as far as I know, that's just a bug in the game itself, and nothing to do with SLR).

Anyway, thanks for all the information. I'd close this issue if I knew that the weirdo dependencies are just for this game, and not every game built with a particular version of GameMaker. Or if you think it's probably the former, then feel free to close it. Either way, if I ever find another game with exactly the same issue, I'll just come back and drop the title here.

[smcv]

So apparently it doesn't like the Steam runtime's libcurl.so.4

Yes: the ABI of libcurl in Debian-derived distributions involved some well-intentioned but ultimately inadvisable decisions between around 2005 and 2018, and unfortunately the scout branch of the Steam Runtime happens to have used one of the affected versions as its baseline, which we can never change now, because it would break games that were previously considered to be doing everything correctly.

So, probably this particular game expects and requires a libcurl.so.4 that is compatible with its bundled copy (which has the upstream/Debian-post-2018 ABI) rather than the version in the Steam Runtime (which has the 2005-2018 ABI); but its bundled copy also has dependencies that are not part of the Steam Runtime (namely librtmp.so.1 and possibly others) so it could never be guaranteed to work everywhere, and in practice doesn't work now that we're using the more constrained container environment by default.

I'd close this issue if I knew that the weirdo dependencies are just for this game, and not every game built with a particular version of GameMaker. Or if you think it's probably the former, then feel free to close it. Either way, if I ever find another game with exactly the same issue, I'll just come back and drop the title here.

Yeah, I think the most likely resolutions for this issue report are either:

• if it's game-specific, then we're most likely to do nothing, leave this particular game non-functional, and suggest running its Windows build under Proton, as is already done by default on Steam Deck (which it seems might also fix game-controller support, as a bonus);
• or if it's a more general GameMaker problem and can be resolved by adding Debian 10's librtmp.so.1 to soldier, then that might be something we could consider

I do enjoy running Linux versions of games whenever possible (to the point where I'm willing to go through some extra steps if a Linux port is only slightly broken), but I don't know how to fix the controller input for this one (and as far as I know, that's just a bug in the game itself, and nothing to do with SLR)

This all sounds like something to discuss with the game developer, or with their engine supplier (seems to be GameMaker, like you said).

For what it's worth, I used to use Awesomenauts (another GameMaker title) as a test-case for the Steam Runtime and it worked fine, although I think that was a much older version of the GameMaker engine.