Also check /usr/bin/ path for nvidia cards e.g. in WSL

[RJKeevil]

Use an additional path to search for nvidia GPUs

[janpfeifer]

So this function is called if a cuda PJRT plugin was found, but it is not sure if there is an actual GPU card installed.

The use case is: a demo docker with all the PJRTs installed shouldn't attempt to run a cuda PJRT if it is running on a computer with no GPUs.

Looking at /usr/bin/nvidia will only detect that the nvidia programs are installed, not whether there is an actual GPU card installed.

Now looking at /dev/nvidia* seems not to be fail proof either ... Let's chat later maybe we could look at:

• ls -ld /sys/module/nvidia*
• ls -ld /sys/bus/pci/drivers/nvidia*

Is any of those not empty in your container set up ?

In the meantime, I'm adding documentation and logging to the function, including logging of the work around: providing the absolute path to the cuda pjrt. See #19

[RJKeevil]

Both of these paths are empty, i think theres some wizardry with Docker Desktop and WSL2 where the container somehow just delegates to the cuda drivers on the host OS. nvidia-smi is added to path, perhaps issuing that command is a reasonably generic way to see if Cuda is present in a system?

[janpfeifer]

I hesitate making the test depend on the installation of nvidia-smi. For instance, the demo docker doesn't contain it, even though it works with NVidia CUDA. Also, I'm not sure about distribution rights of these nvidia tools. The legalese is not clear to me ... but maybe it's an option. Let me search around for alternatives in Windows WSL.

[RJKeevil]

Agreed, I dont think it should depend on it; the current check could still look for nvidia files but calling nvidia-smi could be a fallback for this case? I've looked further in the container, only other evidence I can find for the presence of cuda is the presence of /usr/lib/wsl/drivers/nv_dispi.inf_amd64_adf5a840df867035

[janpfeifer]

I posted the question in NVidia forums:

https://forums.developer.nvidia.com/t/how-to-detect-the-presence-of-a-gpu-card-in-windows-wsl-wsl2/315453

[janpfeifer]

Yes, checking if nvidia-smi is available and then executing it to check is a very viable option. Do you want to implement that ?