Enable GPU support

[mreilaender]

Enables GPU support by adding --cuda to the argument parser and passing it to piper

Closes #1

[edurenye]

Looks good to me, and this would be a great thing to have.
Can you please review it @synesthesiam ?

[synesthesiam]

The piper executable does not support the --cuda flag

[edurenye]

Hi @synesthesiam, thank you for your quick answer.

According to the documentation from here: https://github.com/rhasspy/piper (line https://github.com/rhasspy/piper/blob/6c5e283439f8400aa7a2652aafbedfb77ca3fc22/README.md?plain=1#L176) piper has a --cuda flag.

Is it a bug there that cuda does not work (I see you just commited something in that repo related to cuda) or is it in this other issue rhasspy/rhasspy3#49?

Can we help you in some way?

[synesthesiam]

The Python library for piper supports CUDA. I added the flag for the C++ version yesterday, but it needs the "gpu" version of the onnx runtime so I will need to adjust the build process too.

[ashald]

@synesthesiam is there a way to leverage the latest C++ version that has the --cuda parameter? Thanks!

[ms1design]

+1, @synesthesiam I would be appreciated if you could guide us a bit how to utilise cuda in our Jetson project here (using cuda12.2 and onnxruntime-gpu): dusty-nv/jetson-containers#481

Edit: Solved here: dusty-nv/jetson-containers@4ba5dc4

[roryeckel]

I agree with @synesthesiam, piper supports "--use-cuda", not "--cuda"

Source: https://github.com/rhasspy/piper/blob/c0670df63daf07070c9be36b5c4bed270ad72383/src/cpp/main.cpp#L441

Please see my comment on a related project where I encountered the same issue here: linuxserver/docker-piper#8 (comment)

[edurenye]

Good catch @roryeckel !

Can you @mreilaender update this PR please?

[edurenye]

Seems like after using --use-cuda let's us move a bit forward, but then I encounter this error:

wyoming-piper-1    | ERROR:asyncio:Task exception was never retrieved
wyoming-piper-1    | future: <Task finished name='wyoming event handler' coro=<AsyncEventHandler.run() done, defined at /opt/venv/lib/python3.11/site-packages/wyoming/server.py:31> exception=FileNotFoundError(2, 'No such file or directory')>
wyoming-piper-1    | Traceback (most recent call last):
wyoming-piper-1    |   File "/opt/venv/lib/python3.11/site-packages/wyoming/server.py", line 41, in run
wyoming-piper-1    |     if not (await self.handle_event(event)):
wyoming-piper-1    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
wyoming-piper-1    |   File "/opt/venv/lib/python3.11/site-packages/wyoming_piper/handler.py", line 53, in handle_event
wyoming-piper-1    |     raise err
wyoming-piper-1    |   File "/opt/venv/lib/python3.11/site-packages/wyoming_piper/handler.py", line 48, in handle_event
wyoming-piper-1    |     return await self._handle_event(event)
wyoming-piper-1    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
wyoming-piper-1    |   File "/opt/venv/lib/python3.11/site-packages/wyoming_piper/handler.py", line 108, in _handle_event
wyoming-piper-1    |     wav_file: wave.Wave_read = wave.open(output_path, "rb")
wyoming-piper-1    |                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
wyoming-piper-1    |   File "/usr/lib/python3.11/wave.py", line 630, in open
wyoming-piper-1    |     return Wave_read(f)
wyoming-piper-1    |            ^^^^^^^^^^^^
wyoming-piper-1    |   File "/usr/lib/python3.11/wave.py", line 280, in __init__
wyoming-piper-1    |     f = builtins.open(f, 'rb')
wyoming-piper-1    |         ^^^^^^^^^^^^^^^^^^^^^^
wyoming-piper-1    | FileNotFoundError: [Errno 2] No such file or directory: ''

This issue was already reported in this other issue: rhasspy/wyoming#9

[roryeckel]

Indeed I experienced the same issue in linuxserver/docker-piper#8 (comment) , it seems to be a segfault

[edurenye]

Why do you think it might be a segfault? I was thinking more about a permissions issue, since the error says: "No such file or directory"
But I have no idea

[roryeckel]

Why do you think it might be a segfault? I was thinking more about a permissions issue, since the error says: "No such file or directory"
But I have no idea

I came to that conclusion by running the piper binary inside the container without the python wyoming scaffolding around it. The reason it fails to load a file is because empty string is returned by piper, which is interpreted as a filename. Certainly empty string is not a valid file. But the segfault is the reason for empty string.

[ms1design]

Piper stores the wavefile on filesystem. It's a permissions issue. Piper works perfectly with CUDA and onnxruntime-gpu in jetson-containers: https://github.com/dusty-nv/jetson-containers/blob/master/packages/smart-home/wyoming/piper/Dockerfile

You can find a git patch there, showcasing fow to fix cuda flag issues: https://github.com/dusty-nv/jetson-containers/blob/master/packages/smart-home/wyoming/piper/wyoming-piper_cuda_path.diff

Cheers.

[mreilaender]

@edurenye done. I used pull --rebase I hope that's fine. Let me know if it isn't

[edurenye]

@mreilaender Thanks for updating the PR. The rebase looks fine, but is not enough changing the input argument, what you really need to change is the past argument, so you changed the __main__.py file, but missed changing the process.py file. See what I did here: rhasspy/wyoming-addons@4da560b