Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement push-to-talk functionality #5

Open
chatziko opened this issue Dec 11, 2023 · 4 comments
Open

Implement push-to-talk functionality #5

chatziko opened this issue Dec 11, 2023 · 4 comments

Comments

@chatziko
Copy link

Now that wyoming-satellite gives you more control over the satellite (similarly to esphome satellites), it would be very useful (and relatively easy to implement) to add "push-to-talk" functionality to it. I can imagine countless applications of starting a pipeline either by a physical button, or by various automations.

We could have a button or service on the HA that starts a pipeline directly at the STT stage, skiping wake word completely.

  1. When the satellite is disabled (and ideally idle, see also Stop recording/streaming when disabled #4), the button/service could be used to manually run a single pipeline. The server could send RunSatellite with an argument to instruct the satellite to go directly to STT (a send back a corresponding RunPipeline(start_stage=STT) to the server).

  2. When the satellite is enabled, it still makes sense to use the button/service, to skip wake word and go straight to STT. Since there might be an active pipeline at this point, we should stop it and start a new one (or maybe completely stop the satellite and restart it the same way as (1) above).

I might give it a try at some point, but I'd like to hear @synesthesiam 's thoughts on this.

@synesthesiam
Copy link
Contributor

I hadn't considered (1), though I wouldn't do anything when the satellite is disabled since the connection is broken then on purpose.

In fact, I don't think the satellite would even need to be aware if it's streaming. HA could simply kick the running pipeline from wake to STT. If the satellite is using VAD or local wake word detection, HA could just forward a Detection event and the satellites could move to the correct state.

(2) should pretty easy to do. My plan was to use the event service (--event-uri) in both directions for this. Right now, events always flow from the satellite to the event service. But a Detection event could go the other direction to inform the satellite that it should skip wake word and proceed to STT (via RunPipeline).

@erikLundstedt
Copy link

I would like to use a button connected to the GPIO-pins of the raspberry pi to "wake" it, instead of, or as an alternative to A wake-word, ive had some issues with stt hearing its own tts-"sorry I couldnt understand that"-response, leading to a long (I usually kill the script if it goes on for too long)loop

@llluis llluis mentioned this issue Jan 23, 2024
@PsY4
Copy link

PsY4 commented Jan 31, 2024

This combined with conversation_id (#95) could lead to satellite asking questions/confirmations and directly proceed to STT for an answer...

@chatziko
Copy link
Author

I gave it a try, see #144 and home-assistant/core#114291.

Clarification: my goal was to create a server-side "push-to-talk" button, which can be used to activate the satellite from the frontend, or automations, etc. Not a client-side button on the satellite side.

See home-assistant/core#114291 for design considerations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants