How to Integrate a RAG System or Use AlwaysReddy for a Specific Use Case?

[AtharvaDomale]

Hi everyone,
I'm exploring how we can integrate a Retrieval-Augmented Generation (RAG) system into a specific use case. More specifically, I am interested in understanding how to leverage the AlwaysReddy model for this scenario. Could anyone provide guidance or examples on how to integrate AlwaysReddy in a RAG architecture, or tips for using it effectively in a particular application? Any insights into potential challenges or best practices would be greatly appreciated. Thanks in advance!

[Jobus0]

Unfortunately, AlwaysReddy doesn't support RAG at all. Major changes to the codebase is needed to make it work.

It's been a topic of discussion a few times on the Discord, but so far, no one has really attempted to make it work. Personally, from doing research on this, I think the best solution would be majorly rewriting AlwaysReddy to be built on LangChain or LlamaIndex instead, as that would make it trivial to implement RAG, tool calling, etc.

So besides rewriting AlwaysReddy, if you're looking for a way to talk voice-to-voice with an LLM using RAG and tooling, and don't specifically need the global hotkeys that AlwaysReddy provides, then look into Open WebUI, as it supports those things - even a hands-free voice-to-voice call mode.

[AtharvaDomale]

So there is also the prompt thing in it so can you guide me how can we use that like i want to build my personal assistant for a purpose which will note down my things which i will tell just like that. And alos need help for like which libraries you used for text to speech speech to text, how it happens offline. Like which file is used for which purpose.

[Jobus0]

I don't know much about using LLMs for automatic note-taking. Perhaps you could just use Whisper to transcribe everything you say, and then run the entire transcription through an LLM to restructure it into bullet points. But I don't know of any existing workflows for that use-case.

For local STT, Open WebUI has built-in local Whisper support. Just enable it in the admin settings, and I recommend the "turbo" model.

For local TTS, you use Open WebUI's "OpenAI" option which just means to use any OpenAI API, like All-Talk TTS or OpenedAI Speech. Both of those have Piper built in, which is a very fast TTS AI with decent quality voices.

Both of these STT/TTS options are found within Open WebUI in Admin Panel -> Settings -> Audio.

[AtharvaDomale]

ok sure thanks

[AtharvaDomale]

can you tell where the input and response of user and llm is stored or in which variable, i want to pass this variable to another file so i can store this for a purpose

[Jobus0]

In the always_reddy_voice_assistant action: self.messages. It's a list field, and each user message and LLM response gets appended to it.