WML

✨ WhisperCpp + LlaMAcpp + Mimic3 = Voice Prompted Assistant *Experiment

wlm-demo-vid.MP4

Quick preview

• Test this Notebook

Are you ready to play on your local machine??

• Python 3.10 / Conda / Conda Miniforge

• Create conda environment and activate

    conda create -n wlm-env python=3.10 -y
    conda activate wlm-env

• Install dependencies

  pip install numpy
  pip install uvicorn
  pip install python-multipart
  pip install fastapi
  pip install pydantic
  pip install cffi
  pip install pydub
  pip install pyngrok
  pip install nest-asyncio
  pip install aiocache

• Clone this repo

    git clone https://github.com/beltrewilton/wlm.git

• Getting the models

  • Whisper Model

      cd wlm/models
      ./download-ggml-model.sh base.en

  • LlaMA Model (already quantized to 4-bits q4_0 method) Details about

      ./download-llama-ggml-model.sh

  • Mimic3 Voices

    git clone https://github.com/MycroftAI/mimic3-voices.git

  • Install Mimic3 (Text to Speech Engine) Detail about

      # -> go to wlm directory
      git clone https://github.com/mycroftAI/mimic3.git
      cp install-mimic3.sh mimic3/
      cd mimic3
      ./install-mimic3.sh 

    • Install mimic3 plugin (Python bridge)

         pip install mycroft_plugin_tts_mimic3

• Installing WhisperCpp Python bridge.

      pip install git+https://github.com/aarnphm/whispercpp.git -vv

• Installing (also) LlaMACpp Python bridge

     pip install --no-cache-dir llama-cpp-python

• Building web-client (React)

     cd wlm/client 
     npm install
     
     cp wlm/client/node_modules/@ricky0123/vad-web/dist/vad.worklet.bundle.min.js  wlm/client/
     cp wlm/client/node_modules/@ricky0123/vad-web/dist/*.onnx  wlm/client/
     cp wlm/client/node_modules/onnxruntime-web/dist/*.wasm  wlm/client/
  
     cd wlm/client; npm run build

• Doing some SSL configuration

  • First, Configure/install mkcert See how to install on your local machine
  • Next, generate keys:

   cd wlm/server
   ./generate_keys.sh

• Running the services!

   ./run-server.sh