wire-picovoice-pod

This repo contains a custom Vector escape pod made from chipper and vector-cloud.

This repo is a copy of wire-pod but instead of using Coqui STT, it uses Picovoice Leopard. Leopard is faster, more accurate, and supports more hardware than Coqui, but it is not a totally local solution (processing is done locally, but it uploads usage to a server). The Coqui STT version will still be developed alongside this.

Program descriptions

chipper – Chipper is a program used on Digital Dream Lab’s servers which takes in a Vector’s voice stream, puts it into a speech-to-text processor, and spits out an intent. This is also likely used on the official escape pod. This repo contains an older tree of chipper which does not have the “intent graph” feature (it caused an error upon every new stream), and it now has a working voice processor.

vector-cloud – Vector-cloud is the program which runs on Vector himself which uploads the mic stream to a chipper instance. This repo has an older tree of vector-cloud which also does not have the “intent graph” feature and has been modified to allow for a custom CA cert.

Configuring, installing, running

NOTE: This only works with OSKR-unlocked, Dev-unlocked, or Whiskey robots running VicOS version 1.4 and above.

Linux

(This currently only works on Arch or Debian-based Linux)

cd ~
git clone https://github.com/kercre123/wire-picovoice-pod.git
cd wire-picovoice-pod
sudo ./setup.sh

# You should be able to just press enter for all of the settings

Now install the files created by the script onto the bot:

sudo ./setup.sh scp <vectorip> <path/to/key>

Example:

sudo ./setup.sh scp 192.168.1.150 /home/wire/id_rsa_Vector-R2D2

If you are on my custom software (WireOS), you do not have to provide an SSH key,

Example:

sudo ./setup.sh scp 192.168.1.150

The bot should now be configured to communicate with your server. You do not need to restart the bot to start using voice commands with the new server environment, but you will need to restart him at some point for weather commands to be reliable.

To start wire-picovoice-pod, run:

cd chipper
sudo ./start.sh

To stop wire-picovoice-pod, press CTRL+C with the terminal window on top.

Update

To update to a newer version of wire-picovoice-pod, make sure chipper is not running then run:

cd ~/wire-picovoice-pod
sudo git pull
cd chipper
sudo ./start.sh

If you see an error, run the command it tells you to run but with sudo at the beginning of the command. Then run sudo ./start.sh again.

Systemd daemon

You can setup a systemd daemon for wire-picovoice-pod. This allows it to run in the background and it to run automatically at startup.

cd ~/wire-picovoice-pod
sudo ./setup.sh daemon-enable

To start the service, either restart your computer or run:

sudo systemctl start wire-picovoice-pod

To see logs, run:

journalctl -fe | grep start.sh

If you would like to disable the daemon, run:

cd ~/wire-picovoice-pod
sudo ./setup.sh daemon-disable

Windows 10/11

  1. Install WSL (Windows Subsystem for Linux)
    • Open Powershell
    • Run wsl --install
    • Reboot the system
    • Run wsl --install -d Ubuntu-20.04
    • Open up Ubuntu 20.04 in start menu and configure it like it says.
  2. Find IP address
    • Open Powershell
    • Run ipconfig
    • Find your computer’s IPv4 address and note it somewhere. It usually starts with 10.0. or 192.168.
  3. Install wire-picovoice-pod
    • Follow the Linux instructions from above
    • Enter the IP you got from ipconfig earlier instead of the one provided by setup.sh
    • Use the default port and do not enter a different one
  4. Setup firewall rules
    • Open Powershell
    • Run Set-ExecutionPolicy
    • When it asks, enter Bypass
    • Download this file
    • Go to your Downloads folder in File Explorer and Right Click -> Run as administrator
  5. Try a voice command

macOS 10.15 and above

  1. Open Terminal
  2. Install required packages
    • Install brew with this command:
    • /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
    • Install required packages with this command:
    • brew install opusfile opus pkg-config gcc golang
  3. Install wire-picovoice-pod
    • Follow the Linux instructions, located above the Windows instructions
  4. Try a voice command

Configure specific bots

./chipper/botSetup.sh is there to help configure specific bots. This is not required for operation, and is there for only if you have multiple users using your instance of wire-picovoice-pod and you would like to use different locations/units for weather depending on the bot. It also helps chipper know if a bot is on an older version of VicOS so it can account for that. This is only a stand-in until jdocs and stuff get implemented.

Usage: ./chipper/botSetup.sh <esn> <firmware-prefix> "<location>" <units>
Example: ./chipper/botSetup.sh 0060059b 1.8 "Des Moines, Iowa" F

0.10-era bots

0.10 and below use raw PCM streams rather than the modern Opus streams. This has support for those streams and no special configuration server-side is required for it. However: you will need to get a domain which is the same length as chipper-dev.api.anki.com, make sure to run this on a port that is 3 characters long (like the default 443), add true TLS certificates (can be done in ./chipper/source.sh. make sure to include the chain), and run these commands (SSHed into the bot):

cd /anki/bin
systemctl stop vic-cloud
cp vic-cloud orig-vic-cloud
sed -i "s/chipper-dev.api.anki.com:443/<domain>:<port>/g" vic-cloud
systemctl start vic-cloud

Status

OS Support:

  • Arch
  • Debian/Ubuntu/other APT distros
  • Windows 10/11 (WSL only)
  • macOS 10.15 or above

Architecture Support:

  • amd64/x86_64
  • arm64/aarch64

Things wire-picovoice-pod has worked on:

  • Raspberry Pi 4B+ 4GB RAM with Raspberry Pi OS
    • Recommended platform, very fast
    • 64-bit only (I think)
  • Raspberry Pi 4B+ 4GB RAM with Manjaro 22.04
  • Nintendo Switch with L4T Ubuntu
  • Desktop with Ryzen 5 3600, 16 GB RAM with Ubuntu 22.04
  • Laptop with mobile i7
  • Late 2009 iMac with Core 2 Duo
  • Android Devices
    • Pixel 4, Note 4, Razer Phone, Oculus Quest 2, OnePlus 7 Pro, Moto G6, Pixel 2
    • If you run into an error when trying to execute start.sh, please open an issue. This is a Picovoice Leopard issue and can be solved by editing the leopard module.
    • Termux proot-distro: Use Ubuntu, make sure to use a port above 1024 and not the default 443.
    • Linux Deploy: Works stock, just make sure to choose the arch that matches your device in settings. Also use a bigger image size, at least 3 GB.

General notes:

  • If you get this error when running chipper, you are using a port that is being taken up by a program already: panic: runtime error: invalid memory address or nil pointer dereference
    • Run ./setup.sh with the 5th and 6th option to change the port, you will need to push files to the bot again.
  • If you want to disable logging from the voice processor, edit ./chipper/source.sh and change DEBUG_LOGGING to false
  • There is support for 5 robots at a time

Current implemented actions:

  • Good robot
  • Bad robot
  • Change your eye color
  • Change your eye color to
    • blue, purple, teal, green, yellow
  • How old are you
  • Start exploring (“deploring” works better)
  • Go home (or “go to your charger”)
  • Go to sleep
  • Good morning
  • Good night
  • What time is it
  • Goodbye
  • Happy new year
  • Happy holidays
  • Hello
  • Sign in alexa
  • Sign out alexa
  • I love you
  • Move forward
  • Turn left
  • Turn right
  • Roll your cube
  • Pop a wheelie
  • Fistbump
  • Blackjack (say yes/no instead of hit/stand)
  • Yes (affirmative)
  • No (negative)
  • What’s my name
  • Take a photo
  • Take a photo of me
  • What’s the weather
    • Requires API setup
    • weatherapi.com is implemented, use the 5th option in ./setup.sh to set it up
    • To set a default location, use the botSetup.sh script in the ./chipper directory
  • What’s the weather in
    • Requires API setup
    • weatherapi.com is implemented, use the 5th option in ./setup.sh to set it up
  • Im sorry
  • Back up
  • Come here
  • Volume down
  • Be quiet
  • Volume up
  • Look at me
  • Set the volume to
    • High, medium high, medium, medium low, low
  • Shut up
  • My name is
  • I have a question
    • Requires API setup
    • Houndify is implemented, use the 5th option in ./setup.sh to set it up
  • Set a timer for seconds
  • Set a timer for minutes
  • Check the timer
  • Stop the timer
  • Dance
  • Pick up the cube
  • Fetch the cube
  • Find the cube
  • Do a trick
  • Record a message for
    • Enable Messaging feature in Vector’s webViz Features tab
  • Play a message for
    • Enable Messaging feature in Vector’s webViz Features tab
  • Play keepaway
    • This may only be a feature in 1.5 and lower

Credits

GitHub

View Github