I think this may take more than 1 post, however the goal is simple:
Use Wio Terminal microphone to recognize wake up words and open or close the Azure IoT Door 🚪.
So, let’s start.
I’ll use Edge Impuse, to to train the voice recognition module. I talk about Edge Impulse several times, and I’m sorry if in your searchs you find these 2 images.
Please choose the correct (2nd) one 😁.
One of the coolest features of Edge Impulse, is that we can define a full ML process to train our models. There are plenty of tutorials in the home page, however this is a classic definition on how to train, validate, deploy and use a machine learning model.
Bonus, we can do this in 5 minutes ! I strongly advice to take a look at the Edge Impulse Guides and follow any of the included tutorials.
Ok, I mentioned that I want to use the Wio Terminal microphone to record my audio samples. Now is time to use another amazing tool integrated with Edge Impulse:
CodeCraft is a visual environment that we can use to program several devices. And we have access to :
And, if we work with Wio Terminal, we can also create tinyML models using Edge Impulse. We have 4 options
- Motion Recognition
- Gesture Recognition
- Wake-up words
- Smell Recognition
So, let’s start with the Wake-up words recognition.
Recording Audio Samples to train our model
Once we select the audio model, we need to define our labels and we can start recording the audio data samples. In the next step, I defined 3 labels
- open door
- close door
This step also allow us to create a simple program to deploy to the Wio Terminal:
- When we press the [Button A] it will record a [3 seconds] audio for the [open door] label.
- When we press the [Button B] it will record a [3 seconds] audio for the [close door] label.
- When we press the [Button C] it will record a [3 seconds] audio for the [background] label.
Important: in order to download the CodeCraft program, we need to install a device assistant locally.
Once we have this installed, we can upload the program to our Wio Terminal device and start to record our audios. In the following sample, I recorded 3 audios for each label:
We can now, add more audios and later download them to be used on Edge Impulse. That’s material for my next post !!!
- Download Arduino IDE
- Wiki Seeed – Historgram
- Wiki Seeed – loading Images
- Wiki Seeed – Installing the File System Library
- Wiki Seed – Wifi Connectivity
- GitHub – Arduino_JSON
- Wikipedia – X BitMap
- Edge Impulse
WioTerminal – Posts to interact with a Digital Twin Door 🚪 with Azure IoT ☁️ and Azure Functions
- Convert and use images on the device
- 1st steps 👣, developer steps
- Buttons and Charts time 📊📉📊
- Connecting to Wifi 📶, display local IP and get ready for Azure ☁️ scenarios
- Getting JSON data from an Azure ☁️ Function
- Parsing JSON data from an Azure ☁️ Function
- Display a Digital Twin Door 🚪 state using XBitmap with Azure IoT ☁️
- Display a countdown progress bar 🚥 for the next Azure IoT ☁️ refresh data call
- Open and close the Digital Twin Door 🚪 using the Wio Terminal Buttons
- Training an 🗣️ audio recognition module. Record 🎙️ audio samples for training
- Training an 🗣️ audio recognition module. Edge Impulse for Arduino step-by-step and optimizations
- Training an 🗣️ audio recognition module. Running the model on the device
- Playing sound on the open and close events from the Digital Twin Door 🚪 (coming soon)