For displaying the simple facial expressions I used an 8×8 LED board with a built in MAX7219 controller chip. I also added the LM35 temperature, a simple soil moisture and a light sensor and finally for motion detection I used two PIR sensors. George moves his eyes where you come from and only talks if someone is around. The sensor data is concatenated into a JSON string which is then sent to Node.js through the USB. Using web sockets the Node server publishes this data for the browser. In the browser we use the Web Speech APIs to talk to and listen to people.
Please leave a comment if you would like a bit more detail on the hardware build.
Now we’re talking
The plant can complain about excessive sensor readings which is done using the HTML5 Web Speech API. In its simplest form this is how you get the browser to say something:
I then used this vocabulary and the sensor data in the data object to trigger speech whenever a value is excessive:
Here’s the piece of code that only allows the plant to talk every 3 minutes. This boolean value is then used in the speak function:
When someone waters the plant this delay is disabled to prevent the user from waiting 3 minutes until the voice feedback. The
setInterval is needed to have a time range within the soil moisture level has to increase by 10% to trigger the event.
George can interpret simple questions and answers them using the HTML5 Web Speech API. This is the Hello World example of voice recognition:
There are two helper functions I added to make talking a very simple process:
app.matchWords([array], string) - You need to pass in an array of words as the first attribute and a piece of string as a second. The function then returns true or false whether any of the words in the array has been found in the string.
app.answer(string) - Pass in the piece of string as an answer and the browser will read it out loud.
Here's a simple example that would answer you if you asked: "What's the temperature?" or "Are you hot?":
All these helper functions make it very easy to extend the vocabulary of the plant and build new features.
It might be worth spending a little bit extra time on George. It would be great to package him up into a nicer box rather than having bare wires hanging out from his body. I was also looking at HTML5 web speech libraries to see if anything can make him more reliable as sometimes it speaks and listens the same time and picks up and reacts to his own voice.
Please let me know in the comments section if you would like to see a more detailed tutorial on the entire build.