A physical installation showing linguistic tones in Mandarin in a movement of 4 tree branches. Combining nature and language, I hope we could communicate with nature directly in an abstract way. So the tone of diverse words such as “land土地” “appreciation感谢” “oxygen氧气” ”hedonism享乐” “realization领悟” “Lu Lyu吕璐” can get the same effect.
All the code is available at this github link. here is the code source
I am using a TB6600 and a 42 stepper motor with a torque of 0.42N/m.
So this is the diagram for the propeller side.
I also add the hall sensor to sense the movement so that everytime, the stepper motor would stop at proper position.
Arduino side is really simple, I use a VL6180 to sense if anyone is approaching the microphone or not.
I cannot do this on Raspberry Pi because the sensor’s library do not have a node version.
source code I am using node for raspberry pi and here are the node packages I used for this project.
Mic: record the USB sound input as a new RAW file
Ffmpeg: convert raw to wav
Watson: voice recognition api for Chinese language
cnChar: api for turn Chinese words into tone
Serialport: js for serial communication with propeller to control motors
I think I am having the biggest struggle with the microphone input issue because it is hard for me to stop the stream and do another mic input. Then Pedro helped me by figuring out how to call a node inside another node.
Then ffmpeg would convert the file from .raw to .flac for watson to recognize it.
Then watson API would do voice recognation and pinyin node package would get the tone value from transcipt.