We play music mostly with our fingers, hands, mouths and feet. There is a good reason why we don't use other parts of our bodies like ties, legs or butts for making music. The reason is that the hands with the fingers and the mouth with the lungs provide fine muscle control. One part of our bodies, which also has fine muscle control but is largely overlooked is our faces. The the more than 50+ facial muscles are capable making dozens of distinct facial expressions. Until recently, however it was almost impossible to harness these muscles and use them for music making. Fortunately, new technologies like the TrueDepth cameras built into some smart phones and Augmented Reality tool kits like ARKit in iOS and ARCore in the Android operation systems are finally making this possible. Unlike any other musical instruments the MusEmoji app is played with your facial expressions. From all possible facial expression, which you are capable of doing we have chosen only 12 distinct ones that can be easily done by anyone. We have mapped those expression to the notes in the musical octave.
The MusEmoji app runs on iPhones or iPads version X or higher. It uses the TrueDepth camera to do face tracking and maps some of the most common facial expressions to musical notes. The app is a MIDI controller & synthesizer. It features 12 different MIDI instruments. Developed by Brainventions, Inc. MusEmoji is a member of an entire family of similar apps (MusiGigZ) designed for Making Music in Motion. Check out Sk8SoniX (a.k.a. MusiSk8s) for ice and roller skating, MusiShuZ for dancing, MusiMoto for all kinds of movement activities like walking, jogging, bicycle riding, ice skating and even dancing.
Silly Faces Making Music
Smile, frown, wink, twitch your nose, lift your brows, open your mouth, stick out your tongue and puff the cheeks to play music. Here are some videos of Val's experiments with MusEmoji
To help you master playing music with your facial expressions the MusEmoji app includes views of several popular melodies. Tap the 🎶 button on the menu and swipe left or right to choose a melody. Each melody can be displayed in 5 different notations (SCR - score, DRM - solfege, CDE - American, 123 - Asian, OJI - MusEmoji). You can also watch a video of the chosen melody played by an experienced MusEmoji player.
Compare with EyeHarp
A similar project for making music by people with disabilities!