MusEmoji App
The MusEmoji app currently runs only on iOS 12 and iPadOS 12 or higher and devices model X or higher, which have TrueDepth camera. This camera is necessary for Face Tracking. The app is available for download on the Apple AppStore.
ย
The MusEmoji app is localized in 20 of the most popular world languages including: ar ๐ฆ๐ช, cs ๐จ๐ฟ, de ๐ฉ๐ช, el ๐ฎ๐ฑ, en ๐ฌ๐ง๐บ๐ธ, es ๐ช๐ธ, fr ๐ซ๐ท, he ๐ฌ๐ท, hi ๐ฎ๐ณ, hu ๐ญ๐บ, id ๐ฎ๐ฉ, it ๐ฎ๐น, ja ๐ฏ๐ต, ko ๐ฐ๐ท, nl ๐ณ๐ฑ, po ๐ต๐ฑ, pt ๐ต๐น๐ง๐ท, ru ๐ท๐บ and zh ๐จ๐ณ. To change the app's language to one of your choice follow the instructions in the Help โน๏ธ section of the app.
ย
To play the app your face must be sufficiently lit up. The iPhone screen illumination, if turned up to the maximum, should be sufficient so you can play even while in bed in the middle of the night.
How to Play
For detailed instruction on how to play the MusEmoji app tap the โน๏ธ button on the top menu.
ย
If you want to see yourself as you play, tap the video ๐ฆ button. For proper functioning of the app make sure that your face is always in front and facing straight the iPhone TrueDepth camera. In this mode your face will be tracked and the background will be green ๐ฉ.
ย
If the camera does not see your face it will switch to an idle mode (red background ๐ฅ) and will try to gain your attention again after 2 minutes of inactivity.
ย
To change instruments scroll left or right the instruments scroll view and tap the instrument of your choice. Alternatively you can change instruments by tapping the instrument icon on your forehead when the video mode is enabled. There are 12 MIDI instruments in the app that you can choose form including boy and girl solfege (๐ป ๐น ๐ช ๐บ ๐ช ๐ฅ ๐ท ๐ธ ๐ฏ ๐ ๐ฉโ๐ค ๐จโ๐ค).
ย
You can play musical notes Soft (piano) or Loud (forte). To change the volume/loudness ๐ of the note you play tilt your head slightly up or down. NOTICE: The app is very sensitive to slight head moves and changes in the expressions.
ย
The default octave is C4 4๏ธโฃ.ย To change the octave in which a note is played turn your head slightly left for lower octave 3๏ธโฃ or right for higher octave 5๏ธโฃ.
Face Expressions to Notes Mapping
The Mapping
As seen on the screenshot the white piano keys and the corresponding facial expressions are marked with their solfege notations Do ๐, Re ๐คฅ, Mi ๐, Fa ๐ฎ, Sol ๐, La ๐, Ti ๐ต (C, D, E, F, G, A, B). The 5 black keys (diezes & bemoles a.k.a. sharps & flats) are labeled with the 5 vowels of the Roman alphabet A(C#) ๐, E(Eb) ๐ก, I(F#) ๐ , O(G#) ๐คฅ, U(Bb)๐.
ย
Why these Choices ๐
In developing the MusEmoji app we had fo find and test an optimal mapping between facial expressions and musical notes. We had several considerations. The staring point was to choose the neutral face ๐ when no notes are played. This is equivalent to having fingers off the piano keys. We decided to NOT use the eyes with their up/down & left/right motions so that they can be used for the purpose nature intended them - seeing, reading. We also excluded the eye blinks since they occur naturally (even though they can be used for playing repeated notes) . What was left is the rest of the facial muscles including mouth, nose, eye brows, cheeks and tongue. Our current mapping choices are shown on the screen shot.
ย
Duets and Harmonies ๐ญ
The MusEmoji app can track two faces ๐ฅ at the same time. Just stand next to your partner so that the TrueDepth front camera can see you both. You can play the same or different instruments in the same or different octaves. Give it a try and enjoy. This is a great way to practice playing harmonies.
Practice Melodies
To help you master playing music with your facial expressions the MusEmoji app includes views of several popular melodies. Tap the ๐ถ button on the menu and swipe left or right to choose a melody. Each melody can be displayed in 5 different notations (SCR - score, DRM - solfege, CDE - American, 123 - Asian, OJI - MusEmoji). You can also watch a video of the chosen melody played by an experienced MusEmoji player.
The MusEmoji Game ยฉ๏ธ
Designed to help you master your MusEmoji music making skills, the MusEmoji Game is inspired by the well known Piano Roll, Guitar Hero , Dance-Dance-Revolution and even the original Mickey Egg Catcher and BeerTender games. To play it you do not need any hardware. This is entirely a software-based game featured in the MusEmoji app and controlled by your face.
ย
To start the game tap the โญ button. All notes types, which comprise the selected melody are displayed on the outer red circle as the hours on a clock face โฐ. There are 12 possible notes -the black and white keys comprising an octave. When you tap the start ๐ฌ button the notes will start floating to the center of the circle in the order in which they appear in the musical score of the melody. When an emoji touches the inner red circle - your target - you have to make the corresponding facial expression and keep it until the next emoji in the sequence touches the inner red circle. That will cause the note mapped to this emoji to play.
ย
Tap the level button 1๏ธto change the speed of playing. There are 3 possible levels/speeds - each one 2x faster than the previous.
ย
If the melody requires that you play a note in a lower or higher octave you will be prompted by a ๐ปor ๐บ icon on the bottom. Turn your face to the left or right to change the current octave.
ย
ย
ย
That's it in a nutshell. Enjoy! ๐
Connect
Get in touch with our World Wide Community of MusEmoji users and share your experience.
FACEBOOK
TWITTER
PHONE
EMAIL
TIKTOK
