role: conceptual researcher, prototyper
software: Unity, MIT Singularity SDK, Arduino IDE
hardware: Quest 2, ESP 32

timeframe: 2.5 days

inspiration

The current state of VR technology focuses on creating an immersive experience through sight and hearing. However, as of today, there has not been a VR device that stimulates the jaw, even though eating and food are such integral parts of the human experience. At the 2023 MIT Reality Hack, we created a futuristic jaw haptic device that simulates eating in VR. The haptic jaw consists of two parts- an adjustable jaw brace and a harness. The jaw brace is made of aluminum wire and rubber bands, and can be adjusted to fit the facial profile of different users. The jaw brace is then attached to the harness via springs. The tension on the springs can be adjusted via motors. This means that when a user “eats” a particular food item in the XR experience, depending on the texture of the food item, the tension in the springs are adjusted automatically. This simulates the extent of difficulty or ease in chewing different items in an XR experience.

solution

We created a futuristic jaw haptic device that consists of two parts- an adjustable jaw brace and a harness. The jaw brace is made of aluminum wire and rubber bands, and can be adjusted to fit the facial profile of different users. The jaw brace is then attached to the harness via springs. The tension on the springs can be adjusted via motors.

This means that when a user “eats” a particular food item in the XR experience, depending on the texture of the food item, the tension in the springs are adjusted automatically. This simulates the extent of difficulty or ease in chewing different items in an XR experience.

technical aspects 

I’llByte was built in Unity 2021.3.16f. When the VR controllers interact with a food item, the program sends a bluetooth message via MIT’s SDK The Singularity. Then the SDK connects with the microcontroller on the back of the harness, which spins the servo motors attached to the front of the harness. As a result, the springs attached to the jaw harness are pulled, creating tension dependent on the VR experience. 3 different tension modes are set up for the prototype VR game, creating 3 different “textures” of foods.

challenges 

One of our biggest challenges was being a team of four designers and one game developer that came later down the road. This came with its pros, as we were able to refine the fabrication of our prototype, but this also meant that the development was very slow. Coding detailed interactions in unity with four designers was extremely challenging because it requires a lot of C# coding and only one of us knows C#. Most of our time was spent on testing The Singularity SDK as well, but luckily, we were the first team in the hardware track to successfully test it out. We had intended to use the Ultraleap so that we could use hand tracking instead of controllers, but the Quest 2 cannot connect with Ultraleap easily. The Quest 2 cannot connect with our Macbooks very well either.