Berklee Online Brings Ear Training to Virtual Reality with New Game

A common misconception in the music community is that a “good ear” is something that you are either born with or not. The truth is that it’s a skill you can develop through ear training—and Berklee Online has spent more than a decade helping students sharpen these skills through ear training courses and certifications. Now, with the launch of a free virtual reality (VR) Ear Trainer in the Meta Store, the possibilities are more immersive than ever, regardless of whether or not you’re a Berklee Online student.

“We really wanted to prove that everyone is capable of developing an ear,” says Berklee Online Chief Operating Officer Michael Moyes. “While there are already many great tools out there, we wanted to create a virtual reality experience to help the world have fun with this. It’s not just about practicing to get a good grade—it’s about deepening your appreciation for music and your skills as a musician.”

What is Ear Training?

Ear training is the process of being able to identify, internalize, and replicate musical concepts such as pitch, rhythm, and harmony. Just how you become fluent in a new language by learning how to read, write, and comprehend conversation, ear training allows musicians to increase the speed and agility of their musical communication. 

“Ear training is exactly the same as learning a language, except that the language is music,” says Gilson Schachnik, chair of the Ear Training Department at Berklee. “It’s learning to recognize sounds and translate them so you know what they are, the same way that when we speak, you recognize the sounds, your brain processes at lightning speed, and you know the meaning of what I’m saying.”

As we know through countless examples, some musicians don’t read music but have a fantastic ear, and some musicians read and write notation fluently but don’t have as strong of an ear. This doesn’t stop them from creating some of the world’s most influential music. However, ear training does provide musicians with advantages and ease with their playing, allowing them to understand the whole picture.   

“If you’re going to be playing music with other people or even solo, ear training is an invaluable skill,” says Moyes. “If you think about music as a language of communication between you and an audience, or you and other musicians, being able to hear what’s going on and respond in the moment is an essential aspect of musical mastery.”

Berklee and MIT 

The idea for the VR Ear Trainer started back in 2021 when Moyes took a virtual reality course at MIT. He was interested in how this new technology could build upon Berklee Online’s history of innovation, which all started with the concept that you don’t need to be in person to receive a quality music education. 

“I always want to figure out what that next thing is as far as emerging technologies and virtual reality felt like the next frontier that could take teaching to new places,” says Moyes. “I wanted to start from the ground floor, learning how to do some of the programming, checking out the hardware options, and seeing what other creators are doing.”

After the MIT course, Moyes started contacting the developers of some of the most popular music games. 

“I essentially slid into their DMs and ended up connecting with this incredible range of people, from entrepreneurial teens to major video game developers,” Moyes recalls. “It showed me that anyone, at any level, can step into this space and create something truly amazing.”

Berklee Online’s CEO Debbie Cavalier connected Moyes with Berklee alum Marshall Chadbourne, who has a background in both game development and music. Together they came up with the initial idea for the Ear Trainer. 

“He and I came up with a loose concept for an ear training game and started building it in Unity,” says Moyes, mentioning a software for creating VR experiences. “It started with very basic physics and gameplay—more like rough sketches in a drawing. From there, we added sounds and started play testing. We kept it simple, using a major scale in the key of C as the foundation. Players would hear a note and identify it, selecting the scale degree they thought was correct.”

A Multi-Player Process 

Once they had a prototype, Moyes held a focus group at Berklee Online’s annual Onsite music conference in Boston. Attendees were encouraged to put on a VR headset and play the game, which was also projected on a screen so the audience could see. Participants then answered survey questions like, “Was this helpful to you?”; “Would you use this to practice on your own?”; and “Have you ever used virtual reality before?” Out of about 40 participants, only one had ever tried VR before. The feedback was overwhelmingly positive. 

After Onsite, Berklee Online began working with a former engineer for Harmonix (best known for developing the game Rock Band), his expertise brought a new level of polish to the game. From there, Moyes recruited more Berklee Online staff members, including Chris Kinney and Patrick Waltman. Both Berklee alums and gamers, they knew Berklee’s ear training curriculum well and helped develop the scoring system and gameplay design. Moyes says they were key in making the game engaging and building a good progression system. 

“Part of the experience that I really liked was the bit of pressure it added to your training,” says Kinney, senior director of academic advising and frontline admissions. “I think back to my days on the Berklee campus. Having to perform ear training exercises in front of classmates really made sure you owned what you knew. So, having notes coming at you, or any sort of additional tension or stressors, really presented an opportunity where you couldn’t fake it.”

Once the game felt ready, Moyes shared it with Schachnik. He says he can see how the gaming process—starting, losing, and starting over with the new skills that you’ve gained from losing—can help you learn. 

“I always thought if we could approach ear training the same way, wouldn’t it be phenomenal?” says Schachnik, who also has gaming experience. “Imagine if people were addicted to an ear training game. Instead of it being a chore, it’s fun. So of course, when the VR opportunity came, I jumped in and said, ‘I want to test it.’ I couldn’t wait. ”

How the Game Works

The VR Ear Trainer has three modes: Rhythmic, Harmony, and Scale Degrees. Rhythmic mode provides players with two mallets and a drumming surface. A rhythmic notation appears on the screen along with a sound sample tapping along to a metronome. The player is then asked to tap the rhythm back using the virtual drums, gaining points for every correct passage they can successfully complete. 

“Whenever you think about XR [extended reality], it’s important to think, ‘What makes this better than just doing it on an app?’” says Moyes. “For the rhythm mode, it’s really the physical immersion. You have a controller in each hand and you’re using those as mallets or drumsticks, so you’re actually playing the rhythm and that makes it a lot stickier than tapping it, or simply thinking it.”

Harmony mode is all about identifying the relationship between multiple pitches. Level 1 starts with two notes (intervals), level 2 has three notes (triads), level 3 includes four notes (7th chords), and so on. With the option to play the root note, the game will play the various pitches first separately, and then together. The player must identify the correct harmony out of multiple options by tapping it with a virtual conducting baton. 

“Then there are other levels as well, where you alter these notes and get really funky chords that are heard more in fusion, jazz, or complex film scores,” says Moyes. 

Scale Degrees mode will provide you with all the notes within a scale and the player must determine which note is being played using solfège. You can practice major, minor, or modal scales in any key. 

What’s Next?

The Ear Trainer is Berklee Online’s first official foray into virtual reality and serves as a proof of concept to continue exploring this technology. Moyes has a list of ideas, like having VR Beethoven teach a lesson on composition, or taking a film scoring course and conducting your original piece with a VR orchestra, and overcoming stage fright in front of a virtually-packed audience. He also sees VR democratizing music education in correctional facilities (another initiative he is involved in). 

“We have plenty in the pipeline,” says Moyes. “There are a lot of ideas that are coming out of our community, whether it’s our instructors or the faculty at Berklee or our students.”

Lori Landay, a Berklee professor and author of several Berklee Online courses, including Game Design Principles, has been working with VR for the past decade. She has ideas for a VR circle of fifths game and VR recording studio, where you can save money on studio time. She has even held some of her online classes in virtual reality. Whatever new applications are on the horizon at Berklee Online, Landay emphasizes the importance of educational innovation. 

“I think educators should be open-minded to virtual reality and emerging technologies because our students are increasingly immersed in media,” says Landay. “They are much more comfortable with video and audio and they live in a world that is comprised of screens in a way that is not true of most of their teachers’ generation. In order to understand that and teach to that, it’s important to innovate and explore the possibilities of immersion in educational experiences.”

Since the VR Ear Trainer hit the Meta Store, it has received hundreds of downloads. For those who want to try the game, but don’t have a VR headset, Berklee Online is in the final stages of releasing a two-dimensional app version that will be available in the iOS, Google, and Android stores in early 2025. 

“It’s been gratifying to finally have something out there, but what I’m most excited about is what’s next,” says Moyes. “This is just a baby step. Now that we have seen that people like our game, we have justification to go further!”