Guest Post: Student Andrew Lipian attends the Video Game Scoring Workshop at NYU Steinhardt

I'm excited and delighted to have my second guest post of my blog's history, again by my student Andrew Lipian.  He's been working on this one for quite a while.  Again, I think it's great to hear about these current experiences for students who have attended them.  Here he describes his experience during the past summer in the Video Game Scoring Workshop at NYU Steinhardt.  

NYU Summer Workshop in Game Audio is a Wwise Choice  - Andrew Lipian

Before the snow melts, flowers bloom, and the school busses whir to a stop, up-and-coming game composers will be thinking of how to enhance their audio chops this summer. They need look no further than the Video Game Scoring Workshop at NYU Steinhardt, led by game composer Tom Salta and his team of fellow audio pros.  These included Gina Zdanowicz - founder of Serial Lab Sound - who lead students through the technical side of creating adaptive music systems in Wwise, and Jason Kanter - Audio Lead for Avalanche Studios in New York City - who gave a presentation on sound design and joined in discussions regarding the business of game music.  

Bolstered by such a well-rounded team, this program was unique from others. Composers usually sign up for similar workshops to master the machinations of Wwise or FMOD –audio implementation tools – but this program takes it a step further. As part of NYU’s “Screen Scoring Summer Workshop” offerings, students travel from global destinations such as Japan and Israel to attend a week-long intensive workshop offering one-on-one training in Wwise by NYU faculty and staff. During the week, however, the entire group actively observes each new iteration of their individual interactive audio projects which get critiqued and modified by Salta over the course of the week. I had occasion to audit the 2018 class, watching the students’ creative processes unfold until they caught lightning in a bottle, leaving with a fully functional and implemented game score. 

New York University’s workshop succeeded due in no small part to the inspired choice of NYU alum Tom Salta (HALO, Killer Instinct) to head the project, bringing his years of industry accomplishments to the fore. He shared his various experiences and career journey from touring with Pop music stars, record producing, transitioning to composing video game music to delivering his recent TEDx talk. Everything Tom imparted was solid gold, and he even walked students through his process of creating music for games like “Prince of Persia.” 

Notably, what Salta had championed with this program was unprecedented for NYU.  

“This new focus on music for games was unheard of back when I was here,” Salta explained. For the first program of its kind at NYU, though, it was a resounding success – here’s how it worked:

The students downloaded “Angry Bots,” the game they would implement their music into with Wwise. Angry Bots is a third person shooter with a solid-snake-like character, thanks to the creative voice overs scripted and recorded by fellow instructor, Gina Zdanowicz, infiltrating a Robot Laboratory as the lethal machines attempt to terminate him. Each code he cracks opens a new door, creating opportunities for the score to develop as he delves deeper into the labyrinthine facility. After also downloading Wwise, the students are then asked to begin sketching music for the game using their Digital Audio Workstations (DAWs). The entire class witnessed their peers develop and conceive music in a linear fashion – structured with an A and B section with a loop, for example – into horizontal resequencing, where specific in-game actions “trigger” the next music track. For Salta, watching the students master this concept was a highlight of the course. 

“This was the first time many of the participants ever ventured beyond linear composition -- as in music for TV and Film -- into the non-linear world of video game music,” Salta notes; “The music needs to account for all sorts of gameplay possibilities, yet still feel like a unified, continuous piece. Seeing a student’s eyes light up when they finally understand this is unforgettable.” 

The students – ranging from High-School to seasoned professionals – created scores as diverse as the nations they hailed from. One memorable moment occurred when a composer’s use of a trailer rise (consisting of several string instruments rising to crescendo with tremolos) surprisingly fit the action every time it appeared. Another came when a student from Japan envisioned a lighthearted, comedic element to the level – opting for a jazzy track smacking of something from “The Naked Gun.” Every one of these ideas changed the feel – even the identity – of the game; a reminder of the important influence music has in shaping the playing experience.

None of this would have been possible without the see-what-sticks approach of Salta and his team, who created a learning environment which encouraged inventiveness as much as it spurred technical mastery. All this was part of the plan.

“When I first spoke to NYU about putting this workshop together, my dream was to create a ‘bootcamp’ of sorts that would walk participants through the entire gamut of what goes into creating original music for games. I wanted a hands-on, interactive experience where students would be able to create original music from scratch and put it into a playable video game by the end of the week. Not just talking about the concepts but actually doing it.” 

Most assuredly, it was this atmosphere that lent so many international students their most formative experience in game scoring to that point – and that same atmosphere will continue drawing throngs of ambitious composers to NYU, ready to harness the powers of this interactive medium in immersing the next generation of gamers. These skills, Salta adds, are a must-have for those aspiring to a career in game audio. 

“For anyone coming into the industry now, it’s important to have proficiency in the entire 

game audio pipeline as it will open up more opportunities. Composers who know both the compositional and technical aspects of game music implementation will have a leg up on those who just know how to compose music.”

As the industry changes, the skills one is “expected,” to have, versus those which are “desirable,” shift. These days, if you want to work in video game music, you are expected to know Wwise or FMOD and understand audio implementation, a skillset this course specializes in teaching. 

When today’s composers are ready to take their skills to that next level, Tom Salta and the NYU “Scoring for Videogames” Workshop are there to give them the tools they need to succeed. 

*Program dates for the 2019 NYU Scoring for Videogames Workshop are June 30 – July 5. 

Current News: Day 1 at GameSoundCon

I'm just getting back to my hotel from the first day at GameSoundCon.  Great day!  Totally lifted my spirits from the fact that my flight in yesterday was delayed, the hotel oversold, and I was moved into a craphole where I feel like the sheets are going to give me Ebola.  As I walked into GameSoundCon this morning, George Sanger was lugging his suitcase up the steps in front of me and we started chatting.  When I got to the registration table, Alexander Brandon was passing out the registration information for the N-Z folks, which of course included me!  These are two great composers whose music I admire and whose books I love and have written about (George and Alex).  A great start to the day to help wash away the rough arrival yesterday.

After a mass meeting, I spent the morning in FMOD sessions with Stephan Schutze.  I felt a bit awkward initially when Stephan asked everyone how they identified as a sound designer or as a composer.  Then he asked who didn't raise a hand-- just me-- and I said I identified as a teacher of game music.  He went on a bit of a tirade about how there were a lot of "Bobs" who were teaching game audio and had no business or knowledge to be doing so and it was fraud.  Then quickly followed this with the caveat that he was glad I was seeking out the training that many didn't know they needed or care to get.  After that bump and a technical problem with the projector, the sessions progressed quickly as he showed many of the features of FMOD.  While the class wasn't the sort of FMOD 101 I was hoping for, it was interesting to watch what was possible in the software.  One of the main points he stressed was that folks used to linear media need to get out of the mindset of "get your sounds in ProTools, mix them, and export them off to the game."  Instead, technology today allows for the game engine to mix the sound interactively in the game.  Principles I understand already from readings, but cool to see how he applied them with the software.

After lunch at a gastro-pub, I headed to Alexander Brandon's session discussing his work on the Wasteland 2 and Pillars of Eternity.  Most cool about this session was hearing how he worked in Unity 4 on this game and used various techniques and softwares to conquer audio problems.  Alex seemed to be a fan of Wwise, which was not used in these games, but he is hoping to use in future games.  He also discussed the differences in updates to Unity 5 that are improvements to how the audio works.  While I haven't used Unity much at all except for exploring just a bit of Steve Horowitz' book, it was cool to see that the interface wasn't so much different from the Unreal demos I've seen in The Game Audio Tutorial.  At the end of this session, I was sitting close enough to Winifred Phillips to get her to autograph my copy of her superb book, which I've been reading on this trip.

I was planning to head back to the FMOD session for some talk about generative audio, but instead I headed down to chat with some of the sponsors.  There were numerous tables set up by different groups and products: IAsig, Wwise, RealSpace VR Audio, and ProSoundEffects.  I'm planning to join IAsig next year as this year is my year of GANG.  Wwise and ProSoundEffects are both things that I think could enhance my teaching at UM, so I chatted with them about possibilities for students and educational packages.  The hardest of these for me to describe is RealSpace VR audio.  They had headphones where the audio changed in relation to the placement of a second device around you, changing the mixing as you were stationary in the headphones and the device was moved around you.  Next, they had me put on enormous goggles and a second set of headphones that looked into a beautiful 3D environment and it was possible to move through the environment by looking at various objects in it and hear the sound change in relation to your position in the virtual environment.  I'd never seen or heard anything like that.  Definitely something to keep an eye on in the future!

Marty O'Donnell gave the keynote address.  I forget that he wrote that memorable Flintstone Kids jingle from my childhood.  Never knew it was his daughter who sang the final "and growing!"  I never played Halo, but of course know the music anyway.  I played a bit of Riven, but was way more into Myst-- didn't realize he got into the later sequels in this series.  Anyway, fascinating to hear about his journey and some of his advice.  I may use some of his points and examples in my own teaching.  His three main points were: 1-practice, 2-take risks, and 3-have friends.  Good advice in any field, certainly any musical one!  The end of the talk was a bit of a downer as he talked about abuse in the game audio field and how to protect yourself and your creations.  Again, good advice, but too bad that folks get burned enough to make that a front and center issue in the field.

All in all, a wonderful, tiring, busy day.  Looking forward to tomorrow with some Wwise training and audio roundtables.

Playing Games: South Park The Stick of Truth

It's been winter break in the last week for UM and Vince is out of town, so I took some time to play a few video games, and for the last few days, have been completely enrapt with South Park: The Stick of Truth.  As a die-hard South Park fan (I've seen every episode numerous times, own almost all of the DVD complete sets, etc) this game is amazing!  Any South Park fan will love it.  What's been doubly awesome is that I've enjoyed the game audio so much as well... very clever and craftily done!

I did a quick Google search to see what the top hits for SoT are in terms of the music.  Many of the top hits discuss problems with the music dropping out, the fact that the audio uses songs from the show (Sexual Harassment Panda, Taco-Flavored Kisses, Let's Fighting Love, etc), and that the audio generally is great or is not great.  Further down in the results came some more interesting ideas about the music and sound effects spoofing Elder Scrolls: Skyrim and Diablo.  Even further down in the results came mentions of Gregorian Chant and Middle-Earth.  What's so shocking to me is this: considering Matt Stone and Trey Parker have created two highly acclaimed musicals, South Park: Bigger, Longer, and Uncut as well as The Book of Mormon, the music in this game has, so far at least, gotten very little attention!    

I don't know that this post will ever be a top Google hit for the game music, but I took some notes as I played and would like to add some specificity to the discussion of its audio, and more precisely, its music.  Quite frankly, I find the audio in this game to be genius, and despite the flaws in the music at time dropping out, or the visual glitches/ audio lag-- which can really get terrible at times-- I think the use of music in the game is incredible and deserves special mention.  Here are some of my highlights:

First of all, the biggest thing to jump out to me about the game audio is the variety of it.  There are mock religious songs, Japanese TV-like themes, "elevator" music in Planned Parenthood, and fully orchestrated music at times contrasting with 8-bit sounds.  How many games have this variety of music as part of the standard gameplay audio?  Also, the music is cleverly used in various games, puzzles, and attacks during the gameplay.  I'd say the use and and incorporation of music in this game rivals a Zelda game-- which from me is high praise!  

Probably my favorite moment in the entire soundtrack is when you enter Canada.  The music immediately changes to a two or three part 8-bit soundtrack, and the sound effects are reminiscent of Dragon Warrior.  Check what I mean below in this video around 16:23-17:23.  Not only do you hear the O Canada two part version, but also the music when entering the cave is very NES Wizards and Warriors.  

There are also musical games as well.  For instance, when you're on the alien spacecraft, there's a "Simon" like game where you follow musical combinations to release Randy.  Check out 17:00-17:45 for a funny version of this Simon game.  

Another clever use of typical game music gaming is the DDR like moments when you are convincing the Goth kids to help you or when you use Jimmy's Bardic Song powers.  

I haven't played Skyrim, so I can't comment on references to its audio in this game music, but I definitely hear Diablo in the Storage Cellar.  The guitar strumming reminds me of Tristam's theme.   Listen below from 3:43-4:00 to catch this reference.  Also notable to me are references to the Hobbit/ Shire theme of the Lord of the Rings movies.  You can hear that in the video below as well from 34:22-34:43.  Compare with the LOTR theme in this video around 5:08-5:22.  Both the instrumentation, the shape of the melodic line, and the rhythm of the melody share similarities that make the Stick of Truth music below recall it.  

Many webpages have mentioned that you hear snippets of the show on TVs and songs from the show on the radio.  But it's better than "just" this...  Not only do various televisions and radios play songs from the show, the most clever part of that audio decision is that you can choose to turn on/off many of these devices, adding a level of realism to the audio environment and letting you sculpt certain moments sonically.  

My favorite of the pop song references is one the Goth kids are listening to, All In Your Head.  Being about the same age as Matt and Trey, I immediately heard this as a reference to the Cranberries 1994 hit, "Zombie."  While the beginning of the songs are virtually identical, for me, the refrain is what struck me and made me think to look up Zombie.  Thus, particularly compare 1:00-1:30 below with the main refrain of Zombie-- in the link above, that's 1:34-2:00.  

Now to be fair, the audio is not all perfect...  If I had to pick one moment as an audio fail, it would be the generator in Professor Chaos' lab.  The sound effect there is just clunky-- as if being a small distance across the room would obscure how loud the generator is when you're close to it?  A nice try by the sound designers for realism, but it's overdone and made me laugh when I first heard it.  I noticed in the credits that the audio is integrated via Wwise and after recently playing Epic Mickey as well, I'm wondering if this is just a clunky part of Wwise's integration.  I've not tried it before, but the max/min radius with this generator sound is simply too small and crazy in the storage unit.  

If I've piqued your interest, check out the soundtrack, which I can't believe is already posted online.  I didn't hear the O Canada theme in this, just the "Blame Canada" music, so I'm not actually sure this is the complete soundtrack.  Still, it's cool to be able to hear this music.  I'll leave you with one of my last favorite moments, which is the Japanese TV theme like music that plays toward the end of the game.  In this video, that's around 18:25.  Enjoy!