Surveying Literature: Maestro Mario: How Nintendo transformed videogame music into an art

A couple days ago, I finished reading Andrew Schartmann's new book, Maestro Mario: How Nintendo Transformed Videogame Music into an Art.  I found out about Andrew's book through my google alert.  Ironically, the alert was describing an upcoming book he's working on for 33 1/3.  I saw the title Maestro Mario in his bio and wanted to read what he'd already published!  I don't know Andrew at all, but would be interested to meet him and talk with him about his work in the game music area.  Regarding the book...

First, the good...  

I love the idea that anyone is writing about game music and trying to engage with it.  It's clear that more people are getting in to this field and feeling comfortable putting their interest and time into studying and working with game audio.  Very cool.  Thus, my number one thought having read the book is that I'm happy to see serious study of game audio.  Andrew is a clear, engaging writer whose prose is never over the head or condescending with that typical academic lexis.  I'd qualify it as an easy read, actually.  Enjoyable, quick.  Easy to dip into in you have five or ten minutes before bed or a meeting.

My favorite things in the book were following the footnotes.  (This is how I know I'm a complete nerd in game audio.)  There's a great interview with Koji Kondo as he talks about his early work with Nintendo.  I also loved learning about the existence of The Arcade Flyer Archive.  I've been checking out numerous of these.  Also, Andrew spends a good bit of discussion looking at some great online writing by Neil Baldwin.  Academics must tap into the industry experts, particularly those who want to share methodology and historical stories.  The time to get this information is now, before more of the folks die...

Really liked the discussion at the end of the book about chiptunes.  This was the most unique part of the book for me, probably in large part because I know the least about this offspring (or precursor?) of game audio.  Learning about and hearing Kraftwerk and Yellow Magic Orchestra was awesome.  Some free chip tune software, OpenMPT and Milky Tracker.  Glad to encounter this.

Two other things I enjoyed were the focus on Metroid and The Legend of Zelda.  I don't know where these games have received as much attention as Super Mario Bros, which seems to need almost nothing new said about it.  I also really enjoyed the sound effect vs music discussion.  I didn't know the NES games Eric the Viking or Magician.  Definitely impressed to realize real bird samples are used in Peace in the harbor.  Or how the cricket sounds weave in the texture of the Meryl forest.

Now the bad...  

In terms of content, though, the book was a bit of a disappointment for me.  Granted, I don't think I'm the audience he has in mind for the book-- I've already done a lot of game audio research and work.  For me there wasn't a lot new here; this is more of an introductory work.  In Andrew's own words, he's trying "to bridge the gap between the ivory tower and the video game den."  For instance, the obligatory opening chapter with a brief history of game audio.... I'm so over these.  I know it's the academic tradition to show that you're aware of what else is out there, but there are SO MANY of these history of game audio chapters, articles, etc.  My thought is if you don't have anything new to add to the history (is there really much to add to those already written?) then don't write yet one more.  The one caveat I have is that I had not considered some of the market influences, business aspects, stock values, and the like that Andrew writes about here.  However, that's a very brief part of the history.

And the ugly...  

Three things caught my eye that I took issue with.

1- Asteroids and Space Invaders "employ the exact same principle" regarding game audio.  I couldn't disagree more.  In fact, I play examples of both of these games in my class and point out how much more clever Space Invaders is.  With Space Invaders, as the music increases, the pace of the gameplay also increases, peaking at the end where the aliens are coming down as fast as possible and shooting the last one is the most difficult task.  With Asteroids, the music peaks toward the end of the level, whereas the gameplay difficulty peaks around 2/3rds of the way through the game.  The end of an Asteroids game is, for me, typically trying to hit one last damn asteroid that is proving monotonously difficult.  Meanwhile, the music is so active it usually pulls me out of the gameplay as I feel the disconnect.  Thus, the audio in these two games doesn't use "exactly the same principle."  Instead, Asteroids compares with Space Invaders as a clear example of a copycat game not being as innovative with its connection in gameplay and audio in a critical way.

2- At one point in the discussion of Super Mario's music, Andrew writes that the water music is "incidentally" in C major, the same as the overworld music.  I was shocked to read his choice of word here.  Music theorists are typically coming up with all kinds of reasons why the music in the 3rd movement of a piece has a key relationship with the development of the first movement, or whatever. (Don't worry if that didn't mean anything to you!)  The point is, every piece of game audio is created with a series of explicit choices.  As I've already seen from charting the Super Mario Bros games in "My Gaming Audio History," Koji Kondo goes further than almost any other composer I've encountered to establish a "key" for the game, C major.  It's not by happen chance, circumstance, or "incident" that the water music is in the same key as the overworld music.  It's deliberate.

3- Finally, it was cool to see some of these game music themes transcribed into standard music notation.  I'd heard from Karen Collins that one is supposed to get permission to print transcriptions of game audio and that it's not always easy to do so, but Andrew has numerous examples printed here.  I'd be interested to learn more about how he got those permissions.  The one that is etched in my mind is the overworld theme for The Legend of Zelda.  Andrew put's this transcription in a key with five flats (Db major or Bb minor are the choices, then).  I think anyone who has ever played this game would consider the key of the overworld music to be Bb major.  Maybe he did this for ease of accidental printing, but I got lost in thought wondering how he decided to present the theme in that key.  If I'm right, that it was for the purpose of ease (otherwise, I can't figure out why he'd be trying to justify either of the keys the signature suggests), I'm disappointed.  One of the cool things about game music is that it usually doesn't exist in printed form and I think it's all the more important then for transcriptions to reflect it accurately, not conveniently!


Let me know if you read this book and what you thought of it.

Surveying Literature: The GAT Bonus Videos

I know I’ve been a but slow with updates recently, but I’m in the final week of my summer work and that means show season.  The last two nights have been performances of different shows I’ve been music directing and playing, so that’s a lot of time an energy consumed in a different portion of my musical life.  This week, I pick up new shows where I play supplemental keyboards or percussion, so that’s learning music in a hurry and working on the fly.  I love it!  Anyhow, I’m wrapping up things here and will be back home in about a week and then plan to spend a lot more of my energy burying myself back in game audio as I enter final preparations for my game music course in the fall.

Today I finished watching some of the supplemental videos for the Game Audio Tutorial.  Watching these videos has reinforced for me how much I’m interested in working through the exercises in the GAT.  Part of the problem I’m up against is that I own a Mac and Unreal runs on PC...  but, I’m supposed to be getting a new computer from the University in the fall, so I’m planning to ask them to split the hard drive so that I can work some of the GAT exercises in Windows throughout the semester.  I’m actually thinking of proposing a second game audio level course to the university as a joint offering between the SMTD and the Engineering School that uses this book as a text.

These video tutorials are incredible.  One of the ideas I remember encountering in Aaron Marks book is the importance of learning from experts.  Of course, we say the same in the Music Tech course I teach; the recording specialists who come in and give interviews always emphasize that you learn from being around experts.  Just watching people who have great experience work can teach a tremendous amount, and fast!  These videos are just over three hours of exactly that.  It’s as if you’re in an audio studio watching an expert implement audio into a game as he explains why he’s doing it.

Some of the most vivid memories I have from these movies are simply watching the first ambient room tone being implemented, triggering a proximity alarm, seeing the number of cues needed to create a varied rainy outdoor ambience, the work with gunshot reverb, and the ducking of ambient outside noises in a building.  And these are just considering sound effects!  The mixing section toward the end of the videos was a bit beyond what I’m used to, but I can appreciate how specific a skill it is to change the program and allow for interactive mixing and the situations where that needs to arise.  I also appreciate more than ever how precise of an ear is needed for great game audio work.  I had to rewind and relisten to some sections of the videos to hear how minuet the audio changes were at times.  I can definitely hear them, but I better understand that the game audio ear needs to be highly skilled and experienced to know what to listen for.

Considering my focus on game music, I found the most interesting videos to be 11-15, and the final video so far, 21, all of which outline some of the problems and solutions for interactive audio in games.  I understand conceptually and musically how stems must be composed and looped, building or muting certain tracks depending on the game conditions; what I learned from watching these videos is how that actually works implementing them in a real game setting.  Now more than ever, I can appreciate why game composers need to understand this implementation process.  To understand this step in the game audio process certainly influences the composition stage.  Further, seeing these implementations made me consider more deeply how game music works for the player.  I don’t know if I had ever considered how building music cues can help a player navigate through a maze, but I’ve probably experienced it and similar game audio functions.  I’ve not actually heard the difference between music being triggered with a hard cut versus music that changes at a certain musical beat in a game until now.  Many of these tasks I understand and have encountered from other musical/audio/sound work I’ve done, but it’s different to see them in a game setting.

Basically, like many of the things I’ve read, listened to, and watched, this behind the scene glimpse really helped me to know what to listen for as I study and encounter game audio.  While all the work in the GAT is focused on major console audio considerations, not so much more simple mobile game considerations, this knowledge will help me as I encounter major games in my work.  I’m probably a fair while away from encountering a game this advanced in My Gaming Audio History, but I’ve got my ear more open for these scenarios.  These videos rock!  I’d love to know where I can find more videos of game audio experts at work.  Leave me a link if you know of one.

Surveying Literature: IASIG Game Audio Curriculum Guideline

Yesterday I read through the IASIG Game Audio Curriculum Guideline.  The main thing I thought as I read it was: how cool that this exists and that I created a class not too far outside the guidelines of this syllabus!  I think my class is a blend of mostly the Video Game/Game Audio History class with a little of the third year Game Music Analysis Class.  I'm thrilled to see that both of the documents suggest having guest speakers as a part of the class, since I'm working right now on a grant application as an honorium for guest speakers in the class.  I'm definitely going to apply for some grant funding to join IASIG in the next year and learn more about what they've got available and discussions there are like, etc...  But this document is available online with no special membership needed.

The highlights of the document for me were several, which may just sound like a summary of it, but I'm very glad that I read it.  First, I loved the list of positions working in game audio.  There are nine listed under sound design, eight under the category of musician, ten under voice/dialog/localization, five under audio management, three under programming, and one under QA-- essentially a way into the field.  If that's the way to get a foot in the door, I'd be interested in such work.  Maybe a way to dip my toes into the water and see if I like the temperature!

Also very cool to see the table listing skills needed for the various positions listed above (music, sound, voice/dialog, and management).  I wonder how these skills change depending on the size of the project.  For instance, the table makes it appear that implementation skills aren't very important for a music position-- and maybe not in a huge project, but in a smaller project, the composer might create sounds and implement them into the game as well.  As Aaron Marks would remind, the more you can do yourself, the more money you can make and the more control you have to make sure things are done the way you want!

I'm also struck by the fact that there are so few game specific books listed as recommended texts in the syllabi.  Creating sound effects sounds very similar between film and video and games.  However, there's no reason not to create texts that about sound effects specifically for games.  While the creation process is probably very similar, the implementation and end result is no doubt different.  Undoubtedly the same is true for recording techniques, etc.  One thing I really hope will come out in the next few years is a text by a good game composer about the music creation process-- explaining planning for interactivity-- both musically, practically, and technically.  We need more game specific texts, and not just generally, but with game audio as well.  In any case, the recommended texts that are game audio specific are:

Karen Collins: Game Sound.
Alexander Brandon: Audio for Games.
Rob Bridget: Post-production sound: a new production model for interactive media
Rob Bridget: The Future of Game Audio-- Is Interactive Mixing the Key?
Aaron Marks: The Complete Guide to Game Audio.
Wwise manual: audiokinetic.com

The Bridget writings are now on my reading list, and I'm pleased that I've already read the Collins, Brandon, and Marks.  I spent quite a bit of time playing around on the Wwise website and also added several of their manuals to my reading lists, their Fundamentals Manual and Integrating Audio Guide.  Nice to know that academics have access to Wwise for free!

One of the most common threads I'm really struck by in all this game audio reading is that there's a huge push in every document of the need for (what is here called) "soft skills."  Interpersonal skills, the ability to get along well, work on a team, meet deadlines, be kind to people.  Classical music definitely has a bit of diva-ism about it, but these skills are-- in reality-- needed and possessed by most folks who are successful and working in the field.  But the prevalence with which game audio discusses these features as being important keeps me wondering what working in the field is like?  As a person with a degree in collaborative piano, the idea of not being a team player is crazy, but I'm beginning to wonder if that's a common attitude.

Surveying Literature: The Complete Guide to Game Audio (Aaron Marks)

For the last few weeks I've been pouring through the second edition of Aaron Mark's Complete Guide to Game Audio.  This book is an absolutely incredible resource for anyone considering working in game audio.  It lays out skills and tools necessary in the business, creating music, sound effects, and mixing the audio experience of a game, and also goes into great detail about business expections: where to find jobs, how to negotiate contracts and what standard rates are, branding/marketing an audio business, preparing a demo portfolio, etc.  Reading this book was an amazing experience that I've been enjoying for the last month or so-- and completely worthwhile.  From a historical standpoint, it's wonderful to have everything written down so clearly from an insider perspective working in the business; from a practical standpoint, it's exciting to have everything presented so clearly as steps that can be completed.  Reading the book even made me think that I could create some game audio!  Maybe I'll start getting my game audio portfolio ready...

Undoubtedly, the coolest features for me are the audio examples on the accompanying disc that comes with the book.  I knew early on in my video game music class that I wanted to have the students create some examples of video game music, but I was only thinking of it as a fun, creative exercise.  I hadn't considered (until I read this book) that creating game audio examples was actually practical too; it's the starting of a portfolio-- or at least getting students thinking about making one.  All the exercises suggested in the book, like making your own audio for a video of gameplay, or looking at artwork or having text descriptions of game situations and creating audio, are exactly the kind of final project exercises I'm going to have in my class.  It's awesome reinforcement to hear from a game audio expert that these are practical, worthwhile steps for those interested in game audio.

I'd definitely say this book is inspirational... It made me contemplate if I could compose game audio!  Anyone serious about getting into the industry, or who just wants to see the profession from an insider perspective should give this book a read.  It's definitely one of the best books I've read on this journey so far.

Here are a few other thoughts I had as I read the book:

I mentioned earlier when that I bought The Last Story primarily because the music was by Nobuo Uematsu.  Here I learned that Bobby Prince also enjoys that distinction on box covers.  I'm interested to learn who was the first game composer to be featured in that way.  Did s/he negotiate that?  Was it part of the marketing of the game?

Many of the books I've read focus on creating audio for big, blockbuster games.  This book does have a blend and mentions some portable gaming, but still, it seems that the focus is on the high end.  Early on, Aaron mentions that the current trend in game music composing is orchestral, but that's probably not the case on iPhone games, for instance.  I'm interested in learning more about composing for mobile devices because there isn't much written about that yet and it that might be an easy (and quickly evolving) way to get into the field.

I also particularly liked the section of the book about "getting in the mood" and "setting the scene" when composing game audio.  All the advice about watching similar movies, playing games in the same genre, etc for inspiration...  I loved to get this practical advice.  The idea of creating a sound palette was very interesting... one I wouldn't have necessarily thought of right out the gate, but that's the point of all the tips in the book!

One of the coolest aspects of the book is the list of composers, software, hardware, instruments, sound libraries, techniques, etc. that are sprinkled through the book.  I've recently been grappling with the idea that most music is organized by composer, and have been wondering how that might be possible in the game medium, where genre, platform, and game title have (perhaps) more weight than organization by composer.  Thus, in my attempt to become familiar with more of the folks working in the field, here's an interactive version of Aaron Mark's featured/interviewed composers:

Lennie Moore
Mike Brassell
Jamie Lendino
Richard Jacques
Alexander Brandon
Fernando Arce
Eric Doggett-- had trouble finding a great link, so here's a link to Moondog Media, he co-owns
Mark Scholl
Tom Salta
Watson Wu
Adam DiTroia
Darryl Duncan
Chris Rickwood
Henning Nugel
Jon Holland
George "The Fat Man" Sanger
Tim Larkin
Tom Graczkowski
Aaron Marks
Ron Jones
Keith Arem
Kristopher Larson
Will Davis
Nathan Madsen
Chance Thomas
Matt Piersall
Tim Rideout

It's cool to see that many of these musicians are also conductors, teachers, film composers, etc....  Game composing is just a part of what some of them do!  This list is also all US or UK composers.  Hopefully soon the language barrier will be broken and allow for more composers from outside of the English speaking part of the world to be in a collection like this with similar interview questions answered.


Magazines he suggests subscribing to (that are still in publication):
PC Gamer, Game Informer Magazine


Great websites:
gamesindustry.biz
gamasutra.com
gamedev.net
music4games.net
gamespot.com
gameindustrymap.com
ign.com
gamespy.com
gamejobs.com
audiogang.org
gameaudioforum.com


Best game audio conferences:
GDC (Game Developers Conference)
Project Bar-B-Q
Audio Engineering Society

Surveying Literature: The effects of music choice on task performance (Gianna Cassidy and Raymond Macdonald)

I took a break for a bit today from Aaron Marks' Game Audio to dip into something else and read Gianna Cassidy and Raymond Macdonald's "The effects of music choice on task performance: A study on the impact of self-selected and experimenter-selected music on driving game performance and experience."

Much as the title suggests, the article basically details the process and results from an audio experiment that tested how accurately and quickly the study participants were able to complete three laps on a game course.  They had choices to hear as they drove: silence, car sound effects, or car sounds with: music the participant selected, "high arousal" music, or "low arousal" music selected by the study designers.  Unsurprisingly, the study participants enjoyed the game the most when they were listening to music of their choice that they had brought to the study.  What did surprise me was, in this study at least, that performance improved when participants played the game listening to music of their own choice.  Conversely, they did the worst (drove faster and made more mistakes) when the listened to "high arousal" music.

I really had two main thoughts as I read:

1- clearly music can affect player performance in a game.  While this study focused on a driving game, at some level, it should apply to other genres as well: platform games, strategy games, RPGs, etc...  By extension, this makes the selection of game composer and game soundtrack incredibly important.  The game audio could determine whether or not the player is successful in a game and whether or not they finish a game, really get drawn in to the series, etc.  Any other research on this would be incredibly interesting.

2- As they ask at the end of the article, could certain kinds of music improve driving in the real world.  More research into this area could actually improve driving, safety, and help keep people alive.  This definitely warrants more investigation, though I doubt people will ever want to be forced to listen to a certain soundtrack in their cars.  It seems from the study, though, that music of the driver's choice could be the most helpful for driving accuracy of all, although a close second might be no music, with just the sound of the car on the road.

This made me wonder: what would I listen to as driving music if called to be in a game study?  I'm sure my selection would change depending on the day, but today was thinking that Steve Reich's Music for 18 Musicians would get me focused and in the zone for a driving game experiment.

Surveying Literature: Thematic Unity Across a Video Game Series (Jason Brame)

For the last week and a bit, I've been travelling all over the eastern US visiting friends and family.  Tomorrow I arrive in Lenox for six weeks of summer teaching. I'm hoping to get into a good routine there that allows me a little time each day for video game music studies, although my schedule is crazy with that work. I'm especially planning to work through various articles and books I've brought along for the summer.  Aaron Marks' book has me enrapt right now.  All this is to say I'm sorry I haven't been able to do daily updates recently, but I've still been working, thinking and talking with others about game music and my game music class, and imagining future blog topics. If only time and energy were infinite!

An article I've been really excited to read is Jason Brame's "Thematic Unity Across a Video Game Series."  My insomniac night gave me the perfect chance last night. Koji Kondo is simply one of the greatest living game composers and will undoubtedly be a historical giant in the field. I love not just the music of the Zelda series, but also the importance music making has during gameplay. In any case, an article that presents a musical analysis of Zelda themes was an incredible find and I've been excited to read this article since I found it a few weeks ago.

Perhaps the best thing about this article is that it presents a serious, musical theoretical, Schenkerian analysis of Zelda themes, particularly the Overworld Themes in several games in the series.  It's interesting to see these themes explored in such a classical method.  While this kind of analysis suffers, in my opinion, when appearing in a printed medium (it works much better as an audio or visual recording where the material can be presented more interactively and most importantly heard), for the classically trained musician, this paper is familiar, comfortable, and easily understood.  Bravo to Jason for treating this music in such an esteemed, classical way with no apologies! It's well deserved.

I really took away two main ideas from this article.  The first was the idea of Jason's game score graph.  I love seeing this brilliant visual representation of the way that inactive music can work.  He's developed a system of representing the music as looping or non-looping with boxes or circles, and what music can lead to other music within the game world with arrows.  I'd not considered this kind of analysis and it's a simple one that's easily comprehended.  I'm not sure if composers think of these relationships this explicitly such that they map them out, but I do know from my readings that they do consider these relationships at least some of the time.  I can't wait to show this kind of thing to my students.

The second idea I really appreciated in this paper was the systematic method by which Jason compares musical themes across the various games.  I've been looking forward to exploring game music through a game series as I continue My Gaming Audio History series on this blog (both in series with the same composer (Zelda, FF), the same series of games with different composers (Mega Man, Castlevania), and the same composers in completely different games (David Wise, etc)), but haven't made it far enough to have had those relationships develop yet.  The Schenkerian method is a perfect vehicle to demonstrate these comparisons of compositional structures, and for classical musicians, easy to comprehend.  Jason's simple table summary clearly represents the musical similarities and differences between the game themes.  Plus putting this music into standard notation helps the classical musician to better visualize it.  I also appreciate that he takes time to tie the music to its in game meaning.  Certainly this is a critical step in game music analysis, since the music is tied so strongly to game action.

I'm certain I'll present this as a possible reading in my game music class. While the target audience is clearly classically trained musicians and I doubt many in the class will be drawn to it, I've got a handful of music students in the class.  It gives me an almost Dali-like feel to see game music treated so classically and methodically.  This article really got my mind going!

Surveying Literature: A few short articles

While insomniac last night (perfect evening before a road trip), I read through several short articles I've had saved on my iPad. Nothing groundbreaking in them, but I did want to catalogue that I read them.

"Game on!"  Dominic Milano.  Keyboard magazine makes two appearances on this list.  Maybe keyboardists are just naturals for game music; Tommy Tallarico would agree! This is definitely the most interesting of these short articles to me. It reinforced for me my desire to meet Tommy Tallarico. This short Q and A with him clearly has a target audience of professional musicians. He also lists important websites I want to check out: idea.org, audio gang.org, and gamasutra.com. While a short article with ideas I've encountered before by this point in my research, it's always good to hear one more person reinforce audio software, interactive audio principles, and ways to get in the game audio scene.

"Video Games Killed the Cinema Star. Karen Collins." A brief defense of the academic study of game audio. It's a pity this is necessary. Most interesting is where she points directions toward future research. Some are being fulfilled already since the publication of this article (2007). I should probably browse this journal, Music, Sound, and the Moving Image, to see what else I might find. I like the list of popular music pieces related to games. I want to check out these and others.  Don't think I've listened to Weird Al's Pac-Man before:

"Les Claypool on Making Music for the Mushroom Men Video Games."  Q and A with Les Claypool.  've been wondering what sort of instruments game composers play and/or were trained on. Here's a bass player who composes game music. He discusses some of his techniques. I want to check out these games and give the albums with the music a listen. Definitely a good example of pop music/ game music synergy. And, as a big South Park fan, Les Claypool created the theme song for the show-- it's cool to see musicians who work in many mediums.

"Bloop Bloop, Hey, Bleep Bleep: Emulating Video Game Sounds."  Mitchell Sigman. Here's a super short article about emulating game audio sounds. Interestingly enough, this is in a journal aimed at keyboardists!  One of the most important things I took away from it was that the audio quality can't be too good or the music sounds to modern. There's a reduction in quality necessary from what's possible today to get the true 8-bit effect.

"Game Play to Music Play-- Video Games in the Music Room." Andrew Mercer. Very short-- target audience of music educators.  Gives some basic info about games that feature music prominently in the gameplay. While I've played Wii Music and Rock Band, I don't know Sing It at all and haven't played Dance Dance Revolution. I'll have to give Andrew's Audacity tutorials a watch to learn a bit more about him and the program; I've got only a very basic understanding of it.

Surveying Literature: A Comprehensive Study of Sound in Computer Games (Kristine Jorgensen)

For several months, I've been dipping in and out of Kristine Jorgensen's book, A Comprehensive Study of Sound in Computer Games (How Audio Affects Player Action).  Finally finished it up when I was up north on vacation with no internet!  When I started reading it, I expected the player reactions toward the end of the book to be the most interesting part, perhaps in part because that's exactly what Jorgensen suggests in the beginning of the book.  Instead, in retrospect, the most compelling section was where Jorgensen explains terminology unique to game audio.  The terminology here is enough to satiate any musicologist looking for intellectual fodder.  For anyone considering conducting a formal study of game audio, this book is a must read.  A methodology for studying game audio and how it affects the player is presented in detail and explained thoroughly.

As I read, at times, I thought about how awkward it is to describe gameplay and game audio so extensively in words.  Eventually, most games become widely unplayable as technology changes and they're left behind.  Earthbound (until recently), Diablo, Teenage Mutant Ninja Turtles (the arcade), some of which are now coming out in the same or new versions, others of which are yet to be released on modern systems...  Sometimes, these are amazing games that just become unplayable; and as hard as it can be to imagine, even the beloved WarCraft 3 may one day become one of these games.  Of course, video and audio recording of the gameplay would be a better record than a printed one and access to the interviews and gameplay would also provide greater understanding of the player reactions in this study.

Here are some of the most interesting ideas/quotes/definitions from the book:

"Lars Konzack explains that games must be analyzed on the basis of seven layers: hardware, program code, computer application functionality, gameplay, meaning, referentiality, and socio-culture."  It's definitely good to keep in mind all of the things that can be understood about a game to provide full insight into it.

"Alex Stockburger creates the following categories of sound objects: score sound objects, zone sound objects, interface sound objects, speech sound objects, and a range of different effect sound objects."  For more explanation, one might imagine these as: music, sounds attached to locations, sounds in menus/results of player commands, speech sounds are pretty self-explanatory, and effects sounds examples include magic spells, death sounds, etc.

"When we hear the sound of a gong in Warcraft III, it is not important to know where this sound originates from, but it is important to know that the sound signals a change in status."

Here's a list of terms that I found particularly engaging:

Perceptual fidelity-- "What game developers are striving for when adding sound to a game for the purpose of creating a lifelike and naturalistic environment.  Magic spells, for example, which have no real "audio fidelity" have perceptual fidelity....  Describes how sounds can support a lifelike and credible environment without conforming to standards of realism."

Earcons-- artificial noises and music with arbitrary or symbolic relationship to its referent.  Non-familiar or abstract sounds such as artificial noises and short musical phrases that are not intuitively recognized and that the perceiver must learn.

Auditory icons-- characteristic sounds that can be recognized as sounds of corresponding real world events.  Do not have to be learned.  Are recognized intuitively.

Transdiegetic sounds-- a cross between diegetic sounds and extra diegetic sounds.  Work from both an internal and external function in the game world.  Provides both a system message function and is part of the virtual world to which the game belongs.  Can manifest as both earcons and as auditory icons.

Functions of game audio:
1- Usability functions-- sounds that have a direct relation to actions in the computer game, either proactive or reactive
2- Orienting functions-- concern information about the existence, prsence, and relative location and distance of objects, events, and situations in the game environment.  Extend the player's perception.
3- Atmospheric functions-- provide moods to the game in order to increase the sense of a lifelike universe, player engagement, and the sense of presence in the game world
4- Identifying functions-- sound as a system or recognition.

Surveying Literature: Audio for Games (Alexander Brandon)

Over my recent trip up north I finished reading Alexander Brandon's Audio for Games: Planning, Process, and Production.  I have to say that out of all the things I've read, this book was one of my favorites.  For one, it's written by an industry professional who can give insight that only someone who's lived and worked in the field can.  For another, the writing is in a clear, easy to read style, and the book has a great flow overall.

I was struck by how much space/attention the book gives to organization and workflow.  Of course, this process development is useful knowledge, but not what I'd think would be such a large focus of a book on audio.  From that I deduce that the game audio process needs better organization.  Really, no matter what your industry, if you're in management at all, this book would be a worthwhile read for how to work as an efficient and organized team.  I read this book at exactly the right time considering that I've been playing around with The Game Audio Tutorial and integrating audio, which is a process this book explains methodology for, but gives no practical exercises.  This book and GAT would pair well together in an audio integration class.

Any portion of this book is worth a read if you're interested in game audio, but I easily had three favorite chapters in the book:  The fourth chapter, "Design" presents a really interesting historic discussion of the use of audio in games.  I'm sure I've read most of that information elsewhere, but at this point, the repetition is making everything click and Alexander's writing style is really easy to follow.  Since I'm interested mainly in the creation of game audio and those who create it, chapter seven, "Ideal Production," resonated with me.  Although certainly dated now that it's eight years old, this chapter is an important historical documentation of what has happened, methods, prices, licensing practices, etc.  (Actually, early in the book, Alexander says that if you're reading the book in 2015, don't discount all the info, just up the technological requirements!)  Finally, I loved the last chapter "Looking Ahead" where interviews with Guy Whitmore, Tommy Tallarico, and George Sanger present some contrasting views toward the future of game audio.

While I'm also hoping to better understand how game audio functions in smaller productions as well, this book gives insight into how game audio is produced in the larger game companies.

Other short reactions through the reading:

As someone who's played a lot of RPGs, I love that the book states that RPG audio is pretty much as complex as it gets and if you can do that, you can do any of the other styles/genres of games.

I love the emphasis again on creating prototypes and sharing them with the development team.  Almost all game audio creators write or talk about the importance of that task.  Certainly, it's the absolute clearest way to communicate about game audio to non-audio folks.

In the Workflow section of the book, Alexander describes the follow Operational tasks:
-Lead time (scheduling, budgeting, staff meetings, documentation, programming/design meetings)
-Purchasing: software/hardware
-R&D: reference soundtracks, field recordings, Foley work, experimenting with sound and instrumentation, and most importantly: playing games that relate to yours!
-Prototyping:  Everyone says this is important and much better than text descriptions of audio!

I hadn't considered that games have Quality Assurance staff to play-test games (here focusing on game audio), but the book lists tips for the best ways to get the most out of game testers for game audio.  Bug databases, for instance.  

Who knew that game audio designers used metric tools to gather information about how many times music/sound occurs in a given area?  What are these tools?- the book doesn't go into detail.  Wonder if these can be applied to retrospective gaming studies as well as games in development?  It'd save me a lot of post-its and ink pens....

Ever wonder why turning on your console/starting a game is so loud?  "A lot of developers use their highest volume in the game as the same volume played when the turn on the console and the console logo is played... From there, you need to figure out how voice, sound effects, and music will all interact."

Hierarchy of game sounds: Voice (loudest), sound effects (next loudest), music (softest)-- exception might be a musical stinger.

"What makes an audio asset final?  When it's produced?  When it's integrated?  When it's integrated and mixed?  When the leads sign off on it?  Yes to all."

As usual, here's a list of games that are mentioned in the book:

Pong
NBA Live
Mortal Kombat
Madden
Doom 3
Halo
Tiger Woods 2004
Links
Wipeout XL-- prime example of music licensing
Warren Spctor: Ultima 6, Ultima 7: Serpent Isle, System Shock, Ultima Underworld, Deux Ex
No One Lives Forever and No One Lives Forever 2
Deus Ex: Invisible War-- one of the first games to fully feature a physics sound system
Parappa the Rapper and Rez-- music influenced the game design
Knights of the Old Republic-- LucasArts/iMUSE
Monkey Island 2: LeChuck's Revenge-- first title to use iMUSE
The Dig-- iMUSE with streamed stereo audio files-- incredible audio/soundtrack
Castle Wolfenstein
Ultima Underworld: The Stygian Abyss-- first FP perspective to use an interactive soundtrack
Fade to Black-- adaptive soundtrack (not great integration)
Thief-- ambient sounds make an overwhelming impact
Dungeon Master-- occasional ambient sounds from an early title (1989)
Total Annihilation-- 1997, live orchestra work (Jeremy Soule)
Secret of Evermore-- Jeremy Soule's first sountrack
Russian Squares-- subtle changes to music in a puzzle game
Thief: Deadly Shadows-- audio engine was Ion Storm
Halo: Combat Evolved-- Marty O'Donnell
Grand Theft Auto: Vice City--  players listen to the radio, Madonna/Michael Jackson, etc
Frequency, Amplitude: use licensed pop songs where your goal is to mix and remix the songs
Star Wars and MLB Slugfest: Loaded-- use non-top 40 licensed music
The Legend of Kage-- upbeat pop soundtracks with too loud sound effects for the Ninja Stars
Earthworm Jim and Cool Spot: examples of Tommy Tallarico's composing work
7th Guest and Wing Commander-- examples of George Sanger's cinematic sounding work

As a teacher of technology, I understand that technologies change through time.  Many various ones exist that do similar work and the ones we come to use are the ones we like the most.  Any list is thus incomplete and dated.  However, one thing I was wondering when I started this project is what game technologies audio folks working in the industry use.  Finally, here's a book that really lists many technologies and what they're good for in game audio.  Thus, complied from the book, here's a list of all the software that's mentioned, in some cases with notes about what one might use it for:

mSoft-- sound effects
Hollywood edge (FileMaker)-- sound effects
Pro Tools, Cubase SX, Nuendo-- multitrack SFX (or music) production
Renderware Studio-- audio engine for real time mixing, simple GUI, map editor
Unreal Engine-- map editor for integrating sound
PerlScript-- running batch script
DirectMusic
Sound Forge
Microsoft Project/ Visio-- for making a Gantt chart
Absynth/Reaktor-- among top synthesizer programs on the market
Bugzillla, BugCollector, TestTrack Pro-- bug databases
DirectSound, DirectMusic-- first freely available software to create adaptive soundtracks
Miles Sound System-- middleware
Direct X
GameCODA-- an engine sufficient for simple titles and most more complex projects
Creative EAX-- creates decent reverb for free
SoundMax-- middleware to synthesize sounds in real time
Galaxy Sound System (part of Unreal)-- an example of a custom made tool
Xact-- Xbox Audio Content Tool
Scream-- Scriptable Engine for Audio Manipulation (Sony)
Alienbrain-- file management system
iMUSE-- Interactive Music Streaming Engine
Gigastudio
Visual SourceSafe-- file management for Deus Ex: Invisible War
Logic
Xbox ADPCM (adaptive differential pulse code modulation)-- file type used by Xbox
PlayStation 2 VAG-- file type used by PlayStation

Surveying Literature: Game Development Essentials (Jeannie Novak)

Finished reading today through chapter 9 in Jeannie Novak's Game Development Essentials, the audio chapter.  I really enjoyed this chapter overall.  It makes a great introduction to the ideas of game audio and briefly traces the history and development of game audio as well as exploring the many facets of it: dialogue, sound effects, and music.  Within these, the chapter explores looping music versus adaptive music.  It's easy to read and I think I might assign this chapter as an early reading in my class.  The quick and direct, easy reading nature make the chapter ideal.  While the clutter of numerous interviews make the reading a bit disjointed, I like that this introduces the reader to some of the major players in the field, the lexicon, important websites, and points where the reader can find more information about topics.  

Here are some of the most interesting quotes with my reactions:

Talking about frequency modulation in early game audio, Novak writes that "the results were less than spectacular."  What's the point of that?  Yes, some of the early game audio sound very crude and grating... to me, many of the early Atari sounds are akin to the sound of dialup internet in the 90s.  However, plenty of people have hard core nostalgia with these sounds these days.  And frankly, at the time, these sounds were plenty convincing and perfectly suited and realistic to contemporary players.  I'll never forget talking to my grandfather about the old movies like Frankenstein.  When I asked him how he (or the audience) even thought they were realistic his reply was: "Well, you'd never seen anything like that before.  For those of us watching, it was a new effect that seemed lifelike.  Now, of course, you've seen much better and it looks unrealistic, but at the time, it was cutting edge and terrifying."  Same here.  Why dis the beginning of game audio?  Whatever one may think of the sounds, it was what it was-- the seeds that sowed where we are now-- why degrade it?  

Novak quotes Chad Mossholder": "I don't separate sound effects from music.  To me it is all music.  Go outside and listen to the traffic, the birds, the planes flying overhead, the people talking: It's an ever-changing sonic composition."   This makes me wonder: what is music?  Of course, I've been thinking about that for years, but definitely a great question to pose early on in my game music class.  While I want to spend a bit of time on each of these, I want to separate between sound effects, dialogue, and really focus in on game music, but really, where is the line between them?  

Novak separates sections in the chapter of "looping music" and "adaptive audio."  What I don't understand here in this distinction is that plenty of games have both!  Doesn't game audio typically loop until something gameplay makes it change (into adaptive audio)?  I'm not sure this distinction makes sense to me because at some level the audio for where you are in a game is looping until you do something in the game that changes it.  I mean, in a weird 21st century sort of way, I understand what she means, but also, I think I inherently disagree with this distinction.  Looping audio and adaptive audio aren't exclusive in my mind.  

Overall, a great read.  One I'm going to encourage (or require?) my students to encounter early in their studies.  

Surveying Literature: Powering Up: are computer games changing our lives?

I've literally been thumbing through books on the shelf recently for any references to game audio or game music.  I'm really trying to get my mind wrapped around what's printed and who's researching this stuff.  It seems like the last few years have literally been an explosion into the field, which makes me thrilled to be investigating this at such an exciting time.

Here's a case in point of the minutiae I'm thrilled to be finding!  In Rebecca Mileham's book, Powering Up: are computer games changing our lives, there's a very brief section on pages 293-294 where Mileham points to some research by Gianna Cassidy on how music is integral to player's gaming experience.  Several interesting statistics, for instance: while playing Project Gotham, players performed better and had the most enjoyment when they were able to select and listen to music of their choice.  How cool!  I'm going to check out more about Gianna and her research!  Turns out she's a singer/songwriter as well as a teacher about video games.  From one performer to another, check her out in a duet here:

Surveying Literature: Sound in Video Games (Eric Pidkameny)

I found a copy of Eric Pidkameny's essay, Sound in Video Games, in a larger collection of essays, The Video Game Explosion, edited by Mark J. P. Wolf.  I also know Pidkameny has an earlier essay that's available online, Levels of Sound, and months ago when I found it, I made a note to finish reading it.  That article is a little bit longer, while Sound in Video Games is a short, largely a historic article that briefly charts the aural and technological progress of game sound, and then turns to the idea of Adaptive Audio and synergy between game music and other fields.  This is an excellent, concise game audio history that whets the appetite for more information in a variety of game audio areas.  A great place to start if you're wanting to learn more.

I'm not able to find much information about Eric Pidkameny online.  If he has a website or a bio somewhere online, I'd like to know the link.  One of the cool things about this project is not just that I'm learning more about the audio in games I've played and where scholarship is with game audio, but I'm also learning about who is interested in this kind of study.  It's really becoming cool to connect virtually with these like minds!  I've got a meeting next week to discuss some grants possible from UM that I could win to travel and investigate game music at some conferences.  So hopefully some of these virtual connections I'm making will soon turn into vis-a-vis ones!

As a fan of Final Fantasy games, I was really struck that he takes time to mention the music in Final Fantasy 3 (I would say FF 6) as one of the most important games in terms of game music.  He writes: "In FF3 (6), each hero and villain in the game had his or her own musical leitmotif, which played whenever that character became the focus of the story.  The musical accompaniment was a form of wordless narration, giving insight to the game's characters and events in a way that text alone could not."

There wasn't so much here that was entirely new to me in terms of thought, although I did spend a little bit of time investigating some of the games that are mentioned whose audio I hadn't heard before.  Therefore, in a similar thread with my list of the games Karen Collins mentions in Game Sound, here are the games that Eric lists in this article, in the order he lists them:

Games:

PONG -- first audio
Space Invaders -- first continuous music
Pac-Man -- musical interludes/cutscenes
Berzerk -- dialogue
Sinistar -- dialogue
Dragon's Lair -- digital audio
Super Mario Bros -- new era of game sound
Star Wars: X-Wing -- adaptive audio
Ocarina of Time -- adaptive audio, also highlighting Koji Kondo
Metroid -- these three are some of Nintendo's most popular franchises
Dr. Mario -- also highlighting Hirokazu "Hip" Tanaka in these three
Earthbound -- these three are some of Nintendo's most popular franchises
Final Fantasy 3 (6) -- use of leitmotifs, Uematsu is one of the first internationally famous sound designers
Dance Dance Revolution -- music and rhythm based dancing as a part of gameplay
The Sims 2 -- user designated soundtracks blur the distinction of what is game audio
Grand Theft Auto: San Andreas -- user designated soundtracks blur the distinction of what is game audio
Defender -- these two serve as bookends to understand the progression of audio
Chrono Trigger -- these two serve as bookends to understand the progression of audio
Half-Life 2 -- almost all the soundtrack is diegetic music
Halo 2 -- video game music's first move onto the Billboard 200 chart

Surveying Literature: Hair-Raising Entertainment: Emotions, Sound, and Structure in Silent Hill 2 and Fatal Frame (Inger Ekman and Petri Lankoski)

Inside Horror Video Games: Essays on the Fusion of Fear and Play, edited by Bernard Perron is a collection of essays whose title accurately describes itself.  Within that is a chapter that deals largely with emotional response to game visuals and sounds: Inger Ekman and Petri Lankoski's Hair-Raising Entertainment: Emotions, Sound, and Structure in Silent Hill 2 and Fatal Frame.  I really enjoyed this as a fairly easy read and think it makes a perfect possible reading for my Game Music Class.  It's the kind of reading where having played either of these games will make it easy to connect with.  Although I haven't played either (I do have Silent Hill 2 checked out from UM to get a taste of this weekend), having played games like Diablo and BioShock made it easy for me to imagine gaming in this genre.

I like how the article connects the audio and visual functions in the game with our emotional responses.  The authors often present one or two different emotional responses to a gaming situation, and I found myself often wondering-- is my emotional response really that simplistic?  Of course, emotional responses, just like the reasons that sounds appear in game, are way too complex to be defined as simply as they are here at times, but there's nothing lost in highlighting a couple of possible responses to a gaming moment.  I think considering more deeply about how your emotions are sculpted by the sound of the game is a very useful thought process for players.  The audio team has undoubtedly already done this in reverse, so an activity like this helps you to understand their perspective even better.

Most interesting quotes and reactions:

"The way sound is portrayed can signal shifts between subjective and objective view.  For example, accompanying a shot of a conversation with a muffled dialogue creates a sense that the scene is being viewed by someone excluded from the conversation, for instance a secret observer."
Notes: This is exactly the kind of thing that we know without thinking about, but there's plenty of use in thinking about it!  Sound design choices-- whether a sound should be obscured, how loud it should be, when it should happen, where it should come from, etc...  These are the kind of choices that can really create the perfect atmosphere and draw the player in more deeply when done well!

"Moreover, whereas visual information in games is limited to the screen, sounds (changes in air pressure) extend into space, even interacting with it.  Sounds create continuity and presence, and as a result, confuse the borders of fiction.  This is partly what makes sound such an effective shortcut to emotions."
Notes: I keep coming upon the debate about the relationship between visual and sound in gameplay.  "Who will reign supreme?!"  My response: WHO CARES?  I, for one, wouldn't choose to give up any of my senses during gameplay!  Only visual or only audio doesn't give you the full experience of a game.  The two were made to work in tandem with even more features like gameplay interactivity (using the controls) and tactile reactions (your controller vibrating).  Games aren't just about any one method of connection.  I do like how the authors point out that the sound actually creates changes in air pressure in your room.  Of course, the visuals create changes in lighting in your room...  Ultimately, aren't we imagining them to become more like they are on Star Trek, where you go into the Holodeck and are completely immersed in your gaming world?  In that case, even a sense of smell and taste would be possible with gaming-- and once we've had those, who'd want to give them up either?

"Synchronicity between sounds and visuals is used to create further contrast between human-like movement and sounds that go against the human; some of the monsters sound almost mechanical.  At times, creatures change suddenly from bipedal walking to insectal scurrying.  These abnormalities introduce a threat of contamination because they are blurring the categorical borders between what is considered human versus non-human.  This is enough to trigger both disgust and fear."
Notes: Really interesting.  I hadn't considered how or why monsters were given certain sounds, but all of this seems very true.  If scary is non-human (although at times, scary can be innocent child/ very human!), then this sheds light on why some of the audio choices are made and the effect they have on the player.  This made me think about some work I was doing with The Game Audio Tutorial and how the authors there explain that having a room with no ambient sound is strange, and thus-- you want to create a soundscape for every location.

Surveying Literature: Musical Representation in the Video Games Guitar Hero and Rock Band (Nicole Biamonte)

I just finished reading some chapters in Pop-Culture Pedagogy in the Music Classroom, edited by Nicole Biamonte.  The chapter I was most interested in in terms of the blog is a chapter written by Nicole: Musical Representation in the Video Games Guitar Hero and Rock Band.  The title pretty much gives the thrust of the article-- she explains how the games show rhythm, pitch, and form to the players.  She also discusses the layout of the controller buttons on the guitar and how they're represented onscreen.  I wasn't aware that most versions of Guitar Hero mention the form of the song as you play along.  Of course, this helps the player recognize the music that is coming ahead just as keeping the form in mind helps the musician players remember what's coming up.

I have to admit...  I kind of idealize this style of writing-- where it's as if the author is from another planet, stepped on to Earth, has come across a video game, and is describing what it's like to play to the citizens back home.  It really gives a chance to connect with an audience outside of your area specialization.  In effect, though, this is exactly what Bimonte is doing-- her target audience is musicians, who, I hypothesize, are less likely to play these games since they're making music in their daily work.  All the detail she goes into about how the terms that appear in the game: describing the "outro" as the "coda; often motivically related to intro or body of song," it's as if she's taken Rock Band/ Guitar Hero speak and converted it into classical music language.  She compares the notation in Guitar Hero/ Rock Band with current standard music notation as well as historic neumes.

My favorite part of the article are the activities she suggests some activities for a music class that use Rock Band/ Guitar Hero.  Having those with less knowledge transcribe well known tunes into musical neumes, sort of common to these games' notation, while those with pretty advanced musical knowledge could take a music and simplify it, deciding what pitches, rhythms, and contours would be worthy of being represented on screen and which would be left out.  Super cool!

Most interesting quotes:
"The visual and aural separation of song tracks into lead, rhythm, and bass guitars, drums, and vocals trains listeners to parse layers of musical texture.  Many players have commented that their gameplay experiences have led them to hear textures more completely, not only in the songs with the games, but also in all of their music listening.  -from Kiri Miller's Guitar Hero: A Research Blog.  guitarheroresearch.blogspot.com

"There is as yet no keyboard instrument controller for the games, which is not only because of the difficulty of designing a simplified version but also because playing keyboards requires relatively minimal movement and is, thus, inherently less performative than guitar."
Uhh...  This would be in contrast the the praise for Rock Band III that I encountered in my study of the audio mentioned in G4's Top 100 games.  Here, Scott Porter states that everyone wants to play the keyboard and that it's the best instrument.  Interesting contrast of ideas.....

My initial thought at this point is that this article would be better suited as a possible reading/ possible activities to do in my Tech for Mus Ed class than it would be for my game music course.  Although the thrust is game music, Biamonte is explaining to classically trained musicians how the game works.  The book also contains two other articles that caught my eye: Benjamin Bierman's Appreciating the Mix: Teaching Music Listening Skills through Sound-Mixing Techniques got me thinking about how to talk about mixing music in my game music class.  Then there's Brent Auerbach, Bret Aarden, and Mathonwy Bostock's DDR at the Crossroads: A Report on a Pilot Study to Integrate Music Video-Game Technology into the Aural-Skills Classroom-- which reads as a typical Mus Ed empirical study on using Dance Dance Revolution as a teaching tool.

Surveying literature: searching online databases for game music

I spent several hours today in the UM libraries searching for articles about game music.  I'm now realizing a broader distinction between "game music" and "game audio."  I'm interested in game audio: I want to better understand the broader discussion of game audio (dialogue, voice acting, recording techniques, implementation, etc), but I ultimately want to focus on game music (composers, composition process, musical aspects, musical-technological interaction).  So to go to the next level, I'm turning my attention to individual chapters in books and journal articles with a focus on "game music," and thus turning away conversations about recording dialogue and special effects.  These, of course, are interesting to me as part of the audio experience, but they aren't game music: my primary focus.  I'm also trying to make sure I finish physical materials ASAP so I can switch to electronic materials for the summer when I'm away from home.  I always bring too many books with me...  I'm going electronic this summer.

Tonight I did a very broad search for the top 100 items on ProQuest for "Game Music" and came up with a ton of hits.  Here are my favorites and most interesting quotes, weblinks, and reactions:

-"Game Music Central Presents 'Best of the Best: A Tribute to Game Music.'" Canada NewsWire
Notes: Gamemusiccentral.com no longer exists in this form.  Online media and game music... can they coexist?  How does video game music market itself and connect with others?  How does it react with the world around us?  How do artistic concerns drive it?  

- "Game Music Gaining Interest, Money, Respect." Melinda Bargreen.  Seattle Times
Notes: NBA Live 2003 became the first soundtrack to go Platinum.
Alistair Hirst: "having an orchestral soundtrack is seen as a 'big plus, bringing value to the game.'"

-"College orchestra plays thumb wiggling video-game music."  Stacey Hollenbeck.  Tribune News Service.
Notes: As of Oct 2007, "6 of the 11 marching bands from the Big Ten schools  had played video game music at some point in their half-time shows."  Is UM one of these?  I can't tell via the web, but would love to know if any readers have info on a UM half time show involving game music.  I did find two blog entries by the composition department at MSU about game music...

- "'Live': Game music 'the radio of 21st century.'"  David Stabler.  The Oregonian.
Notes (2009): 68% of American households play video games.  They spend more time playing them than watching TV, reading, or going to the movies.  The average game player is 35.  40% are women.
Tommy Tallarico: "When you watch a movie, you hear the music once and six months later, the DVD comes out and you might hear it again.  Within one year, you probably hear maybe three hours of John Williams' music.  Compare that with "World of Warcraft," where a player is hearing the music 40 to 60 hours a week."

-"Video-game music isn't just kid stuff, it's a concert." Melinda Bargreen.  Seattle Times
Notes: In 2003 EA reported: "40% of respondents said that after hearing a song they liked in a video game, they bought the music CD.  Several emerging pop-artists, after being featured in video-games, now have hit the Billboard charts, according to Billboard's Feb 15 issue."

-"Game Music in the Conservatory."  Dayton Daily News
Notes:  Michael Sweet: "We're seeing kids want to play the violin, french horn, or other instruments because they've heard it in a video game or seen it at a game music conert."

- "Concertos for Your Console:  video game music is now so good, fans want to see it performed live.  Robert Colvile.  The Daily Telegraph.
Notes: Jason Hayes: "People who aren't gamers still think video game music is all kind of like Pac-Man.  But for composers there's always a new challenge-- you're basically collaborating with the player."

-"'The Fat Man' rules in computer game music." Mary Hellman.  San Diego Union-Tribune.
Notes: Johnny Wilson: "George Sanger was the first to take a music composer's attitude toward the product and to integrate the music with the visuals. The score for Loom was revolutionary... What you had was true scoring, with themes for characters and scenes.  It reminded me of what John Williams had done with 'Star Wars.'"

-"In a Nod to Lush Film Scores, Game Music Gains Texture." Matthew Mirapaul.  New York Times.
Notes: "Computers and game consoles now support multiple audio streams-- up to 256 separate "voices" on the Xbox, for example.  Whether the music source is a MIDI synthesizer or a prerecorded sound file, the systems are powerful enough to combine and modify the streams in interesting ways-- provided that the composer has provided the options in advance."

-"Using Living, Breathing Musicians In Game Music."  Jack Wall.  Game Developer
Notes: "If your budget doesn't permit the recording of a live orchestra, then do your best to record enough living, breathing musicians to bring the game alive.  And please, if you are composing for a game trailer that will play next to John Williams' Episode III trailer, remember...  it will be playing next to John Williams' Episode III trailer."

-"Staying in Tune with the Times: Video Game Music Has Come a Long Way." Nick Lewis.  The Ottawa Citizen.
Notes: "Since many young people today play video games more than they listen to radio or watch TV, video games are breaking talent the way MTV did in the 80s.  A recent poll of core gamers aged 13-23 showed 55% of them learned about new artists from video games."

-"Hot for Game Audio."  Damian Kastbauer.   Game Developer.
Notes: Reviews much of the same literature I've read or have ready to read: Collins- Game Sound, Brandons- Audio for Games, Stevens/Raybould- The Game Audio Tutorial, OpenAL documentation-- (I need to learn more about this), Bridgett- From the Shadows of Film Sound, Sangers- The Fat Man on Game Audio.  This may be the gem of my search: my investigation of this may be best summed up by perhaps my favorite social media post: "This exists" (the Game Audio Podcast).

My beloved, NPR, garnered two links:

Lindsay Totty.  "From Mega Man to Final Fantasy, Live Video Game Music."  Ok, so maybe I can convince the U to let me travel to ONE music conference...  Where should it be?  MAGFest?  GameSoundCon?  Project Bar-B-Q?  Also...  I've just been reading/watching interviews with Koji Kondo where he emphasizes the importance of melody in game music composition.  Interesting that Nobuo Uematsu does the same here: "Lindsay: Uematsu echos the thought that it's melody that makes for great game music.  Uematsu: It's pretty easy for me to come up with melodies.  24 hours a day, it's in my head all the time, and I struggle to choose which ones to use."

David Greene.  "Video Game Music: Big Music, Big Money."  The classic Greek conversation between student and master.  How much does it cost to make game music?  Tommy Tallarico: Extremes: $50,000 to $1,000,000.  The average: $250,000 to $400,000.

Surveying Literature: Pro Tools 10 for Game Audio (Greg deBeer)

Yesterday, I finished Greg deBeer's book, Pro Tools 10 for Game Audio.  Clearly the book is in part an advertisement for Pro Tools 10 and how it can apply to Game Audio.  Most music tech people I know are using Pro Tools for serious studio recording, and I don't think of it as something to learn for game audio, but I might now!  Really cool to get step by step instructions for how to use Pro Tools this way.  Here's a superb interview with Greg deBeer's about his work with dialogue at Sony, although now his title is Senior Sound Designer.

Similar to The Game Audio Tutorial, following this book provides access to some sound files the reader can manipulate.  That said, one of the differences between the two is that The Game Audio Tutorial focuses more on how to integrate the audio into the game environment, where as Pro Tools 10 is more of a walkthrough about all the ways Pro Tools can apply to game audio, making samples, etc.  I appreciate that this book lays out definitions of common words to encounter in the industry.  For instance, I now understand the difference between Foley, Sound effects, and Backgrounds.  And also walla and futz.  Music specific terms to know are: score, cue, song, soundtrack, temp, diegetic/extra-diegetic, transition, motif, and hit point.

Interesting to think about localization from an audio standpoint.  I always wondered why there were so many groans and screams in cinematics in games.  That kind of stuff doesn't have to be re-recorded, but any intelligible dialogue or sound, of course, does.  If classical singers really were into their languages in the right way, they might make an interesting living helping with localization, since the languages: French, Italian, German, and Spanish (FIGS) are the same that classical singers commonly train in.

I also hadn't considered quite how different sound effects for movies are from games.  Whereas movie audio happens as a pre-determined sequence of events, game audio is dependent on player action, not a one time fixed order.  Games have to be able to account for this without sounding repetitive or boring.

While most of the book seems aimed at making audio effects, the chapter on music had some interesting points as well.  Using music in games from commercial music libraries is one thing I hadn't really considered.  Of course, I understood this at some level considering that in an earlier blog I discussed Nintendo's use of stock chanting in the Fire Temple level of Ocarina of Time.  Also, the difference between layering music and jump cuts in the music.  With many games today, there's probably a combination of these elements.

Some of the specifics about audio level for cinematic mixing are good for me to encounter and explore more.  But one of my favorite comments from the end of the book is one I tell my Mus Ed Tech classes: check your work on multiple speakers!  Anyone who's worked with audio knows that a mix might sound fine on your laptop or headphones, but sound completely different once you've plugged in to larger speakers.  Great advice.  Think not only about how the audio sounds with your set up, but with others-- and for the audiophiles who are often creating these games, this might mean using earbuds instead of a 5.1 surround sound setup.

Reading these "how-to" manuals the last few days has opened my eyes to some expertise from folks who are working and making a living in the game audio profession.  I'm constantly surprised by how much of the literature focuses on effects and not on music.  Recording techniques can carry over from dialogue, vehicles, gunfire, etc to music, but the focus in these books is not on music.  Of course, in the case of The Game Audio Tutorial and Pro Tools 10 for Game Audio, this could be because the authors are talking about FPS games where explosions are king.  I understand that the FPS genre is the best selling one, but platform gaming sounds/music deserve more than a passing mention in these books.  Furthermore, although I've enjoyed learning about all these different game audio areas, I'd like to see more specific information about music composition for games and techniques for that rather than the broader focus on game audio.

Surveying Literature: The Game Audio Tutorial (Richard Stevens/ Dave Raybould)

I just finished reading Richard Stevens and Dave Raybould's The Game Audio Tutorial.  This book is such a wonderful reference, and very practical too.  What I appreciate the most about it is that the exercises and material are of great interest to me, especially since implementing audio into games isn't a task I've done before.  The authors here repeat the idea I first encountered when reading The Fat Man's Guide to Game Audio: understanding how the audio is placed into the game is a critical step for those working in game audio.  One doesn't necessarily need complete mastery of it-- of course, that also depends on the size of the audio team-- but any knowledge in this area informs other decisions one makes as a game audio creator.  The authors have a perfect analogy, equating learning about the programming aspects of game audio to learning some basic phrases in a foreign language when you're traveling abroad.  Not only does it help you, but the natives appreciate that you've taken time to do this.

I appreciate that the authors want the information in the book to be practical.  They reject at the outset the idea that audio creation (asset creation) can't be separated from implementation.  I'm not sure I completely agree with them, but I appreciate their viewpoint.  Having the final product in mind is always important, but for a person getting into the field, there may be times when the pendulum is more toward one of these sides than another.  For instance, their book provides online links to learn more, and among other things, gives access to a sound library and game world where you can try out the methods described in the book.  This is in part because their focus is not on how you create your sounds, but how you put them into the game.  While the authors have simply supplied files for you to manipulate and play with, as I consider this, I realize I'm very interested in the creation of those audio files in the first place.  I see how these ideas are inseparable, but also appreciate that there are times when one's focus may be completely on one part of the equation.  

It's nice to read a book that simply commits to a method and software as a tutorial guide.  The authors freely admit that software/hardware will changes over time and with various projects, so there's no one method to learn that will help with all situations.  Of course, depending on what one's developing for, the details of how the system works can be very closely guarded.  However, if you learn any method/software, you can use those techniques similarly in whatever the actual situation you find yourself to be.  The "Here's a task, we're going to explore it with this technology, and then later you can use these same ideas in other situations" is a classic line from any technology education-- given how quickly the field changes-- and one I repeat often in my Tech for Mus Ed class.  

I'm looking forward to playing around with the interactive component of this tutorial this summer and hope to work through the implementation chapters in the beginning of the book.  Having done this will give me a huge understanding in implementing game audio-- the area of game audio where I feel the least knowledgeable.  Or to spin it more positively, I might say they're where I have the most room to grow!  

Here are quite a few of my reactions to the book:

This book struck similar points as The Fat Man's book.  Carrying a recording device with you everywhere, having regular planning and strategy meetings between the game designers and the audio team, and trying to get involved with the game as early as possible.  Although I shouldn't be surprised, the stressed importance of organization and great record keeping with audio tracks is interesting to see.  Clearly, this is critical in so many ways, perhaps most basically seen when you're looking for a sound and organizing sounds on your computer, but the authors also stress it early in the audio creation process.  For instance, you'd be wasting time by working with dialogue that's whispered, no matter how dramatically expressive, if it's during a loud gun battle.  

Many of the audio details that the authors group into "dialogue" are also useful for capturing samples of live music.  Particularly I'm thinking about microphone placement, recording equipment, editing of these samples, etc.  While I hadn't really thought about it, I do know a fair amount about this from teaching Tech for Mus Ed majors.  Reading a book like this is a constant reminder to me of how much I don't know, so it was good to realize also what I already bring to the table because of my training and work.  

One very interesting idea to me has to do with realism in game audio.  While it may seem that audio designers want audio to be as realistic as possible, this isn't actually true.  The player wants a guided audio experience through the virtual world.  Whereas our brains and ears already do this, sound designers are faking this in games (and movies).  Imagine a game world that could recreate a fairly true acoustic as a part of the game engine (I don't think this exists).  Put this scenario into a loud bar.  There'd be times to cheat this to have more emphasis on an important sound over others-- a glass breaking or a sentence of dialogue-- rather than on all the ambient noise of the bar patrons (walla).   

Game audio is often thought of as needing variation.  However, there are sounds that don't want variation-- the sound of a power up, or of damage being taken, for instance.  This sound needs to be the same each time because of the information it conveys to the player.  

The audio field in a game can convey much more information about the environment than the visual field.  Sounds are often presented in a complete 360 radius from the avatar, but the visual field is typically smaller.  I want to think more about the visual range of information vs the audio range the player receives.  

I love that there's a focus on Leitmotifs and their importance.  That's really one of the main features of game music that drew me into examining the genre more, as can be seen from my video about Leitmotifs in Final Fantasy 7.  

When you're conceptualizing game audio, don't just talk about it, create it!  This is especially helpful as many game designers may not have the same musical vocabulary as the audio folks.  This way, you've got concrete feedback about how to proceed.  Setting music to artwork and text is a great way to get the entire gaming team focused early in the process.  Don't forget that music draws the player more deeply into the gaming experience.  If engineers are creating the level as they listen to your music, it's influencing the mood and already getting them more immersed in the game world as they create it!  

Compared with the step-by-step approach in the rest of the book that largely teaches how to implement game audio, the Music Systems (Chapter 4), flat out says: we assume you know about music.  If you don't, the authors suggest studying, "pitch, rhythm, harmony, timbre, texture, instrumentation, and how to structure these into a musical whole."  This list, while short, is helpful as I consider what the most important musical features are for studying video game music.  I'm already building an understanding of this through my blog work, particularly as I go through My Gaming Audio History.  

The consideration of game genre brings up all sorts of musical expectations.  A musical tune that might be perfect for a platform game could seem heinous in a FPS.  This applies to sound effects as well.  Navigating a menu in a racing game should sound differently from the menu in a Sci-Fi game or a medieval RPG.  What are the effects of game genre on audio and vice versa?  I want to explore this line of thinking more.  

I love how the authors remind the composers to think about differences in gameplay when considering music.  How is the audio experience of a game different for a person who blasts through a level quickly versus a player who takes time to move cautiously, versus a player who's a completionist?  

"As game composer, sometimes you have to advocate for no music."   

How do you move from one musical/sound environment to another?  Hard cut, cross-fades, interrupting it with sound effects, etc.  I'm going to keep this in mind as I continue My Gaming Audio History.

In step with the practical nature of the book, the authors remind that game music is not meant to be listened to separate from the gameplay.  Instead, it's a integral part of gameplay.  I appreciate this point, but I also think that game designers are aware of how important audio can be.  Great audio can only help games.  In recent years game music concerts have grown in popularity.  While I might be the exception I want to state outright: contrary to many who say, "no one buys a game for it's audio," I do.  I buy and play games that are supposed to have good audio, sometimes for the sole sake of experiencing their audio.  

One of the best things in this book is the final "Next Steps" chapter.  It asks questions that can help guide interested people in the field.  As an educator, I'm particularly interested in how the authors describe building a game audio portfolio.  While my game music class is very much an introductory level course, any progress toward creating something useful for the participants in this way is both practical and logical.  

Game Music Online: 8-bit Analysis

A few posts ago I mentioned I'd stumbled upon Jason Brame's blog 8-Bit Analysis doing some internet searches.  I've taken some time between now and then to read through it throughly and really enjoyed his postings.  Reading through a blog a few years older than mine definitely gave me a heads up on some of the challenges of maintaining a blog long term.  The hardest thing as I see with many of my favorite blogs are continuing to update regularly.  Also I realized that by linking to YouTube videos, over time my links will become broken and require updating.  I guess that goes with the territory of online blogging!  

If I'm honest with myself somewhere inside, Jason's blog is in some ways what I thought I wanted to make of this blog.  He does some very specific analysis and lots of detail with just a tiny piece of music.  However, as I read through it, I realized that I've definitely got a different audience in mind.  He shows the most impressive transcriptions and musical analyses, but the people who can read that sort of information and process it are few.  His blog is aimed at classically trained musicians.  So, even though I know basically all of the music he's listed (SMB 3 World Map 7 music is a little vague to me because I sucked so much at that level!), the advantage of having transcribed the music in standard notation is that a classically trained musician can look at it and hear it very clearly in the mind regardless of linking somewhere.  That said, some of my favorite posts were where, despite the transcribed versions, the videos he links to make his point perfectly clear with only aural means.  This post where he compares the water music of SMB and the Title Screen theme from SMB 2 is a perfect example.  

He's also got several links up at the bottom of the blog that I want to follow to keep learning more.  We would have overlapped at UNC by a year, but I don't think I knew Jason.  I'm definitely going to give his article "Thematic Unity Across a Video Game Series" a read soon.  

Surveying Literature: The Fat Man on Game Audio: Tasty Morsels of Sonic Goodness

This weekend I finished The Fat Man on Game Audio: Tasty Morsels of Sonic Goodness.  The Fat Man is George Alistair Sanger.  Sanger has a team of friends, Team Fat: Dave Govett, Joe McDermott, and "The Professor" K Weston Phelan, who have made their game music central in their lives.  This book reads at time like a self-help book, a philosophy work, sometimes with rambling stories and a crude illustration or two.  However, it's a powerful glimpse into the mind and philosophy of a person who specializes in game audio and there's incredibly important information here, especially in the Hitting the Rubber with the Road section, where Sanger includes detailed descriptions and photos of the best ways to work in the field.  Sanger believes the same issues that have been present in game audio will play out as historical themes throughout time, and hopes that his book will begin such a documentation.  As weird as it is, Sanger knows that his book contains important historical evidence and that game composer methods need to be written down and documented.  He acknowledges that he's possibly writing for someone who pulls the book off the shelf years in the future, yet at the same time, the information contained within is critical.

Sanger is often quoted as saying that video game music is such a young field no one is dead yet.  However, with the death of Steve Jobs just a couple years ago, it's clear that the pioneers of technology: those who created, oversaw, and steered the early days of technological development aren't immortal and won't be here forever.  One day-- perhaps soon, we'll want to know things about the first generation of composers-- many of these questions I'm already wondering now: who knew who-- and how, what connections there were between people in games music, what their working methods were, etc.  Along these lines, I loved seeing the pictures in the book showing what sort of setup these working composers used.  Of course, as new technologies develop, office/work spaces and methods change constantly, so it's great to have documentation starting already.  I love Sanger's emphasis on playing real instruments and capturing live sounds rather than just relying on the computer.  I wonder if this is a pervasive feeling in the field?  It seems as the technological ability to include samples has improved, many games and composers have taken advantage of it, although now it's easier than every to simply compose with virtual instruments.

Despite a classical music training, Sanger makes most of his references in the work to popular musicians.  At numerous points he mentions game designers wanting something to sound "John Williams esque."  The Beetles were obviously a huge influence on him given the number of times he references them.  As a post-Beetles aged person, I wonder who game composers closer to my age would site as major influences?  I suppose it's possible that video games are old enough now that game music could be inspiring new composers to go into the field.  It's cool to see pictures and tales of Sanger and his friends in bands; I knew about Uematsu and the Black Mages, but glad to hear jam sessions are  more prevalent than I thought.  It's easy to forget that game composers aren't just digital guys, they play instruments too!

One of my favorite of Sanger's points is that you have to make compositions your own.  If someone asks you to create something "like" something else-- steal certain aspects, and then make it your own.  I couldn't agree more-- one of my mentors Shirley Verrett used to say to her students:  "It's when you sing with your own voice that people sit up and take notice." Making something similar to but different enough is the credo of popular arts-- enough similar to be accepted, but enough unique to stand out and have people take notice.  Finding that perfect balance is life.  Along these lines, some of my favorite moments came from the philosophical parts in the book, for instance, the quote: "Often the only sense of growth in your career will appear to be in the seeming importance of the party doing the rejecting."

One idea I hadn't thought about before is that Sanger describes game music as it relates to a family setting.  My family moved our tv with nintendo up into a bonus room when I was about 10, so we cleared the problem of only having one tv before the SNES came out, but it's definitely a consideration.  His point is this: while the kids play the game and are immersed in it, interactively, visually, and with audio, others around may not be able to see the screen and certainly aren't playing along.  In other words, the kids are immersed, but the parents might only hear the game.  When games were in arcades, they were dedicated environment for game play and video gamers.  Once the gaming scene shifted inside the house, suddenly a room for gaming and gamers was the family room in the house.  I actually never grew up not knowing a home console-- my parents had an Atari that we watched them play occasionally.  When the Nintendo came out, we got that too.  Sanger makes the point that music for kid's games shouldn't be annoying, simplistic, or too repetitive-- not only does it irritate the parent, the child gets tired of it too and turns it off.  Composers take note: don't just stay in tonic!

I hadn't thought of the difference between music that is used many times versus music that is just for a one time game event and how those things compete for audio memory space.  For each time you make a piece that only plays once, there's less space possible for the themes that play for the bulk of the game.

A final point that really resonated for me was Sanger's comparison between the recording industry and game music.  As mediums change: LPs-->Tapes-->CDs-->Digital, the previous incarnations become not only forgotten, but unplayable.  Even though I still have CDs in my storage closet (and great ones!) the only working CD player I own is on my computer.  I don't use one with my home stereo stereo.  VHS-- I own a few still, but I've got no way to play them.  The same happens in games: as the consoles change through time, some games will become unplayable.  Game systems have a lifespan where games are developed and released for them and then technology moves on.  Yes, some of the past games will be ported and become available-- legally or illegally, but other games for whatever reason won't.  Eventually, the old game consoles will become difficult to work and break.

Most of all, reading this book got me thinking about the history of game music.  Will people care to look back to the first chiptunes once they're old and long forgotten?  What will happen as game composers grow old?  Some are moving into consulting, others publishing about game music, some are beginning to teach.  In classical music we learn the idea that rarely does a musician just do one of the following: perform, teach, write about music, composer, arrange, repair, or sell music or instruments... most often, musicians careers embrace multiple of these.  Perhaps this is true of game music composers as well.

For even more Fat Man, I found this video interview with Sanger shortly after the book came out.

For more current readings:  The Fat Man Blog on O'Reilly.  As of writing, last updated Jan 2009.