Middleware for music

Original Author: Byron-Atkinson-Jones

There is one aspect of game development that I historically had problems with and that was audio. I would concentrate all my efforts into getting the engine, game code and graphics right but audio would take a back-seat. In fact in the earlier years of my indie company I released games without any audio. It became rapidly apparent that I was leaving out a vital component.

When it comes to game development I was obsessed about being able to do absolutely everything myself. Not because I don’t trust anybody else to do it but rather because I have an autotelic personality and that means I have an insatiable thirst to learn everything. So when it came to doing the music and sound effects for my games I just had to learn how to do it. It’s a long arduous path but I’m now at a stage where I think I can do some basic composition, however if you were to ask my wife she would say that I haven’t quite gotten there yet.

As with all other areas on game development like coding, any musical ability I now have is self taught and my path of musical education took me along lots of different routes, some involving hardware like Synthesisers and others through software such as trackers, Reason and Ableton Live. My education is by no means complete and I suspect it will be a long on-going part of my life as I learn more and more. Like anything else in life it requires practice, practice and more practice to remain proficient.

Hardware or software?


In my earlier games I played live on a keyboard and recorded the audio straight through an audio interface into a sampler like Audacity. I then converted the sample into an MP3 file that I played in games. There was no track and my timing isn’t perfect. The pieces are all improvised, which really shows as some of them go wrong at some point. I’m getting better at playing the keyboard but it will be a number of years until I am proficient enough to be able to play it accurately, my fingering technique is all over the place. In some of the pieces I have more than one piece of hardware synchronized to either provide the beat or the bass lines.

As time went on audio production software, or DAW’s (Digital Audio Workstation) got better and now I tend to use them rather than play live. This has changed how I work because now the piece becomes more considered rather than let chord progression and intuition guide me. I still just sit at the keyboard and jam every now and then just to keep the practice in.

My particular DAW of choice is Ableton Live. I also have Logic Studio 9 and Reason 5 – both equally capable pieces of software but I find Live has the edge in ease of use and ability to simply experiment in composition.

If you are starting out the good news is that a DAW doesn’t have to cost the earth and there are some very capable cheap versions out there, in fact Ableton make a cheaper verion of Live called Live intro. It doesn’t have all the features of the full version of course but in order to make the kind of music I did for this blog you don’t need the full version.

Use the sample libraries

Early on in my musical exploration I came across the website of company that made sample libraries and they claimed their products were used on TV shows like CSI. As a CSI fan this drew me in like a moth to a light and I shelled out the cash to get it. I wasn’t disappointed and since then I went on to get most of the products they put out since they are of such high quality. The company is called Sample Logic and can be found at:

Don’t you still have to be proficient in music theory?

So, how are the various sample libraries helping me, don’t I still have to know a lot about music theory? Music theory will help me out and I still continue to read everything I can find about it but in the meantime these libraries are so flexible that all you need is an ear and some ability to tell when something doesn’t sound right.

Even though these libraries can seem to pull some musical miracles it’s not going to be a substitute forever for a lack of musical knowledge, for instance, what if you wanted to write a piece that doesn’t quite fit into the spectrum of genre that the libraries outwardly appear to support? For that you might need a bit more help from music theory.

I have no doubt that while something I put together from these libraries might be passable, in the hands of somebody who is proficient in music theory and practice, amazing things could be done. Until I reach that stage I am happy to keep experimenting and refining.

Decomposition

The piece I put together for this blog is an example of how simple it can get. The whole piece of music is made up five midi clips in Ableton live. A midi clip is simply a block of notes to be played in order so these five blocks form up the whole block of music.

The structure of the music is simply the order in which these clips are played and turned off. Ableton Live allows these midi clips to be arranged into scenes and inside these scenes you can turn clips on and off. So if you listen to the piece of music for this blog you will notice towards the middle end I turned off the guitars to let the lead loop play in relative isolation and then I turned the guitars back on again.

I have already said the contents of these clips are the notes that get played to make up the music, but given how complex the example music is just how simple can that get? Stringing notes together to make music can be complicated requiring lots of knowledge of rhythm, scales and harmony among others.

Well, in my piece each clip contains exactly one note and given that there are five clips that means the piece of music is composed of 5 notes in total, no more, no less. Yet when you listen to it there are clearly more, a lot more, what gives? This is the power of the sample libraries and the Kontakt player. Each note triggers a sequence to be played in the Kontakt player and that sequence can be very simple or as complex as Sample Logic can make it.

To show you how this piece was put together I made a screen-cast. In the first half I go through each of the clips, showing the notes contained within them (which as I already point out consists of just one) and then solo that particular clip so that when it’s played you just hear that one clip. Once each clip has been played individually I quickly go through the entire piece by playing the scenes that make it up which gives the music its structure. You can find it at:

That’s got to be cheating!

Is it cheating? I struggled with that initially so lets examine it a bit more. If I were to go to Turbo-squid and download a 3D mesh to use in my game is that cheating? What about if I purchase a texture set to be used? Sound effects? Each item on it’s own is just one small element in a larger piece that makes up your game; it’s the combination that makes the piece not just the components that make it up.

Entire genres of modern music were born from sampling other people’s records and putting them together into new arrangements. In fact many artists out there use stock samples that come with DAW’s in their songs. The artist Rhianna used a loop that comes with Logic Studio in her song Umbrella and once you hear the loop on it’s own you can’t but help notice it.

One UK periodical on modern music production went so far to claim that the modern use of samples was not too far removed from using a session musician, i.e. somebody who came into the studio to record parts that you don’t have the in-house capability to play. I’m not so sure I fully subscribe to that but the principle is sound enough.

There’s also another set of factors that comes into play and that is of time, money and ability. If you have a game that you have to make but you don’t have the time or ability to construct an engine to use what are you to do? Do you spend time and money to obtain the knowledge or do you go out, research the available engines out there and buy the rights to use one in your game?

This is partly why I call the types of sound libraries I have been talking about Middleware for music. They take a large part of the necessary knowledge and time out of the process of learning how to produce that kind of music and allow me to get my games out in a reasonable time frame.

Ultimately my goal is to make music that compliments my game and I will use whatever gets me to that goal and to be honest these tools exist to be used to make music so why not use them? In the end it’s up to you to make a decision about which path you choose.

Why not just pay a musician to make the music for me?

Good question, there are a lot of very talented people out there that are more than willing to make music for my games for a price and in reality that price might be a lot cheaper than paying out for the software and hardware. I would certainly recommend going down this route for the ultimate in timely music production, depending of course on the artist you find to do it for you. It would certainly take out some of the hassles of worrying if it was good enough.

However, having said that I personally find it difficult to describe what kind of music I am after. With art you can do a quick sketch to get your idea across but with music the most you are going to have is references to other pieces of music. This is where the really good musician comes into play though – they should be able to take you game, play it and get the feel of the kind of mood you are trying to capture and then create some music as appropriate.

When I sit down with my keyboards and sample libraries music is allowed to evolve and I am able to shape it into something that I had visualized. Sometimes I can get a bit carried away and the music I create has nothing to do with the game but it’s fun. As I said above I am a bit autotelic so I have this burning desire to learn everything I can about all aspects of game development, not just the code, but music and art, I have also enrolled on art courses to improve in that aspect too. It would be great to one day release a successful game and be able to say I did all aspects of it.

So what was the point of all this?

I guess the message I was trying to get across is that having no talent in music production should not be as big a barrier as it seems. Software and even hardware exists that makes the whole process of creating music a lot more open to the masses without the need of years and years of practice on an instrument. It’s not going to replace such dedicated work but it does allow you a foot in the door, which might otherwise have been unattainable. The technology exists to get those of us who would struggle to create music a step closer to being able to produce a mood piece for our games without too much blood, sweat and tears. It’s not a substitution for a really good musician but if it’s just you doing it then it’s there to be used.