Multi-Threaded Programming 2: Communication

Original Author: Joseph Simons

The next step in learning about multi-threaded programming is going to involve seeing how threads communicate with each other. This is important so that it is understandable why many of the pitfalls when programming with multiple threads exist. I am going to stick with x86-64 based architectures for simplicity, but this is pretty applicable to all computing devices from my experience.

If you want to read up on other things that I have covered, then here is my previous post in this series:


Reading From Memory

So to start, let’s see what happens when a thread reads from memory. First, the processor will request a load from a specified memory address to store it in a local register. Since we are just dealing with a single process, I will avoid discussing the details behind TLB, so just consider this a physical memory address.

This will then hit various cache-miss. If it had, then it would be called a cache-hit and the data at that address would get to the core much faster.


In this case the L1 and L2 caches are specific to each core.

 So after the address has made it all the way to the main memory of the computer, the data at that location begins its long trip back to the processor. Note that in the real world with virtual memory addresses, the location of the data could actually be on the hard drive, meaning that we need to wait for that slow thing to locate what we need before we can get access to it. Along the way back to our waiting core, each level of the cache is updated to have that data stored, so that any future access to it will be much, much faster.

Because each trip outside of the caches is slow, a single read will pull in data around the address requested. The size of this data is equal to the size of a single affinity correctly), this type of optimization can be difficult to actually implement.



I stands for Instruction, AKA code. D is for Data.

Writing To Memory

A program is pretty useless without writing something back to memory, so let me briefly cover what happens when a thread does just that.

It starts by specifying what data  to write and specific memory address to write to, just like with the read earlier. The core will execute this write instruction (typically referred to as a store), which will be put on the accordingly. Only after the cache line that has the updated data needs to be replaced will the updated data finally make it to main memory. Fortunately, the thread that executed the write instruction doesn’t need to wait until the write completes before moving on to the next instruction.

One thing you do need to keep in mind is that with modern volatile, which in terms of multi-threading in C/C++ (which I will be using for examples in later posts) doesn’t help you accomplish this.

When multiple threads are writing to the same data location, whoever executes last is typically the winner (the actual behavior may depend on the underlying hardware). However, since threads are controlled by the operating system, guaranteeing which one that will be is nearly impossible. So you have to be extremely careful whenever you have multiple threads writing to the same area. Fortunately, we have a few tools that will help us do this, which I am going to cover next.


Atomic Operations

Finally, we are going to touch on a vital piece of communicating between threads, and that is atomic operations. As I just mentioned, when dealing with multiple threads operating on the same data, guaranteeing the order of operations is nearly impossible. Even if one thread is executing ahead of another, that thread can be Atomic operations fill in this important role. These are implemented directly on CPUs as operations that cannot be interrupted (performing multiple operations in a single instruction with specific constraints), so they will operate in a serial manner regardless of other thread or operating system interference.

The fundamental atomic operation is the Compare and Swap. What this does (as the name implies) is that it performs a compare of the data before swapping it with different data. This is so you know that you are operating on data that has not changed (since another thread could have come by and beaten you to the punch).

Let’s look at a simple example of an increment using some pseudocode. Assume we have a Compare and Swap function called CAS that returns a boolean indicating whether it was successful:

// our function signature
  bool CAS(void* AddressToWrite, int CompareValue, int SwapValue);
  static int x = 0;
  local int y = x;
  while (CAS(&x, y, y + 1) == false)
    y = x; // fetch the new value

Assuming this is in a function that can be called by multiple threads, we need to protect against the possibility that any other thread can change the value of x from the time we read it into our local variable to the time we increment it. If that does happen, then we need to read in the new value and try again, keeping in mind that we can be behind yet again due to some other thread. However, we should eventually succeed unless we find ourselves in some situation where other threads are continuously hitting this section of code, which in a normal program would be neat impossible.

Also using our Compare and Swap function, we can implement a simple mutex lock. We can have a variable that will act as the lock value. Then we can attempt to acquire the lock by seeing if that value is 0 and then setting it to 1, and releasing the lock in a similar but opposite manner. Some psuedocode for those is below.


static int lock = 0;
  while (CAS(&lock, 0, 1) == false);



// technically this should always succeed assuming
  // we successfully locked it in the first place
  while (CAS(&lock, 1, 0) == false);


Next Time…

For the next post in this series, we are going to look at a couple of different simple algorithms and see how we can use what we have learned here in order to make them operate in a multi-threaded (or concurrent) manner.

If you have any interest in other areas dealing with multi-threaded programming, please let me know in the comments and I will see if I can make a future post answering any questions or topics you would like covered. Thanks for reading!

Don’t Lose the Beauty. Don’t Lose the Art.

Original Author: Alex Norton

I’m sure this will will bet yet another inflammatory article from me, and for that I apologize, but this is something that I strongly believe needs to be discussed. I’ll try to keep it short, but it is something worth thinking over, if only for a moment.

Firstly, an anecdote. I was at work the other day, and several colleagues were formulating a plan to purchase a crowd-funded coffee machine. They figured that if they got a dozen people to chip in for it, they’d be able to enjoy nice coffees for a small fee. Anyone who didn’t chip in would be stuck using the basic coffee machine. The plan was solid and they got their dozen people, in the end deciding on one of those new pod-based machines you see George Clooney swooning over.

When asked if I’d like to participate in this exercise, I refused, which shocked people, as I am a big coffee drinker and have one or two cups at work every single day. I also like fancy coffee, and, to my co-workers, what they were going in for was as fancy as they could get. I was asked why I wouldn’t participate and I told them that I enjoy making my coffee myself. I grind my own beans, I have a lovely direct immersion brewer and I take as much time making a cup as I do enjoying it. Due to this, I was laughed off as being a “coffee hipster” when I could have easily had the convenience of quality coffee at the press of a button from one of these “pods”.

Am I a coffee hipster? Yeah, a little. But there’s a reason for it, and it has nothing to do with coffee. It is, in point of fact, the topic of this article, as this exchange got me thinking.

The art behind a craft is a beautiful thing, and that art is being lost.

Now, this applies to many, many fields, but in my example above – with the coffee – my argument against pods is not in any way to be taken as my thinking the pods are bad, or that they make bad coffee. My argument is that if everyone relies on a pod and a finished machine to make their coffee, then people will forget – and thus lose – the beautiful art of coffee brewing. But, as I said, this article isn’t about coffee as such.

They exist in virtually every field, but on the topic of software development – my own field – many of you here reading these articles will have come across a coding purist. Many of you may well be coding purists yourselves; people who believe that – as an example – a video game or other piece of software which is not written in Objective C or C++ is not a real application – that it hasn’t been done properly. There are many arguments for this, of course, and I’m not in any way attempting to discredit them. I strongly believe that all programmers should acquire a thorough understanding of not only the “how” behind coding, but also the “why”. When you hit F5 (or equivalent) to compile your code, it shouldn’t be some magical process that one does not understand. The better you understand the underlying “why” of code, the better a coder you will be.

This being said, one cannot allow this way of thinking to be absolute. As the years – and indeed months – roll on, many new technologies are being crafted to make our lives easier. In coffee, there are pods. In video game development, there are development engines such as Unity, UDK, etc, and in other fields there are other technologies. These technologies are, in most cases, created by people who DO have a deep understanding of the “why” behind the field. They are simply creating a guided path through the art for those who would otherwise fear to tread it themselves.

To mock, or otherwise think less of, those who would seek to make use of these tools is to forget the purpose of the art – which is to create beautiful things. It may be a perfect cup of coffee, it may be an exceptionally fun video game, it may be a hand-crafted wooden box. At the same time, however, I believe that to rely too heavily on these tools can be equally as wrong, but I’ll clarify that in a moment.

I was witness to an exchange recently where a young man – relatively new to the field of video game development – was told he wasn’t a “real programmer” because he wasn’t writing his entire project in C++. Some of you may be nodding at this, thinking that these critics are absolutely right, but I ask you this: What does it matter what tools are used if the end result is fun and works well? The language which he used to make the game is merely one tool of many with which the game is made. If the man in question understands the why, the craft, of video game development, and the result is a beautiful thing, then how is his achievement any less wonderful than that of someone who has done everything from the ground up?

Now please don’t misunderstand me. I am a strong believer in learning a craft as thoroughly as possible. I believe that the knowledge of the “why” is too important not to obtain, but if you set out to build a house using only a hammer and a saw, you’re going to have a hard time. It will make the house no less beautiful if you use a power tool or three, and yet you will use those power tools much more accurately if you understand how to perform that same task with only manual hand tools.

In fact, this highlights the double-edged sword that is the topic. If one wants to make beautiful things, one cannot allow themselves to fall too far to either end of the spectrum. At one end you have the purists, who insist that everything should be done manually and traditionally, and at the other end, you have the ignorant (and I mean that not as an insult, but instead in the literal sense) who rely solely on pre-built tools which do their work for them. I propose that both are the wrong way to operate, regardless of the task at hand, regardless of the field. It is for us to find a happy medium somewhere between the two in which to craft our beautiful things. You may lean slightly one way or the other, but that doesn’t make you wrong.

In many cases, the end justifies the means – especially when dealing with a product to be sold. Hard work and thorough understanding show up very clearly in a finished product, and ignorance leads to flimsiness and potential unreliability.

It is for us to focus on the beauty of our craft. Don’t lose it. Think of how far things have come in your field since you were a child, and think what the landscape will be like when our children are our age.

ESports Experts Weigh In On Growing Opportunities

Original Author: John Gaudiosi

Also posted on:

Some top eSports executives convened at Game Connection Europe in Paris, December 2013, to discuss the opportunities in the growing space.

The advent of Twitch, Azubu and livestreaming over the past three years has helped fuel the explosive growth of eSports, or electronic sports. With companies like Valve (Dota 2), Riot Games (League of Legends), S2 Games (Heroes of Newerth) and Blizzard Entertainment (StarCraft 2) offering millions of dollars in prizes with annual tournaments and seasons and full-on leagues like Major League Gaming, Electronic Sports League and others turning pro gamers into cyber athletes; there’s a new way to connect with the elusive 18 to 34 year old male gaming demographic.

“When I first got involved in esports it was very niche,” said Steven Arhancet, director of eSports at Curse. “Back in those days we were talking thousands of people around the world and we’re joining proxy servers to watch games and download the demos. No one could really get involved. There were lots of small sites, but there was no real viewing for your average user.  You had to be quite specialized and involved in the scene. Nowadays thanks to the development of flash streaming anyone can be involved with the click of a button.”

Sam Braithwaite, director of eSports at S2 Games, remembers when eSports prize pools were $15,000 versus the $8 million Riot gave out this year. In the early days there were also issues with actual payments being made on occasion, which left some pros high and dry without social media to reach out to.

“Esports the past couple of years has grown in a professional way where every single person is accountable, including the players and the tournament organizers,” said Braithwaite. “With Twitter, Twitch and Facebook, professional gamers have a huge followings that would have jumped down those companies throats. But back then people could get away with almost anything. Companies could abuse professional players. They could get away with exploiting them and their fame to draw attention, and then stiff them. In terms of the biggest growth I’ve seen with eSports it’s definitely accountability and professionalism.”

ESports is something that is growing worldwide. The rise of livestreaming has opened up new opportunity for brands to connect with pro gamers during practice sessions. And then there are the huge global events that Major League Gaming, Electronic Sports League, World Cyber Games, League of Legends Championship Series and other big events that happen annually.

Riot Games sold out Staples Center League of Legends Finals as a pivotal moment for eSports. In addition to the 15,000 people who packed the stadium, over 32 million people watched the competition online.

“These events cost a lot of money to put on and can’t happen every week, but what’s really encouraging news is that teams in between events are getting huge numbers as well,” said Paul Kent, COO for Gfinity. “I’m aware of a team in France that’s less than six months old who are getting 9 million uniques per month. Two or three years ago that was unheard of. That’s the most encouraging thing, that people are now associating themselves with the teams and the players rather than just the big events.”

David Miller, vice president of sales and marketing at Azubu, said that the numbers that livestreaming companies like Twitch, Azubu and YouTube are getting today are already on par with traditional broadcast programming or sports content. With eSports growing at a fast clip, what the eSports industry needs to help legitimize it is a watchdog group to offer real concrete numbers like what Nielsen does for TV.

“There are a lot of different numbers that get bounded about like overall views, video on demand views and peek concurrent views,” said Miller. “There’s a lot of work that we all have to do to educate the sector. Video advertising is set to grow to $6 billion by 2017 with annual growth rate of 13 percent year on year between now and then and that’s great news for everyone in eSports.”

One of the keys is making the “suits” understand the potential of this audience. With companies like Coca Cola, Papa John’s Pizza, American Express and Mazda already involved in eSports, the tide is turning in favor of more money flowing into events. Miller said the fact that Twitch viewers are spending twice the amount of time glued to eSports programming as Hulu’s 55 minute average is also important for advertisers and sponsors interested in connecting with the elusive “gamer” audience.

Simon Bennett, head of eSports, EU, at Wargaming, said that eSports players are integral to the health of the game industry, especially with free-to-play games.

“It was just fans making events and enjoying themselves with other groups of dedicated individuals,” said Bennett. “For free-to-play games like World of Tanks and League of Legends, a fundamental part of the marketing strategy is to utilize eSports. It’s been proven that eSports players are worth a lot more than a normal player because they have such an influence over the community in general that they directly influence the purchase of gold items, skins, characters and other micro-transactions.”

ESports is not a fad. The Internet has eliminated the need for television, and allowed for the live consumption of events with huge engagement. Even practice sessions keeps fans glued to their screens, opening up new ways to connect with a core gamer demographic.

Need A Spot?

Original Author: Tim Borrelli

“The way this day typically goes is that if one of us sets a personal record (PR), the other one has to set a PR to keep pace.”

I wrote that in a training log a few months back, and it got me to thinking.

This kind of training applies not just to setting PRs, but to our overall philosophy of pushing each other to continually train and improve. The same kind of mentality should always be present in an animation team (or any team) setting- if we all work to improve and push each other to improve, our overall output will be more efficient and of higher quality.

Some days, we’re in the gym and one of us just isn’t feeling it. It would be easy to just quit, walk upstairs and eat some ice cream. It would be easy to just say “it wasn’t happening today” and have that be that. But we’re a team. Teammates are there to pick each other up when they need it, and to work with each other to figure out what will work that day in the gym. We don’t want to let each other down. We know that if one of us gave up, the other may not have a successful training session or worse- neither of us will make any progress towards our current training goals.

There are also times where even though we are confident in the lift we are about to do, we need someone to make sure we don’t injure ourselves- it’s called “spotting” someone. Having a spotter also allows a lifter to go higher in weight during a session- not only due to being safer from injury, but by giving a confidence boost to the lifter that they can lift the weight.

Animation is the same way. There are plenty of days when I or another member of the team just cannot figure out a pose, a motion, timing, etc. An animator will plan it all out, it’ll sound great in their mind, and when they go to execute, they just can’t get it. When this happens, the team looks at it together (i.e, we “spot” each other). We throw out ideas- some good, some bad, some that will work better for another motion. Some are just so over-the-top hilarious that we know we can’t use them, but have fun talking about them. The goal is to come away from that interaction with a fresh perspective on the motion and try it again. I know that for me, sometimes just that interaction with the members of my team is enough to get me over the hump.

This same interaction needs to occur when motions are completed and just aren’t working from either a style, gameplay or quality perspective- while I am the lead and do the initial approvals, I try to encourage the team to give feedback on everyone’s motions, mine included. This pushes each of us to continually put out high quality work that fits within the vision of not just the animation direction, but of the game.

For various reasons, however, not everyone is part of a team. This may be due to working better alone, being at a small studio, or being a student or jobless and trying to break in. Aside from those who prefer to work alone (who wouldn’t be on my team anyways), I encourage the rest of you to go find yourselves some teammates! Get on Twitter, post on deviantart or polycount or wherever. Meet people via online networking, learn from them, teach them, let them help you improve while you yourself encourage others to improve. Get a spotter, as it were.

Use these powerful forms of social media the right way and you’ll find yourself open to a world of people who are trying to do the same as you: improve their craft and become a meaningful part of a team.

Agile Game Development is Hard

Original Author: Rob Galanakis

I’ve spent the last few weeks trying to write a blog post about why Agile software development is inherently more difficult for games than other software. I searched for some fundamental reason, such as games being works of art, or being entertainment, or being more difficult to test, or anything about their very nature that makes game development different from other types of software development.

I couldn’t find one. Instead, I came up with reasons that are purely circumstantial, rooted in business models and development environments. Nonetheless, it is the situation we are in; the good news is, we can change it.

4+ Reasons Agile Game Dev is Tricky

Number one: the insane business model based on packaged games. Develop a game for years, market the hell out of it, ship it, profit, repeat. Crunching hard is probably in there, as is going bankrupt. Each year fewer and fewer games garner a larger share of the sales, and budgets are often reaching into the hundreds of millions of dollars to continue this model. This is pure insanity, so development methodologies of greater sanity, like those based on Agile principles, simply cannot thrive. Often they struggle to even take hold. Don’t underestimate the depth of this problem. We have a generation of executives and marketers (and developers) who know only this model, and trying to explain to them how you need to be flexible and iterative with releases and develop with tests can feel like a losing battle.

Number two: We’ve equated Scrum with Agile. Agile embodies a set of principles, but we’ve equated those principles with a (limited) set of tools: the Scrum project management methodology (you can substitute Lean and Six Sigma in the previous example; this phenomenon is not unique to games). If you’re ever tried to impose Scrum on an art team, you can see how much of a disaster it is. Rather than take Agile or Lean principles and ask “what is a good way to work that values these principles?”, we just institute some form of Scrum. I’ve seen many people dismiss Agile because Scrum failed, which is a shame. And like Scrum, I’ve also seen forms of soulless Kanban implemented (soulless because it doesn’t support the principles of Kanban, like limiting work and progress, managing flow, and understanding constraints).

Number three: Game development was late to the Agile party. Software has had about 15 years to figure out how to apply Agile to business and consumer applications and websites. While “flaccid Scrum” now seems common in games, that’s relatively recent; combined with multi-year development cycles in these so-called “Agile” shops, there hasn’t been much of the learning and reflection that underpins Agile. On top of this, Agile is in a period of maturity right now and is being appropriated by project management, so it is difficult to innovate in the methodology space to come up with an alternative to something like eXtreme Programming that works in game development.

Number four is pretty interesting: Game sequels are not iterations. It is very common to build up mountains of debt to ship a game, and then throw away and rewrite those mountains for the sequel. This worked okay because sequels were usually much more disruptive than innovative so there were more opportunities for rewrites. In contrast, consider that the MS Office UI stayed basically the same from 1993 to 2006. Now as games are entering a loosely defined “software as a service” model, our development priorities must change. We need to be able to iterate month-by-month on the same codebase and pull things forward. This is a new set of skills we need to develop.

There are a number of smaller items that are less important but still should be pointed out:

  • Game development hasn’t embraced open source and is on Windows. Many developers and executives I’ve met have a distrust of OSS (CCP’s use and support of Python and other OSS is a source of pride for me and others here) and the majority of game development is on Windows. The Agile movement has strong roots in OSS and Linux, so aside from the cultural differences between the two communities (which should not be underestimated), there was just a lack of engagement between game developers on Windows and Agile evangelists on Linux.
  • Game development reinvent wheels. The availability of lots of excellent open source middleware has given non-game developers a leg up on focusing on their core product. If you had to develop your product and webserver, you’d incur not just the cost of developing both but of splitting focus. Game development has historically done a poor job of using middleware and has often reinvented the wheel; this has probably historically been due to the desire for maximum performance and ridiculous deadlines and business models. With more hardware to spare, I suspect this will change and we’ll see things like HTTP used between client/server instead of custom RPC stacks.

Reasons Agile Game Dev is not Tricky

Finally, there are a number of arguments I have thought over and rejected, including:

  • Games are art and art cannot be iterated on like other software.
  • Games require too much ‘infrastructure’ to make anything playable.
  • Games want users to spend time, not save time.
  • Games are impossible, or at least significantly more difficult, to test.
  • Fat clients are difficult to distribute.
  • Frequent releases are confusing for games, which are traditionally content-heavy.

Call to Action

There are solutions to all of these problems, but it requires getting to the core of Agile’s principles, and even more importantly, the Lean principles those are based on. What game development needs is a new set of practices and tools, better suited to our technological problems, that fulfill the same principles and can be mixed and matched with existing Agile practices and methodologies. Some ideas or topics for discussion in future posts.

Multi-Threaded Programming 1: Basics

Original Author: Joseph Simons

This will be a series of posts in which I will cover a good portion about what I have learned in terms of multi-threaded development in the realm of video games. I am writing this since I have been inspired by informative series of posts here such as Alex Darby’s Low Level C/C++ Curriculum. So my goal is to explain how multi-threading works at a low and high level, as well as looking at a few common scenarios that can benefit from these techniques. I might even touch on a few situations you can get yourself into and some techniques for debugging them. I hope you will enjoy reading about it as much as I enjoy discussing it.


I am not the first person to make this joke…


What is Multi-Threading?

Lets start at the very beginning and cover exactly what a thread is and why having more of them (up to a certain point) is a good thing.

Every program that runs on a modern computer (such as your PC, game console, or smartphone) is a threads associated with them. These threads are what execute the actual code for the running program. Threads run code independently but share memory. This allows them to operate easily on the same data but perform different calculations. The fact that they share memory is the powerful, double-edged sword of their functionality.

A modern CPU is typically composed of several cores. Each core can run one thread at a time (though context switch, and while it isn’t exorbitantly expensive, you still want to avoid it as much as you can to achieve optimal speed.

The main parts of a thread are the local data storage. The program counter keeps track of what part of the code it is currently executing. The registers keep track of the current values of the executing code. The stack holds any values that don’t fit in the registers. And the local data is program specific, but generally keeps track of data that is local to that thread. This local storage is accomplished via a lookup table, managed by the OS, that will have each thread look up into its specified index to see where its data is kept. In short, this allows you to have variables that you can access like a global, but are in fact unique to each thread.

So, taking into account all of this, multi-threaded programming is when you use multiple threads in your program. Typically, you are doing this so that you can execute your program faster than you would using a single thread. Another common usage is to separate out your UI from the rest of the program, so that it always feels responsive. The difficulty in doing so lies in designing the systems in such a way that they can make use of multiple threads in these manners.


Show me your cores!

Why do we need it?

In short, we need this in order to make the most use of the available resources. But the real question is more likely, “Why do we need it, now?” and this is due to the trends in computing.

Since the dawn of the CPU, as years have gone by hardware manufacturers have been able to increase the speed of computers by increasing their 5GHz for consumer use, that left ten years where chip designers had to think laterally. And this means that one of the key innovations was putting more CPU cores onto a single chip. So no longer could you expect your games to be faster just by putting in new hardware, you had to design the game so that it could work well using multiple threads running on multiple cores.

One of the first areas where multi-threaded computing required games to adapt was with the release of the Microsoft Xbox 360 and Playstation 3, and for two similar but different reasons. The 360 had three cores with Hyper Threading technology (HTT), meaning that it could conceivably run six threads in parallel. The PS3 had a single core with HTT, and seven Synergistic Processing Elements (SPE), in a new architecture called the Cell Broadband Engine. The 360 had unified memory, so that all the threads could access all the memory. This made it easy to implement multi-threaded applications since you didn’t have to worry about who could access what, unlike with the PS3. Each SPE there had only a limited amount of memory that it could make use of, meaning that as a developer you had to carefully design your algorithms to take this into account.

Also, from a pure performance standpoint, the CPUs in the consoles were slower than their traditional desktop counterparts. While they were clocked similarly in speed, they were simpler chips that didn’t have a lot of the fancier technologies such as Out-of-order execution that made the most of the available power. So this forced developers to cope with multi-threaded strategies in order to make their games stand out amongst the rest.

In the desktop world, multi-core CPUs were quickly becoming the standard, but it appeared that developers were slow to adopt their programs to make use of the additional cores. A lot of this was likely due to two main factors. First is that the desktop world has a much broader range of hardware to support, and so they tend to design with the lowest common denominator in mind. Meaning that they wanted to support older systems that likely only had a single core available. Second is that multi-threaded programming is more difficult, and it takes a different approach to handling it. Couple this with the fact that a lot of game teams tend to reuse technologies, so in order to take advantage of multiple cores a lot of things would have to be rewritten. Also, graphics tended to be a big bottleneck in games, and submitting all the stuff for the GPU to draw was restricted to being done in a serial manner by the APIs (DirectX or OpenGL, typically). Only with the latest versions of the APIs (released only a few years ago) has it really been possible to make the most use of all the available cores in modern CPUs. And now with the latest generation of consoles upon us, developers have no excuse to not use heavily multi-threaded game engines for the top tier games.


Next Time…

For the next post in this series, I will cover how communicating between threads works on a hardware and software level. This is important to understand what is exactly going on with the CPU in order to avoid many of the potential pitfalls that are inherent in multi-threaded programming.

If you have any interest in other areas dealing with multi-threaded programming, please let me know in the comments and I will see if I can make a future post answering any questions or topics you would like covered. Thanks for reading!

Next Post – Multi-Threaded Programming 2: Communication

The Lasting Excellence of Legacy of Kain: Soul Reaver’s Writing

Original Author: Ben Serviss

Raziel in Legacy of Kain: Soul Reaver.
Raziel in Legacy of Kain: Soul Reaver.

Legacy of Kain: Soul Reaver is widely regarded for its excellent writing. There’s a reason for this that may not exactly surprise you: its writing is excellent, and deserves to be studied as an exemplar of what game writing can be.

First released to critical acclaim in 1999 for the original PlayStation, Soul Reaver had many things going for it. It had an intriguing “phase shifting” gameplay mechanic that innovated in design while stretching what was technically possible on the PS1; excellent voice performances across the board; an open, free-roaming world two years before GTA3’s release on the PS2; and of course, that fantastic writing, penned by Amy Hennig, who went on to write the Uncharted games.

What made the writing so good? First off, the phrasing and language was consistently artful and well-composed. When most games were content with throwing cliché after cliché at you, Soul Reaver strived to be imaginative and poignant whenever possible.

Before you go any further, it’s highly recommended you watch the opening cinematic to the game, embedded below. I’ll discuss this cinematic in greater detail below as well.

Aside from the text itself, the game’s high-level narrative design is expertly constructed to make the world more engaging for players. In Soul Reaver, you essentially play a zombie vampire who runs around killing monsters and eating their souls for energy. Yet by humanizing the events that lead up to player character Raziel’s destruction and rebirth, the focus is on the characters and not the spectacle, creating meaning where most games are content to throw set pieces and playthings at the player in an attempt to cover up the absence of meaning.

The opening cinematic also expertly positions a specific game mechanic: gliding. On its own, gliding around a level doesn’t seem that compelling. But when you factor in the narrative reason behind why Raziel glides and not flies, it increases the significance behind the mechanic itself, reminding you of Kain’s brutality and Raziel’s fall from grace every time you use it in gameplay.

But it’s not just the writer’s language, or the game-specific narrative sensibilities that make Soul Reaver special. At the core, it’s the mastery of the craft of writing that elevates the game’s writing to heights occupied by few.

– Line By Line –

To fully appreciate how well-crafted Soul Reaver’s writing is, let’s analyze the cinematic you just watched, line by line, and see for ourselves what makes it work so well.

Let’s begin.

“Kain is deified. The clans tell tales of him. Few know the truth. He was mortal once. As were we all.”

First off, for a game rooted in lengthy backstory, there is strikingly little in the way of spoon-fed exposition. Though the game is a sequel of sorts to the 1996 game Blood Omen: Legacy of KainSoul Reaver takes the game in such a new direction that it’s fair to assume that players may not have prior knowledge of the universe or characters.

Even so, the cinematic begins by immediately addressing real things of import, with zero hand-holding for the uninitiated.

“Kain is deified.” Immediately a question is placed into your mind: Who is Kain? The cinematic doesn’t start pedantically with “The world of Nosgoth is filled with vampires and demons.” From the start, the writer’s confidence that you can handle complex scenarios compels you to sit up and take notice.

“The clans tell tales of him.” What clans? Another question placed into your mind for later. The word “clan” is evocative of feudal systems, which isn’t far from the truth in this case.

“Few know the truth.” There is more to Kain, whoever he is, than what he seems – but what? Yes, it’s another question introduced into your mind.

“He was mortal once. As were we all.” The question from the previous line is resolved – but in its place, two more appear. So Kain is immortal now? And who are the others that the speaker mentioned? By alternating posing new questions and giving answers, the writer engages with the viewer, making something as passive as a cinematic an interactive experience.

“However, his contempt for humanity drove him to create me and my brethren.”

With such a degree of “contempt for humanity,” Kain is clearly set up to be a bad guy. But this prompts a question – who is the speaker? If Kain made him, then he can’t be natural – what is he?

“I am Raziel. First-born of his lieutenants. I stood with Kain and my brethren at the dawn of the Empire.”

A straightforward answer to the previous line – now we know who the speaker is. Clarifying this link will become important later on as the writer establishes empathy for Raziel in the player. Also, Raziel is explicitly described as Kain’s first born, inviting a father/son dynamic that comes into play later.

“I have served him a millennium.”

The extent of Raziel’s loyalty is established, as well as the vast lifespans of these creatures.

In an excellent example of writing for economy, the writer communicates both of these ideas with a single line.

“Over time, we became less human and more… divine.”

Another example of a superb line written to perform multiple functions. As Raziel says this line, the camera is firmly set on Kain’s horrid face as he opens his mouth to bear his fangs, illustrating the inherently contradicting ideas of the vampires’ monstrousness balanced by their sense of self-perceived eloquence, a hallmark of the series.

The second function is easier to catch on repeat viewings. At this moment, Kain is recoiling in surprise at seeing Raziel’s wings, setting up his act of punishing Raziel moments later.

“Kain would enter the state of change and emerge with a new gift.”

A bit of misdirection here – it isn’t really clear what this means until you see Raziel unveil his wings a little later. Regardless, this line sets the viewer up for a quick rush of retroactive insight when you finally see Raziel’s wings.

“Some years after the master, our evolution would follow.”

Continuing the set up from the previous line, leading the viewer along…

“Until I had the honor of surpassing my lord.”

The first payoff from the set up. Suddenly, you realize why this is a big deal – Raziel has clearly passed a boundary that had not yet been breached.

Kain’s vague initial reaction gives no real hint as to what he will do, and his tentative inspection of Raziel’s wings builds tension and suspense, since we don’t know Kain well enough to predict how he will react.

“For my transgression, I earned a new kind of reward.”

Keeping the tension in for a final moment as Kain circles around, raising his claws…


Kain slashes down, irreparably destroying Raziel’s wings. This moment is not only the crux of the opening cinematic, but it’s the inciting incident for the entire game. It sets the events of the game in motion, shows you just how spiteful and cruel Kain is, and sets Raziel up as a tragic figure that you feel for. It does all three of these things in one powerful, meaningful moment.

It is now 1:37 into the opening cinematic, almost halfway through the 3:28 running time. So far, it’s established the dark, brutal world of the vampires, their eons-long life spans, Kain’s position as creator of his minions, the vampires’ nature of evolving into different forms, the depths of Kain’s cruelty and pettiness and Raziel’s fall from favor – and all in a compelling way, without resorting to blatant exposition or spectacle. The focus is entirely on telling the story.

Let’s continue:

“There was only one possible outcome: My eternal damnation. I, Raziel, was to suffer the fate of traitors and weaklings, and burn forever in the bowels of the lake of the dead.”

Explaining the logic behind Kain’s brutal brand of justice – that the only way to deal with his eldest subject even hinting at a challenge to him would be by execution by such a horrible method – tells you all you need to know about him.

“Cast him in.”

This line is notable for several reasons. It is the only line Kain says in the entire cinematic – and it is the command that is directly responsible for damning Raziel.

Before he says it, Kain consciously walks away from Raziel as his minions bring him to the edge, revealing that his disgust is so great he can’t even bear himself to look as he condemns Raziel to death.

Finally, it is also the only line that is lip synced by a character in the entire cinematic. This gives it a remarkable immediacy and uniqueness – made all the more frightening for what it signifies.

“Tumbling, burning with white-hot fire, I plunged into the depths of the abyss. Unspeakable pain. Relentless agony. Time ceased to exist. Only this torture, and a deepening hatred of the hypocrisy that damned me to this hell.”

Since we already feel for Raziel’s plight after witnessing Kain’s uncalled-for cruelty, this lengthy description of his fate furthers this empathetic connection, showing you the ramifications of Kain’s ruthlessness. Making you hate him even more.

“An eternity passed, and my torment receded, bringing me back from the precipice of madness. The descent had destroyed me, and yet… I lived.”

Raziel has been destroyed, but has somehow been reborn. The reveal of his missing jaw and blue glowing eyes hints at an unholy quality, but you don’t remark on the monster he’s become. Instead, as he throws the scarf over his shoulder to hide his deformity, you feel for what he’s gone through and lost.

This becomes important during gameplay, as when you press the circle button to hold open his scarf and literally inhale the souls of slain enemies through your desiccated throat hole, you still feel that Raziel is a noble creature put upon by Kain, and not a horrid monster in his own right.

“Raziel. You are worthy.”

The journey concluded, the writer leaves us with more questions. Who is speaking now? Are they involved with Raziel’s resurrection? Why do they think Raziel is worthy? Note that the idea of whether Raziel is worthy or not comes directly in opposition to Kain’s quick judgment to toss him out as someone unworthy of life. In other words, finally, at the end of his suffering, there’s a reason to have hope.

– The Stage Is Set: Now Play –

It is exactly when the player has been introduced to the world (a vicious vampire kingdom), the main players (Kain and Raziel), the dynamics between them (master/servant; father/son; tyrant/martyr), the values at play (jealousy/acceptance, cruelty/mercy) and the protagonist’s goal (revenge) that the game begins. The player is now ready to fully live in this world.

The tight construction of the opening cinematic is indicative of why Soul Reaver’s writing works so well. There are no pointless CG set pieces or elaborate action sequences that do nothing to further the story. Everything here is driven by the purpose to convey story in an efficient, compelling manner – and to this day, it all works beautifully.