breakingbread: From the ancient pyramids of Anubis, to the modern
skyscrapers of the Lijiang Tower. Overwatch... Ouch...
A game once crowned as game of the year in 2016, beloved by players all around
the world, and even getting a Metacritic score of 91 out of 100. And a game
with a steep downfall, update draughts with no new content, real seasonal
events, and slow balancing changes have upset the community for years.
Finally put into eternal slumber by the management of
Blizzard Entertainment on October 3, 2022, it was replaced
by an arguably inferior version of the original project.
Players who "purchased" the game were out of luck.
They had no choice but to suck it up and play their all-new and improved version.
But that's not why we're here today.
Today, we'll be exploring all the intricate little details that
made this game, and hopefully, at the end, revive the version that
you want to play, and not what a game company dictates unto you.
So ladies, gentlemen, omnics, and other sentient beings, welcome
to my presentation, and welcome to Reverse Engineering Overwatch.
Or, rather, how Overwatch developers accidentally gave me
an instruction manual on how to reverse engineer the game.
Whoops. Part one. Before we start, let's go over the boring stuff. Like, who am I?
Hi, I'm Lena. But online, I mostly go by breakingbread.
I've been programming for about 10 years now, and I'm 23 years old.
I'm currently studying mechanics and electronics.
I have a love-slash-hate relationship of computers,
and maybe that's just Stockholm Syndrome.
Who knows? And I am an occasional fan of Overwatch.
I've been playing since 2016 and still love Overwatch to this day.
This entire project of mine was and is a labor of love.
Overwatch is my all-time favorite multiplayer game.
Without it, I wouldn't have met my best friend and many other people
I cherish today, and probably wouldn't have started programming.
That's why I would sincerely like to thank everyone, the people
on screen right now, who have worked to bring Overwatch to life.
I still believe that this game engine is beautifully designed.
In my past 10 years of playing, I've barely ever experienced any crashing.
Hell, any bugs at all.
Regardless, I will be dunking on developers a bit during this talk,
but remember, everything I'm about to show you is pre-release code,
and probably had been fixed and improved upon in the past few years.
At least I really hope it did.
So, without further ado, how the hell do you even reverse engineer a game?
When I started in 2024, I had really no idea on how to do
that, and to be completely honest, I still have no idea.
Don't be intimidated by reverse engineering, though.
It is scary to start a new project, and yes, sometimes adventures just pay off.
Let's backtrace, and let me show you how I got here. So, what do we know?
Well, not much. Overwatch was created out of a failed project codenamed Titan.
It reuses many of the same systems and underlying rendering engine from
it, as well as other proprietary software within and outside of Blizzard.
These include Tank, the rendering engine, Domino, the
proprietary physics engine, and BS Core for networking.
Plus, there are some other miscellaneous external dependencies, such as
open source software, and Enlighten for Lighting, and Vivox as a voice chat.
So, let's just dive in head first.
Now, we have to ask ourselves the important question.
What version do we actually want to start with?
Using the latest version has the benefit of being able to capture client
-server network traffic, and the ability to debug the game, which may
help in later versions of development, but it also has its drawbacks.
An ever-evolving anti-cheat, which slows down progress a lot.
Constant updating of the game, so you need to stay up-to-date with the latest
changes, plus very mature systems and, presumably, less debug information.
That was removed over the years.
Since the game engine is deterministic, though, and the client
needs to be able to assimilate almost everything that the
server does, the client should also be mostly feature-complete.
So, I chose the beta version 0.8 from 2015, which was uploaded by
a kind stranger to archive.org, and I chose it for several reasons.
The game is at its earliest state.
All major technologies and systems already have to be present in one form or another.
That allows us to get a basic grasp on how the game functions,
making reverse engineering of later versions much easier.
Some debug assertions may also still be there, which got removed in later versions.
In retrospect, not entirely. And there is a worse anti-cheat.
As far as I know, there is neither a network dump, nor a dump of the running game.
This may make development harder, but have you
considered that it's just pretty cool now, isn't it?
Getting to play the Overwatch version, which most people were never able to play.
Since this is the first version, not much of the anti-cheat was in the executable.
Everything can be removed, and the original code before obfuscation can be recovered.
I won't go into the specifics, though, for obvious reasons, but let's just say that
executables for any other platform than Windows don't have any anti-cheat at all.
Quite the opposite. Some even have more debug assertions.
You should probably take a look at that Blizzard.
So the de-obfuscation is a challenge for the viewer.
That's really the only thing I have to say.
Except, at some point, it is your own fault that
someone starts to reverse engineer your game.
Part two, some interesting stuff.
The first thing you can see when opening a cleaned executable
is all the strings, which have been invisible before.
There are about 20,000 to 30,000 strings, depending on the game version you have.
On Windows, about 5,000 of them are "encrypted".
Let's take a look at some examples.
"ClientInInstance", for example, seems like a messaging protocol.
In fact, it's JAM, which we're going to get to later. VM and ViewModels.
Obviously, user interface code needs to interface with some other components and stuff.
Think of the play button, or of keybinds, or the hero select screen.
And I'm really glad that some of this stuff even has names.
And "Failed to Load Statescript Graph".
This is really cool because they forgot to remove a lot of debug assertions and
strings from state-script, which really helped in the reverse engineering process.
Fun fact, they also forgot to remove the list all command options function in the beta.
Just enter an invalid option, and it spits out everything.
Overwatch also saves some game settings into the .ini file.
Usually, not many of them are set invisible to the user, though.
There are quite a lot of them, though.
On screen is just an example of some of them.
There are more than 200. Those can get changed during
runtime, although that may have not been wanted.
JAM stands for Joe's Advanced Messages.
Now, you may ask my dear viewer, who is Joe? And he has an answer for us.
Fortunately, Joe Rumsey gave us a presentation about this exact
system in 2013. It was, and still is, extensively used in World of
Warcraft and Overwatch, although Overwatch uses a different version.
At the heart of it, it's just a simple way to describe
messages for server communication, similar to Protobuf.
Messages all have a unique ID and are grouped in something called protocol pools.
For this talk, though, this is pretty much all we need to know about.
The implementation details are/ were not really important for me,
since I never planned to reimplement this mess of a system anyways.
We can hook everything in the executable, so why use many word when few word do trick?
Hooking the functions that associate protocol pools and some various other
information gives us a tale, which already looks much friendlier than just strings.
The most important thing you see on the screen is the
static pool name and the ASSOCIATE columns right there.
Let's zoom in a bit so everyone can see it.
The relative virtual address in the ASSOCIATE columns leads the
dispatch function to process incoming messages from the server.
Let's take a look at an example, "ClientInLobbyConnect".
This dispatch function itself can be boiled down to being only one big
switch statement, with the second argument being the JAM message itself.
It already got deserialized in a past stage, and it's just a C++ class.
And yes, an error is indicated by 1.
I was stuck for a long time trying to figure out why I could only mess
around with the login screen, until I took a look at message 5017 right here.
The function here loops over some structure it receives
from the server and puts it into a hash map of a class.
At this point, I did not know that this was the CASC storage handler there.
So let's talk about CASC.
Content addressable storage containers is how Blizzard
saves game assets on disk, Overwatch is not the exception.
Let's take a look at an open source project which implements CASC, CASCLib.
Data is encrypted and gets decrypted by keys sent from the server.
So let's take a second look at the JAM function and at the cross references
of the class it references, and suddenly it becomes much clearer.
This is the first non-stream CASC key being
emplaced right here in this initialization function.
Okay, if that JAM message emplaces CASC keys streamed by the
server, why don't we just call it with the keys it expects?
Let's emplace all the CASC keys at startup and call the
JAM handler function to load all the data packs it expects.
And... we're in the lobby screen. Part 3.
Now this is where it really gets interesting. Resource loading.
So now that we have somehow managed to convince the client to go into the main
menu screen, I wanted to know what exactly I was even doing, which was a mistake.
The resource management is cursed.
Keep your sanity and just let Overwatch handle everything itself.
But if you're sadistic and interested, there are many types of resources.
Here's just a few of them. Resources are identified by eight byte IDs.
Four bytes are for miscellaneous stuff such as the
resource type, the region, platform and locale.
And four bytes are reserved for the actual identification.
The ID is just an incrementing ID, which is not a name hash, unfortunately.
Easy and straightforward?Well, no. Let's take a look at an example.
This resource ID here is the entity ID of the tracer gameplay ID.
It has an incrementing ID of 1. So the resource type should be hex 400, right? No.
It is actually 3. Now, why is that?
Well, first, a bit mask of the resource ID is taken.
This will result in only the last two bytes remaining.
Once this is done, the entire thing is bit flipped.
Which basically means the most significant bit is now the least significant bit.
And so on. So the type is 2? No.
To put the cherry on top, the number then gets incremented and now
we finally have the actual type specifier for an entity definition.
3. This is used extensively in code.
So if the game needs to combine an incrementing ID with a type specifier,
it always calls the flip bits function and does this operation in reverse.
Why this is being done, I have no idea. And at this point, I'm too afraid to ask.
Structured data.
In Dan Reed's presentation about networking scripted abilities, he
briefly describes the Overwatch exclusive data format, structured data.
This data format is the basis for a lot of the game's code, such as
system configuration, model configuration, animations, UX, styling and
layouts, and, you know, just one thing he forgot to mention as well.
Server and client data serialization. I wonder why he neglected to mention that.
Also similar to Protobuf, STUs are compiled.
No compiler or STU files have slipped out of Blizzard yet though.
An STU instance is just a C++ class with some virtual functions and some data.
They are only data and have no game logic, with one exception,
supports inheritance, attributes and runtime reflection.
Which means that there must be some supporting metadata somewhere. And there is.
It took me embarrassingly long to find out how to read this properly.
At the heart of it, it's just all the fields you would really expect,
like the name hash, the size of the instance, a factory function, field
specifications with default values and constraints, and some other fields.
This right here is just an example of a virtual function table of an STU class.
Most of those functions only return zero or one.
They got removed in subsequent versions though. What does that leave us with?
In the end, we get something like this. The STU Explorer.
Being able to perfectly decode and display structured data is something
I have postponed for a bit too long to the detriment of myself.
STU does show that it was a good decision to start with the beta.
There are two different kinds of STU.
Probably because the whole thing is a mess, as you will shortly see.
Version 2 was released at around game version 1.14 and apparently
removed a lot of stuff, including the CRC32 name hashing.
That's why you see all the names on here, because that's version 1.
Version 2, in this case then, means that it's better than version 1. I would like
everyone to now please give a big applause to the guys and gals of DataWatch.
They just casually have three files right here, sitting with
over 2,000 CRC hash-to-name mappings sitting in the GitHub repo.
One last thing before I move on to the next topic.
Let's go back to the STU Explorer and have a look.
There are quite a few fields which are lists.
This is a structure I lovingly call STU BullshitList.
So why, my fellow Brigitte enjoyers, did I call this structure BullshitList?
Well, see this int64 field called flag right here.
If it is 1, the first void pointer is the pointer to the list data.
Otherwise, it's turned around.
In essence, the list will look different depending on what flag is used.
I have no idea why this is done.
And I will also spare you how STU maps work, because that one's even worse.
Statescript. Is the heart of the gameplay and anything that happens in-game.
Was documented by Overwatch developers themselves, so thank you for that.
Is a great piece of code. Basically, it's a graph-based scripting language.
It controls some movement, such as Tracers blink, animation, spawning of
cosmetics, abilities, some user interface interactions, and much, much more.
Scripts can either be synchronized with the server or unsynchronized. Think music.
I need to rush a bit over this incredible system, though, because of time constraints.
If you want to know more about it, there is always
the presentation from Overwatch developer Dan Reed.
In the heart of it, there are four different types of nodes.
Entry points, of which there are just two.
Play script, which sends a pulse immediately on script load.
Or game message, which sends a pulse if a "game message" arrives,
which is just a resource ID, with what we've discussed earlier,
with some optional STU data that Statescript can process.
Actions. Stuff that gets executed immediately, like playing sound effects. States.
Execute long-running tasks in the background and
sends pulses on different output plugs accordingly.
States and other nodes can have multiple output plugs, which can get pulsed.
Conditions.
If a pulse reaches a condition, it checks for some config var and
sends it along the way, accordingly, depending on the results.
In total, there are two different types of entry nodes.
113 actions, 157 states and four types of conditions.
Not all too many, honestly. Statescript nodes are configured by config vars.
There are about 990 of them and they are able to access game data.
For example, an entity's position or team, filter entities
of a condition, or even execute a more complex expression.
For example, if variable x is greater than zero.
There's one special type called ConfigVarDynamic,
or as Dan Reed called it, state script vars.
These can be changed by the script and even by other scripts and entities completely.
Statescript state nodes subscribe to these variables and get notified
before and after they get changed. Dynamic variables can also not
only be a primitive value such as an integer, string or similar,
but also a dictionary of values or a reference to a running script.
They're stored in var bags in both the script itself and the entity.
The entity does have three of them. One is always allocated.
And another one only when the entity is the local entity.
The third one, I don't know. I've never really seen it referenced anywhere.
And during my exploratory process, I also accidentally reinvented math.
You can thank me later for that. Part 4: Systems.
Let's talk about how all the different game systems play nicely together.
Overwatch doesn't go the traditional way of other game engines such as Unity and Unreal.
Rather, it makes use of an entity component system architecture.
In an ideal world, this means that entities have components
with state and systems just iterate over components.
Let's take Tracer as an example.
It has a health component, a movement state
component, and a Statescript component amongst others.
And here's just some systems I just wrote down off the top of my head.
In reality, there are many more and they do differ based on the entity admin.
Speaking of, entity admins.
An entity admin is just a fancy class which contains all the entities,
systems, and factory functions, and other metadata for the system itself.
Overwatch actually doesn't only have one, it has four.
The lobby entity admin, which has some singleton components, which means
that they only exist once, like your current logical button states.
It is mostly responsible for UI and login stuff.
The lobby map entity admin, which is pretty much the coolest entity admin.
It's an entity admin within an entity admin.
Its only purpose is to load and display the background map for the lobby
menu and is located in a singleton component of the lobby entity admin.
And finally, the game replay entity admins, which interests us.
Everything in-game is contained in here.
When a game is replayed, think killcam or play of the game, it seamlessly
switches to the other one and simulates an entirely different world.
MapSystem. It is comparatively simple to trigger the client to do a map load.
The game's server just sends one JAM message, which sets the
map system to loading and emplaces the wanted map resource ID.
They were even so gracious to leave a map load state enum to string function in for us.
So let's do that.
Just put in a map ID and set the state to 2. And, when
doing this, Overwatch rewards us with this lovely site.
The Numbani loading screen does really look beautiful.
There's only one tiny little thing left to do, spawning entities.
Entities, if they're not effects, the map itself or static
singleton components, get spawned out of STU entity definitions.
The class itself looks like this.
All information in the game or all information
the game needs to spawn an entity is inside here.
Fun fact, skins just overwrite some components in these definitions.
So they basically made children gambling addicted to change an 8 byte value.
Fortunately, we can leverage one class which will do all the heavy
lifting of spawning entities for us, like getting all the component data.
So let's try that. Yes, so we're now finally in game and can move about, right?
Not so fast. I was neither able to move, nor to aim my camera around.
Players in Overwatch are not, as one might be led to believe one entity.
There are actually at least two, a controller and a model entity.
The controller holds information such as your account
information, connection state, and the currently selected hero.
Meanwhile, the model entity is the actual model which you move around
in-game, with its state script instances for abilities and such.
Those two entities are referenced by entity IDs.
The system which handles spawning is called the mirror system.
Unfortunately, it basically just deserializes updates from the server,
so there's no real information on how this all did work in the beta.
But as far as I can see, it hasn't really changed much.
But the game obviously knows how those two entities play nicely together.
Here's some notes I made when exploring the
systems which handle and references both entities.
Combining this information leaves us with this.
A controller contains a component with three entity IDs.
They tell the game which entity the player aims with, moves with, and is seen on screen.
Or what the player sees on screen.
A model, on the other hand, just contains a reference back to the controller.
So that's how Overwatch handles all things, handles things
such as D.Va, and presumably in later versions even Echo.
Part 5. Let's go to the live demo. That's a bit too far.
Let's just activate this little... Okay.
Screen scaling seems to be working perfectly. Yeah. Does it work?
*Overwatch music plays* No.
We're just going to have to make do without resizing the screen.
I hope that everybody can see it, since the beta version
of Overwatch really doesn't like being scaled up or down.
So, unfortunately, you'll just have to zoom in IRL. I'm sorry.
Now, once we press this little button, you can see that
we are developers, of course, so we have dev mode enabled.
And can now finally look at the main menu screen. *Overwatch music plays*
Now, unfortunately everything right here is server controlled.
So, if we go into the tutorial, it basically just
tells the server, hey, please let me go in-game.
But I prepared a little demo for you, which we can hopefully watch in a few seconds.
Thats ominous music, that is not supposed to happen.As you can see,
Overwatch: Travelling to Hanamura *Overwatch music plays*
breakingbread: a lot of this game is very scriptable, so you can do a lot of stuff.
Unfortunately, hero spawning is also done by the server.
But for this demo's sake, I just took that function and...
Tracer: Cheers, love! The cavalry's here!
Nice!*Applause*
Now, unfortunately, we can move around now, but for example, recall doesn't work.
It does depend on the server to tell it what to do. So, let's just walk outside.
That is significant delay, unfortunately.
Now if I can start the second demo that would be awesome. No that is wrong.
Here we go. Let's start this instance.
Unfortunately, I wanted to do this with another player, which sits in
the audience, but due to some technical difficulties, that does not work.
So, we're going to have to do it locally. So, nice. Open it with a second screen.
Overwatch: Travelling to Hanamura *Hanamura music playing*
breakingbread: *Music deterioates* Hold up. That is a very bass boosted demo. Nope. And about five FPS, perfect. Let's see. Second try before
I just give up.
Overwatch: Travelling to Hanamura *Hanamura music playing*
Tracer: Cheers, love! The cavalry's here! *Gameplay sounds*
breakingbread: Haha, hello! Now unfortunately I'm really torturing my little Framework
laptop right here, so that is only like 5 FPS.
But yeah, as you can see, it is possible to make a game server,
and it is possible to spawn a second hero. Let's go back.
One other thing which I want to show you, is the
beta had something called a map system right here.
Yeah, the weapons are shooting, but basically, you can see a map over
here, which I don't even think that Overwatch 2 has at this point.
So, that's probably just some remnant of Titan. I don't know.
As you can also see, that is not Hanamura, but this
is Anubis. So, it is only one map. So, let's go back.
Let's see if that works. Yes! Okay. So, as you
can see, I was able to figure multiplayer out.
About one month ago, I was just casually strolling through Overwatch and
its networking system two days before having a very important math exam.
And I found out that... I found one peculiar class called bsObject
and bsDataStore. At least, I think it's called bsDataStore.
I haven't found anything public, which is named similarly, but I
do think that it's BS stands for bitstream, which doesn't use one,
but it uses these three different system for networking protocols.
Why they didn't use STU and have done their own bitstream
serialization for movement and movement state, I have no idea.
Fun fact, if I've read that code correctly, the beta has a fixed update rate of 62.5 hertz.
But, yeah, I must disappoint you a little bit.
The demo was, as you can see, held together by thoughts, prayers, and flex tape.
Nothing else than the movement state is synchronized.
No state script, animations, breakable entities, etc.
But, we now have the possibility to do so.To fix the game.
It is finally time for the community to liberate Overwatch and create our own game.
The one you and me envisioned all over the years.
That right here is just an incomplete list of what is still left to be done.
Now, you may ask yourself, when will my research and my tools be released?
And to that I say...Please go wild!
And it does include, but it's not limited to, a bad window manager,
editable STUs, really bad network serialization, misaligned resource
struct, which it is so bad, that's something from Overwatch, pain.
And unfortunately, I could not fit everything which I have found in this talk.
In the next few weeks, I will gradually extend my project with
all the structures which I have already found and defined, though.
But please give me a little bit of time.
I need some holidays off from Overwatch and university.Conclusion.
Overwatch can be read like an open book.
There is enough debug information so we can create our own version.
The game engine is beautifully designed, especially the entity component system.
And this game will not forcibly be taken away anymore.
It's time for us to make the rules.