Thank you. It's great to be back in Hamburg after 12 months, essentially.
I have a lot to talk about today, so I really don't want to bore you too much about
my bio, so I will instead have my presentation glitch out right in the beginning.
What's happening? There we go.
So, as *unintelligible name* was already mentioning,
I kind of am a person that likes a tough challenge.
Like early in university times, I was looking into video game console emulators.
It was always mind -blowing to me how you could get games running
on a platform that they were never intended to be ran on before.
There was lots of stuff to learn about, lots of cool
people to meet, and it was just an amazing experience.
During uni, what I was doing instead was studying physics.
I really wanted to go into academic research and research theoretical
physics, understand how our universe works, you know, just
everyday things, and there's a few funny pictures from that era.
It turned out academic research wasn't really my thing, so what I ended up
doing instead after uni was to get back to software development and join
Imagination Technologies, where I worked on Vulcan Driver for PowerVR GPUs.
One thing led to another, and I was doing the same kind of work,
but for the open source Mesa project, where I was working on the
Vulcan Driver for AMD GPUs and on the shader compiler in particular.
So, also lots of valuable lessons.
So, after all of these tough challenges, we'll look
at a different challenge today, and that's FEX.
Maybe? Oh, this is starting out amazingly. I love it. There we go.
Wait, not that FEX. There we go, FEX-Emu.
So, we're an x86 emulator for ARM systems.
And actually, I wanted to get to know you a little bit here.
So, how many of you, show of hands, would call themselves
gamers, whether casual or, like, really hardcore?
That's a lot. Okay.
How many of you have a technical background as a software engineer, for example?
That's also a lot.
I'm going to assume that's representative for the whole of Germany. Sure.
How many of you - This is going to continue like this, isn't it?
How many of you run Linux devices? Here's the twist, though.
How many of those Linux devices run on ARM platforms? Oh, come on, guys. Okay.
I expected the number of hands to go noticeably down, and they went down
with the last question a little bit, but not as much as I anticipated.
So, thanks, folks. Thank you.
So, what I was going to say is ARM devices are actually a little bit tough to get by.
I mean, sure, there's Raspberry Pis, which I'm guessing most of you own.
But, like, if you want to get a consumer laptop like this one, for example,
that runs on a Snapdragon chip, those are actually really difficult to come by
because most vendors still base their hardware on x86 like AMD and Intel CPUs.
But a few vendors are testing the waters. Lenovo is one of them.
I think Dell has released an ARM -based model.
And, of course, there's Microsoft Surface.
And I don't need to mention that Apple has, ages ago,
adopted ARM for their MacBooks and other devices as well.
So, on Linux, we kind of have an easy time dealing with these big
architecture changes because most of our software is open source.
So, we can just recompile the world whenever our hardware changes.
What we can't so easily do is recompile proprietary games that we still want to
play or things like Photoshop or office suites that we might need to use for work.
So, seven years ago, Ryan Houdek actually started out with the FEX project
because he wanted to create a solution that is able to run th is type
of software on Linux platforms even if they use these new ARM chips.
And me, personally, I joined development around four years ago.
So, it's been a while so now.
So, I feel like hopefully I deserve being on stage here today to tell you about this stuff.
If you're asking yourself the question what actually is involved in
emulating an x86 CPU, well, first let's look at the whole system that
we have because there's more to it than just the hardware underneath.
On top of the CPU, there runs an entire Linux operating system, (maybe,
thank you) a Linux kernel that does things like making sure that
your game cannot read the passwords that are saved in your browser.
It takes care of file system abstractions and it bundles all of the device drivers.
On top of the Linux kernel, we then have the various base libraries
in your system, like the C runtime, the Java JDK, the .NET runtime.
We have graphics libraries like OpenGL and Vulkan, and,
of course, we have game engines like Unity or Unreal.
And only on the very top of the stack we see the actual
game, which , free opportunity for a game developer to pick
a really, really amazing name here, so don't miss out.
If we look at what this all means for facts, we're just
gonna ignore the middle layers for the time being and
just focus on what happens as far as the CPU is concerned.
So, the problem that we have is that the game expects to run on an
x86 CPU, but we have an ARM CPU now, so that doesn't quite work out.
What we can do, though, as an emulator is we can translate
on the fly the game's binary code from x86 to ARM.
And we do that using a tool called a binary recompiler, which, well, as the
name suggests, it doesn't compile a high -level language program like C to
assembly, but rather takes existing assembly code and recompiles it to a
different architecture from x86 to ARM64. What does it look like in practice?
Well, here's some very simple toy example.
The specifics don't really matter too much.
I just picked four really simple instructions here.
And the key here is we don't directly go from x86 to ARM
because that kind of munges together three different tasks.
We need to decode the x86 instructions.
We need to pick the corresponding ARM instructions that do the same thing.
But then we also want to do this in a way that is really optimal
because we want to play, for example, games at like fluent frame rates.
So, we can't just turn one x86 instruction into tens of ARM instructions.
But rather, we want to have as close as possible to a one -to -one mapping.
And for this, to meet this end, FEX uses what's called an intermediate
representation, where each of the instructions is compiled to a
group of FEX internal operations that have a very specific meaning.
But the key point here really is that given this intermediate representation,
we've done the decoding part, and we can now focus on the optimization part.
So, we can take our internal representation and make sure, like, do
things like constant elimination, for example, to make sure that whatever
code we use to ultimately generate ARM code is as minimal as possible.
And this much easier done on an intermediate representation than on x86 directly.
And only after this step, we pick the final ARM instructions.
And you can see, using this approach, we get fairly close
to the original input assembly code as far as size goes.
If we ignore the IR for now, I picked a slightly more involved program.
Very simple loop structure here.
And we see the ARM code gets a little bit longer than before.
And I just wanted to take a moment to highlight all the
different, like, fundamental issues that we're facing here.
The first one is, if you, like, pause the video if you're watching
offline or just look at it later, you would see that most of
the stuff up here more or less directly maps to the x86 code
but the bottom part is more or less overhead we just incur
every time we enter our exit out of an emulated block.
That is just something we can't avoid.
We can make sure to keep this kind of static overhead as small
as possible, but it's always going to be there, unfortunately.
Another thing is this specific, sorry, I should highlight it here, this decrement
operation, you would think this just maps to a simple subtraction on the ARM side.
You just take a register and subtract one from it.
And indeed, we can find this instruction on the right side in form of the sub s instruction.
But looking closely at it, we also find that there's a cset and an
rmif instruction, which you wouldn't expect that, but the reason
it needs to be there is something that's called flag handling.
So, whenever you have arithmetic operations that add or subtract numbers,
there's a chance that you're over or underflowing your number from, like,
32 -bit, from the available value space in a 32 -bit or 64 -bit number.
And for this purpose, x86 sets flags on most of the instructions,
even if the program doesn't actually ever read those instructions.
But as a recompiler, we can't statically determine this usually.
So, we need to generate this corresponding code unconditionally.
And it's only thanks to IIR that we can get a somewhat feeling of
what the next instructions are actually reading as far as the flags
are concerned, and then keep this code as minimal as possible.
Without this kind of optimization, we wouldn't have three of these instructions in total.
These three. But we would have, like, 10s or 20 of them. It's much more heavy.
And we obviously don't want that. As far as the...
So, these are the fundamental issues that we're
facing, but there's more pragmatic issues as well.
The simple fact is, the x86 instruction set is huge.
And we're not talking about, like, 10s or hundreds of instructions,
but depending on how we count and how many of the different
instruction extensions, like SSE and SSE 2 and whatnot, you count in.
You can reach up to thousands of...
I've even read numbers like 6 ,000, for example. So, it's a lot.
And for every single of these instructions, we need to make
sure that our mapping to the ARM code is actually correct.
It adheres all of the corner cases that could come
up, especially in regards to things like overflows.
You could imagine how things get tricky there and maybe
not so obvious as far as the specification is concerned.
And we still need to make sure that it all runs fast because we want...
We don't want to have a slideshow.
We want to actually use the applications that we emulate.
And for that, we need a fast implementation of this whole thing.
Despite this massive task, FEX doesn't lag behind other proprietary
solutions like Rosetta for Apple hardware or Prism for Microsoft hardware.
As a matter of fact, the AVX and AVX2 instruction sets were first implemented by FEX.
And to be fair, Apple gets an honorable shout out here because they were kind
of neck -on -neck with us as far as the hardware -accelerated implementation goes.
But yeah, FEX was the first one to have a software -powered AVX and AVX2 implementation.
And I think that's really, really cool.
No talk about x86 emulation would be complete without
a mention of the ugly, ugly memory model of x86.
To illustrate what the problem is, I've prepared, again, a very simple toy example.
We have two threads in our program
and the first one is just trying to get the second thread to compute a result.
So, for that end, it's just setting some state to zero,
like the actual result, and a done indicator flag.
And then it's just sleeping until the done indicator
flag changes its value from zero to something else.
The second thread then computes the answer to the universe and
everything and stuff, which, as you might know, takes a while.
And after that, it sets the done flag to one.
The other thread will then wake up and print the result.
And now the question is, what does it print?
Naively, you might expect 42. In reality, that is exactly what happens
on x86, but it is not necessarily what happens on ARM platforms.
And why is that?
Well, there is an intuitive assumption that we make here, and
that is because the done flag is set, all of the rights to
other variables would also be visible in the other thread.
But this is not actually a guarantee on ARM platforms.
On ARM platforms, what you could see instead is that the observed
effect is that first done gets set, and then the result gets assigned.
And at this point, it obviously would be a race condition.
So, the thread one might wake up first and print the still outdated result variable.
So, it's printing zero in that case instead of 42. This is
extremely difficult to resolve from an emulation perspective
because we still need to adhere to the x86 semantics.
So, there's multiple different things we can do.
The brute force approach just makes all of the memory operations atomic.
There's a huge performance loss.
Basically, it cuts performance into half at least for most games.
That's not something we want. There's a few tricks you can use.
You can take some inspiration from from the .NET framework,
for example, which uses a thing called half barriers.
That helps a little bit, but and also there's ARM instruction set extensions like LRCPC.
Again, those help a little, but they don't fundamentally solve the problem.
We have kernel patches that reduce some of the overhead.
It doesn't solve the problem entirely.
So, at the end of the day, you need to combine multiple different
approaches and then sprinkle a few smartly chosen heuristics on top of it.
For example, one heuristic we have is specific to Unity games.
There's a really, really fun pattern that we observed in that
there is a circular buffer that is used for I don't actually
remember what, but it's used by every single Unity game out there.
And what we can do is we can just match something like relative stores with an
offset of exactly 61 bytes, which is exactly what that ring buffer is using.
And if we do this, we can disable these strong memory model semantics for everything else.
And it's a massive performance boost for those Unity games.
It's a bit of a hack, but just because it's so consistently works across
such a wide range of the library, it's really worth it in this case.
But yeah, the bottom line is there is no silver bullet here, unfortunately. Yeah.
So, this is all there is to know about the binary re compiler, at least on a very high level.
I'm obviously glossing over so many details. What about the kernel layer, though?
So, there's a couple of different things that FEX needs to take care of here.
First of all, the game is expecting to be able to launch threads.
And those threads should look like on an x86 kernel.
So, when you catch, like, seg faults , crashes, for example.
Actually, some game engines expect to be able to recover from crashes.
This is how they do garbage collection. It's completely ridiculous, but it's fast.
So, they do it. We need to be aware of this.
We can't assume that if we observe a crash, that FEX crashed. No.
We need to be aware of it.
We need to handle those crash handlers and notify the game of it so
that it can take care of it as it would expect on a native x86 system.
There is the file system layer, which the game might be trying to access x86
libraries on a system, and it wouldn't expect to read an ARM library instead.
So, we need to do some redirection for file system accesses in that case.
There are system calls that we need to wrap because the ABI doesn't quite match.
So, the way that an application prepares parameters and then
ultimately calls a system call, for example, to allocate memory
or different things, that's a different mechanism on x86 and ARM.
So, we need to wrap every single system call and decode it manually and then
invoke the corresponding ARM system call using the right mechanism correspondingly.
That's pretty straightforward for the most part.
There are a few tricky corner cases. But it's still a lot of work.
There's hundreds of these system calls.
So, you can start to see a pattern. x86 simulation isn't magic.
I mean, maybe it is. But it's also a lot of work. So, sadly.
And then the final piece that I mentioned before is the GPU driver. (Thank you.)
The GPU driver, which simulated to the system calls, there are differences
in how that is actually being accessed that we need to take care of.
So, all of that we subsume under a kernel compatibility layer that sits on
top of your ARM Linux kernel and makes the system look like an x86 system.
I'm going to skip through the, like I had prepared a couple
of detailed slides about how system call wrapping works.
If you're watching offline, you can take a look.
But I think I summed it up quite nicely. Could you please work?
Oh, this is amazing. I love it. Okay. It recovered.
So, if you recall the old diagram, there was one layer that
we got, only one layer, actually, that we ignored so far.
And that is the one with libraries.
So, the game engine, the graphics API and the base libraries, they
are not actually all that different from the game code itself.
As far as FEX is concerned, they are just one huge blob,
one huge complex system that's interacting with each other.
But it all goes through the interface of the kernel or through the x86 recompiled code.
So, for the purpose of fax, it's just one thing.
We can emulate all of it without really any special handling to that end.
And so, we can draw a nice separating line between these two layers of the diagram.
And we can see we really just have the unmodified x86 software.
We don't need to go through the entire game library and port each game individually.
But rather, we have one universal solution that makes it all
work on ARM hardware that is just pretending to be an x86 system.
And that's essentially what fax is in a nutshell. But there's more.
This is how you can implement this whole thing, and it will work.
But there's just two more little tweaks you can do to
make things run a lot smoother, even on top of that.
And that is code caching and library forwarding.
Library forwarding is one realization that we had the Vulkan
libraries are actually really, really expensive to emulate.
And we kind of already have them on the system.
So, why don't we just take what the application is doing as far as function
calls into Vulkan and just forward that into the corresponding ARM Vulkan system?
And that's exactly what we can do by hooking into any accesses to the
Vulkan library and injecting our own little Vulkan bridge in between that.
The Vulkan bridge doesn't do a lot.
It just ships basically a little bit of magic to do just what I said.
It calls, it redirects every single Vulkan library
call into the corresponding ARM Vulkan functionality.
And this has two benefits.
The first one is we don't need to go through the
costly emulation of the original Vulkan library.
So, that saves recompilation time and it saves the overhead
of the blown -up ARM assembly code that we generated.
But we're also entirely sidestepping the kernel compatibility layer because we're
now talking about native code that doesn't need to go through this layer anymore.
And the other thing that's actually quite fresh off the press is code caching.
So, with code caching, you only need to use the recompiler on the first,
ideally, you only run the recompiler during the first run of your game.
During that first run, FEX will keep track of all the blocks that have been
executed, all the locations in the code that the game was trying to run.
And on the next run of the game, FEX will pull ahead all of
the compilation processes that previously were done on the fly.
And it will collect one huge binary and save it to disk so that subsequent
runs of the game do not need to use the recompiler anymore in a lot of cases.
And the benefit of that is you get much lower and faster loading times and
you have a vastly reduced risk of microstutterings while playing the game.
The last thing you want to happen is that you're playing Fortnite and
all of a sudden you're losing like five frames of the gameplay just
because you're meeting a player that wears a skin you haven't seen before.
And that's exactly the kind of problem that code caching hopes to resolve.
All right. I hope this wasn't too heavy.
There is maybe now a time for a little bit. My slides are not cooperating.
There we go. A little bit of demo time. Do you want a demo?
Do you want to see this actually in action? Joke's on you. This is the demo.
I'm actually not sure if the breakages we've seen are due to what
I'm just going to show you in a bit or due to the slide framework.
But, yeah.
So, as you might have guessed, this presentation is
actually a web slide deck that's running in Firefox.
And hopefully I can exit full screen mode. Thank you.
And this Firefox is actually an x86 build that I downloaded off the mozilla .org website.
And I was giving it a try for this-
And I was giving it a try for this presentation.
I didn't actually know if it was going to work. It just did.
And what I like about it is, and it nicely illustrates one of the
qualities of FEX, that this is really well integrated into the system.
If I'm- Oh, I don't actually have a taskbar set up on this monitor.
So, this thing is actually showing up in your normal taskbar, along with other applications.
If I'm dragging the window handles, you can see my silly wobbly windows effect.
It's just like a native application to all intents and purposes.
It even has GPU acceleration, and I wanted to open one of these.
Let's see if I'm picking - Oh, no. Love it. Love it. Okay.
This is my time to take a drinking break. So, even that works, apparently.
And, yeah, we're getting GPU acceleration in the browser. So, amazing.
Are you ready for more? I just need to find where I had it.
This might take a while to start up, but, yeah,
you can see this is actually a ThinkPad X13s .
A really nice piece of hardware to develop on.
And it's shipping four performance cores and four efficiency cores.
All of them are ARM -based.
And hopefully, in just about any moment, we will see something pop up on the screen.
But ...
Oh, popping up on the other screen, like popping
up on the other screen and popping up again.
So, we have a Steam library, and we can run games directly
from the Steam client, as you would ideally expect.
And it's again launching on the wrong monitor. I hope I can ...
Yeah, I can drag it. Please? Well, okay. So, this is my window manager acting up.
We'll work with it. You can see the important bits. Yeah.
So, this is really just a Linux multi -monitoring thing now, not FEX.
I just really wanted to take a moment to show you that FEX is also capable of
running 3D games at fluid frame rates, as hopefully we'll see in just about a moment.
Once the loading screen passes. There we go. I have no idea what I'm doing here.
Here. Is it? It's ... Yeah, okay. It's ... The frame rate is ...
So, this screen is laggy. I'm not sure if this one is. Slightly.
So, it's supposed to be fluent.
And again, I'm blaming multi -monitoring here for this one.
But it's still playable. So, yeah. 3D games work.
There's also people who've tried, like, actual AAA games, like
Cyberpunk, Doom Eternal, Ultimate Epic Battle Simulator 2. I don't know.
I've heard about Cyberpunk. I don't personally play those games.
But apparently, they work in FEX, if you are curious.
I'm just going to go back into the presentation and finish up really quickly.
So, is FEX actually used in practice today by anything? Yes, it is.
The Asahi Linux project, who's supporting Linux to Apple
devices, was one of the earliest big adapters of FEX.
They're using it to enable Steam gaming like I
just showed you on these devices out of the box.
They have a lot of really fun extra challenges.
Sven is going to give a talk about this on the last day of Congress, I think.
So, make sure to give that a watch if you're still around or have access to the internet.
Parallels Desktop unexpectedly started using FEX
at some point as part of the x86 emulation backend.
They wrote a nice blog post about this.
The Code Weavers folks, who's the company driving forward
wine development, integrated FEX into their project as well.
And, well, you might have heard about this one.
Valve has been working on a new VR headset called the Steam Frame, and
FEX is going to play a nice role in there to enable x86 gaming, x86
VR gaming on this new device, scheduled for the release for next year.
Perhaps this is also a good point to disclose they've
actually been putting actual money into this project.
So, a couple of developers of FEX, me included, were actually paid to improve FEX.
So, none of this would have really been possible without Valve's monetary support.
So, thanks again a lot for that. The elephant in the room.
I've just been talking about Linux games so far. I'm glad nobody complained.
So, what are we actually going to do about Windows games now?
You might now not really try to run Super Kitten Adventure anymore, but
Puppy Simulator .exe, which is a .exe, so you know it's a Windows game.
The normal approach on Linux for this is to just use
Wine and its re -implementations of the Windows libraries.
But we still... Oh, and those run on top of your x86 Linux libraries.
But we still have the problem that our whole underlying
system, the kernel, the CPU, and the libraries are ARM.
But FEX is accurate enough that you can use it as a layer
in between that to build an even more complex stack.
So, your recompiler and the kernel compatibility layer are powerful
enough to run Wine, which then can run a Windows game just like that.
That is actually exactly the setup used by Aasahi Li inux.
But there is a different approach, in fact.
And how that works becomes apparent if you have two ideas.
The first one is Wine, the runtime part of Wine, is just that it's a runtime.
It doesn't really need to be x86 for a strong reason.
So, we might as well start and just move it down
the stack and recompile Wine as an ARM -executable.
I'm skipping over many steps here, but conceptually that's the idea.
And then the other trick that we're using is we're not compiling to
"normal" ARM64 code, but rather to a variant of it called ARM64 EC.
EC stands for Emulation Compatible. The idea here is that the recompiler output...
I mean, we're in full control over it, so we might as well
make it a little bit easier for x86 code to interface with it.
And what this allows us to do is to shift the output of the recompiler up several
layers in the stacks so that it basically only applies to the games code anymore.
And the underlying Windows libraries, because we already re
-implemented them anyway, we can also recompile those from x86,
not from x86, like we have the actual source code for them because
we re -implemented them, so we can just compile them to ARM64 EC.
Again, skipping over steps here.
But what that allows us to do, we have native code providing the functionality
of Windows library now, so we don't need our x86 Linux libraries anymore.
We also need to...
Like, we can't throw away the entire kernel compatibility layer.
We only need a basically small shim that almost isn't even worth mentioning anymore.
And so, with that, the diagram becomes a lot more simple.
Contrast that to the previous approach. So, the two don't really have a name.
One is just x86 Wine running in FEX.
And the other one we sometimes like to call Wine on ARM in analogy to Windows on ARM.
It's kind of the same thing.
And indeed, this whole ARM64 EC dance isn't something
we came up with, so I will not take that credit.
It's actually the original thing that Microsoft used when bringing up their Surface laptops.
And indeed, I could talk a lot about this, but this has been much
better summarized by Darek on his blog, where he goes into all of
the nasty details about how this mechanism is working on Windows.
And it's a tremendously valuable resource, so if you
want to learn more, I recommend checking that out.
The bottom line is, we get a much simpler emulation stack with this approach,
while at the same time getting better performance out of it, because
there's so many fewer components that we need to emulate in the first place.
None of this stuff here needs to run through FEX
anymore, so that's why that's working so much better.
We're still having some issues actually getting this
set up to users, but that's something we're working on.
Hopefully, we'll get there and provide a convenient solution for that soon.
But yeah, with that, it opens the question, what comes actually next?
And I'm going to gloss over this really quickly,
because I want to leave some time for questions.
Basically, we have a product launch coming up.
There's going to be so many testers, and I hope that FEX is going
to surprise everybody and how well it actually is already working.
But inevitably, there's going to be issues that come up.
So, we're going to work on improving compatibility where possible.
We're going to improve performance to make sure that
everything's working as smooth as people expect.
And yeah, that's the plan for the next months.
And with that, there's one last thing I need to mention these times and days.
We don't use AI. Not out of principle. Not out of principle.
It's just not a problem where it helps. There's just no reason to use it.
So, neither the presentation used it nor FEX itself in any of its code.
That's all I have. Thank you so much.
What an awesome project. Thank you for presenting it there.
Unfortunately, we do not have any time for questions left.
I heard that there is a lot of questions on the internet.
So, maybe can you tell us for the people here in the room where they
can find you if they have any questions after the talk and also to
the people on the internet how they can ask these questions to you.
Tony: So, for the people on the internet, thanks for watching the stream already.
The best place to reach me is probably Mastodon or Matrix.
Same moniker: @ neobrain :matrix .org.
For the people who are still around, I am probably going
to head back to the hotel right after this thing, but-
Herald: People in the room, please keep quiet a bit until we are completely done.
Thank you.
Tony: For people at Congress, I will probably be hanging around the Dolphin Emulator assembly.
But if I'm not around there, just hit me up on
Mastodon or Matrix so we can still chat in person here.