[MUSIC]
Okay, welcome to Millyways. Welcome to our next talk.
We all know how hard it is to set up some kind of service on the connected intertubes.
But it's even harder keeping it running, you know, with lots of people wanting to access your service, malicious intent or not.
And our next speaker, Craig, will give us some insights on the history of DDoS attacks and how you can handle them.
Please give him a warm applause.
[APPLAUSE]
Okay, thank you. So at first, so you know a little bit about me, I have lots of IT experience and for 20 years I'm already doing, working in IT and doing CCC events.
So the interesting thing maybe is that I'm also with the CCC NOC, the Internet Manufacture.
So the area I'm talking about is mostly in the premium managed hosting area because that's where I'm employed.
So it's a high security environment, mostly web hosting.
We host some government websites, which you surely know.
And of course, due to that, we get a fair share of DDoS.
So parts of this talk, I have to give a short disclaimer.
I will do a very short introduction to DDoS, talk about the changing landscape from 2012 to just now, talk about surviving application layer DDoS, and then we'll have a short few key takeaways.
So the disclaimer, everything I say is publicly available knowledge.
Maybe you don't know where to find it, but if you look, you can find it in Hoos, Dns, Trace, Root, Rao, BGP, Toadz from Ben Jojo.
In public biddings, there are often technical specifications which explain what they're expecting from your DDoS mitigation.
There's news articles, sometimes our own customers have been leaking some kind of information to the press.
You can find stuff on mailing lists, Github, our blogs.
There's actually been some Kleine Anfrage in the Bundestag about DDoS.
So it's funny because we're also hosting the site that has the PDFs for that, which is fun for us.
So, yeah, we also have a privacy policy on the websites which explains some cookie stuff we're doing.
Of course, you can just DDoS us and then you will find out that we have mitigation and how it works.
Of course, some x-axis values have been removed.
And some of the information I'm talking about is not directly from the company, but from, for example, Cloudflare quarterly reports or stuff.
So it's not internal secret knowledge.
And, of course, I'm talking about DDoS in general.
So what is DDoS?
Of course, there's an XKCD for it.
So it says, "Hackers briefly took down the website of the CIA yesterday."
People hear that the website has been hacked.
But what computer experts like you hear is someone tore down a poster hung up by the VCI.
So it's not hacking. It's not infiltrating the systems.
It's using a service that is offered by the website just too much so that no one else can use it anymore.
Right?
So you send too much, you overwhelm the target by sending too much network traffic, too many sessions, too many requests,
too many dynamic requests to searches, for example.
And just think of it of too many people visiting a shop and ordering food,
which we've actually done at the chaos communication event at the end of the year.
There were just too many people visiting the Dunkin' Donuts.
And then there was just no possibility for random, normal people to get the donut anymore. Right?
So some questions for you so that I get to know the audience a bit better.
Who of you has been DDoSed?
Well, quite a few.
Who was down as a result?
Okay.
Who is confident they can survive DDoS at work?
Oh, someone like maybe. Okay. Yeah.
And who has done DDoS?
I see some hands. Good luck you're not being videotaped.
So yeah, okay. I also did it, but against our own websites, of course.
So let's talk about pre-2012, the good old days, as I call it.
There wasn't many issues with DDoS.
We had probably less than one attack per year.
No proper DDoS protection.
But why didn't we have proper DDoS protection?
Well, our customer said, "Well, this is not in the public bidding budget."
So it's an Ausschreibung, a public bidding, and it's not part of it.
So you don't have to provide it.
So if you're down because of DDoS, that's all fine with the customer.
Well, okay. And they also said they don't mind if they're offline.
So my boss also back then, he said, "Yeah, we're within SLA.
DDoS is not our responsibility, of course.
Why should we build something that's not being paid for?"
And they always stop after an hour anyways.
So I still ask the mitigation provider.
So back then it was like 30 terabytes traffic per month,
or like 30,000 euros. That's less than 100 megabits.
So very expensive back then.
Then came 2012, so Anonymous gathered,
and they used Twitter to distribute pictures like this.
So they published this,
and people were supposed to go on a website to attack some sites with WebLog,
which is a pretty easy tool to use.
And of course we also followed the invitation to the channel
and to monitor what they were doing and how they were attacking us.
Back then we didn't have good protection,
so actually we were down for a few minutes due to one of these attacks.
But they said quite, well, that's the red part.
So they said, "Oh, let's try it again tomorrow with WebLog."
So thank you for telling us about the tooling,
so we could just mitigate it easily.
And unknown to us, GCHQ DDoSed Anonymous, like Anonymous Chat IRC,
with Rolling Thunder, which is their own DDoS platform they were using.
And people got searched and arrested
because they took part in these attacks from their home computers.
So yeah, people got arrested, and as a result attacks became less powerful even.
So 2013 to 2014 there was lots of innovation going on.
New trend was UDP reflection.
So Spamhouse got DDoSed for several days, CloudTra got DDoSed.
They were actually offline for some longer time.
In the US, the CISA published a report about these attacks,
and that made them even become more widely used.
So how to survive that? Well, have UDP filters from an ISP.
For us as a host, I'm hosting HTTP websites, HTTPS.
Why would I need UDP on that?
So that was really, really easy for us to filter.
So 2015 other things happened.
In 2014, the Ukraine-Donbass war,
and on the 7th January 2015, the Ukrainian prime minister visited Berlin,
and they wanted some money.
And as soon as he lands there, the foreign offices are down,
the German Bundestag, the Chancellor Merkel's website, everything is down,
and they have a prepared message that they are responsible for this
due to the Ukrainian war.
And they don't want us, don't want Germany to give any money to Ukraine.
So back then, we didn't have, because of the public bidding thing,
we only had very small connectivity.
The NOC from our ISP actually removed the UDP filters we had gotten previously.
We got black hole in Frankfurt.
We got black hole everywhere.
We don't have much spare hardware, no general layer 7 mitigation solution.
So to survive, we just used Cloudflare.
There were some online protection, origin protection issues,
but after like six to eight hours, we were online again, which was fine for us.
Our customer was also happy about this, but of course not everyone likes Cloudflare.
Well, they can look into your traffic, you have the privacy concerns.
There was some press article on Netzpolitik.org, which was taken very seriously.
And the ability to look into all layer 7 traffic was an argument
for doing something else and not using them anymore.
So we started our own very small modus mitigation.
It's not like we're not trying to be Cloudflare.
We try to do the on-prem part very well.
And if we need a partner to do something more, like big traffic,
like UBDP attacks, that's outsourced to, well not outsourced,
but they implement filters we tell them to implement.
So 2015 to 2016, there was also some more innovation going on on the DDoS front.
Layer 3, layer 4 attacks became more popular.
We saw small spoofed Synth floods, usually maybe around one gigabit,
which is the typical size of a rented server.
And of course low and slow attacks also got popular.
Like slow-posed things.
There you can see the referrer actually.
It's pretty nice of them to send the referrer, so it's easier to block.
And how to survive this?
Of course you just increase capacity.
Always make sure you can handle line rate.
And well, have an upstream that can filter for you.
And you of course need to rethink all timeouts.
If someone connects to your web server, why would they be idle for like five minutes?
That takes up valuable resources.
So 2016 to 2017.
The previous attacks continued, but there were WordPress pingback attacks,
which are also pretty easy to filter out.
Their pattern base cannot be changed.
But of course when the first attack like this hits you,
you need to implement a filter.
Previously you didn't know about this.
And there's lots of different other layers of an attack that can hit you.
So you need to know about these in order to filter them.
2017 to 2021.
Now it's getting more recent.
There were more and more mostly dynamic requests.
So for example you use a /?id and some random thing,
because then it has to hit your backend server and your cache can't deliver it.
So yeah, targeting backend resources actually.
There were also new sources like squid proxies, micro tick router botnets,
and I think one of the first botnets with more like 10,000 bots attacking us with layer 7 attacks.
So from 2021 to February 2022 there was even more layer 7 attacks,
more HTTP, more pipelining, also due to the Meres botnet using pipelining,
high requests per second attacks, and again random parameters in order to bypass caching of course.
Lots of new sources, hack GitLab servers, which are usually pretty powerful,
which is way worse than having an IoT botnet attack you,
because these servers are just so much more powerful.
Bigger residential botnets and a lot more attacks via Tor.
And also rented IP space was now being used.
Yeah, so they would just rent IP space and use it for a while just to perform layer 7 attacks.
So February 2022 to now.
Well, of course the second Ukrainian war started.
And well, what do you think? Did we get more DDoS?
Of course we did.
So there are new sources now, which is also in the CloudFlare quarterly report.
There's dozens of clouds using at the same time.
So suddenly you have like, oh, Google is attacking us, Microsoft Cloud is attacking us,
Linode and everyone is just doing layer 7 requests.
Attacks through Tor also increased.
Google published some information about that.
They had an attack that had 1.3 million requests per second over Tor.
And then this is from the source of Team Simru.
I hope I pronounced it right.
They found out that No Name, this No Name Russian actor,
that they had two interlinked hosting providers,
and these providers are only used to perform layer 7 DDoS.
So there's also now the DDoS project by No Name.
They run kind of paid DDoS.
They are promising that they will, at a later point,
they will pay for people who have performed DDoS attacks.
So they pay like a few cents per request that they have been sending.
And of course, again, this is an old method, also used by Anonymous before.
They use, well, now they use Telegram instead of Twitter,
but you see they are asking other people to help attack.
And some of these people already have botnets,
so you have a lot of different botnets suddenly attacking at some point.
And they are also now using paid expensive proxies.
So behind all of this, there's some money,
and this costs some money to implement if you rent proxies
that cost like a few thousand euros per day.
So, yeah, they actually managed to take down lots of sites.
That got a lot of media attention, and they are really a serious threat.
And Anonymous Sudan even took down Microsoft, right?
There were these issues with Teams, which some of you may have experienced,
and they had very good powerful layer 7 attacks.
And of course, current attacks, current events cause attacks.
So when ukraine-wieder-aufbauen.de got online, you saw on the other slide,
they were immediately, they were starting attacks.
Or if they, when they decided to deliver tanks to Ukraine,
of course the attacks start again.
So, but how to survive this?
We've seen that a lot more attacks are layer 7, but why is it problematic?
So, oh, something flew away. Okay.
It's a bit windy. Okay.
So why is this problematic?
Well, it's hard to distinguish from a residential botnet and proxies and cloud.
Like, is it a legitimate source?
And these sources, they're doing full HTTP requests to your site,
which can, which use up some resources.
This causes high load, and they're querying dynamic websites,
which means they try to attack your backend capacity.
So should we block on known patterns?
Is that a good idea? Or maybe requests per IP?
What about proxies, like a telco, a camera grade nut proxy?
Or should we limit bandwidth per IP? Also bad with the proxy, probably.
How do we distinguish an attack from real traffic?
Or should we just use more capacity, add more VMs,
have Kubernetes, horizontal pod autoscaler, and just spawn up a thousand instances?
What's the best way?
To understand this, I have some details.
This is the HTTP that's sent by the classic webloic attack,
which I've shown before. That was the interface, and this is what it actually sends to you.
So this is the interesting part.
The impact is you'll get a cache bypass.
They are trying to hit backend servers, and well, of course, backend server load, of course.
So what happens if you block on vr-legend?
Well, this happens. They just use this parameter.
It's basically the same, just you don't need the second parameter to do a cache bypass.
But this one, if you decided to filter on the ID thing,
well, they can just rotate it instead of K, U, Y.
They will just write something else.
Again, you have a cache bypass.
What about this one? This even looks valid. It's not valid.
Because the correct one would be 61.
But 62 will also hit the backend if you don't have any filters in your caching
and your load-belinke sync frontend.
And it even looks valid. It's harder to see.
What about this one? You use a search term.
You just send queries to the search all the time.
If you have 50,000 bots, and each bot does a request per second,
well, you're probably down immediately.
And this is, of course, valid.
Even this, this is a valid backend query.
Maybe your load-belinke sync can handle 30,000 of these requests.
But what about 300 or 3 million?
Then you will have an issue with bandwidth, possibly two,
or with memory pressure or things like that on your load-balancers.
And what's an engineering solution?
Well, just increased capacity.
How to do? One attacker with a hacked GitLab server can maybe do 30,000 requests per second.
One server, how many search queries can it answer per second? 30,000?
Well, you can have one server for each attacker, right?
So horizontal pod autoscaling is not a good option.
The only thing, the top priority is stopping abusive clients from reaching backend servers.
How are we going to do that?
So I'll quickly introduce you to a typical setup.
So, well, you have the internet, traffic flows from top to bottom.
You have the internet, your routers attached to it.
Maybe you have a mitigation appliance, a firewall load balancer application server,
and you're able, of course, to monitor everything.
So should we use a mitigation appliance?
What about TLS 1.2, TLS 1.3 Tefric?
If you terminate it on the mitigation device, it has to have this performance.
Usually mitigation devices that can handle like 10 gigabits or more of traffic per second,
or let's say 100,000 handshakes per second, they're super expensive.
They're not scalable. You can't just horizontally put 10 there,
so you can handle, let's say, a million requests or something.
They don't have a good API generally, no custom settings.
You can't build custom stuff. They're super expensive.
So, of course, not an option.
Well, let's use data we already have.
So from monitoring, we already know how many requests usually happen on which site,
how many traffic is transferred, how much requests my backend can do,
what's the usual baseline for my backend.
We all have that data.
But should we block someone who's doing more traffic?
What about the proxy user, for example, in a company,
and then there's like 10 people going to your site?
Immediately, that would be higher, right?
So the solution is you verify the clients.
You don't use an appliance. You use your load balancer, inspect the queries,
and if it's over a certain limit, you can just redirect it to service
that verifies that the client is valid.
Of course, you have to write some code.
There's some solutions available, but not very good ones,
so we have to write our own, actually.
So I hope this is not too small.
Yeah, I'll just quickly go over it.
So basically, it works like this.
If an unknown client has done too many queries, he's being redirected to this Inodia solution.
It looks if there's a valid query, well yes, then it can access the backend.
If it doesn't have a valid cookie, it's being redirected by the load balancer to the Inodia system.
It presents a task, the task has to be solved. It can be a JavaScript, button click, a capture, whatever you want to do.
And then if the solution is correct, it sends an error. If it's not correct, it sends an error.
If the solution is correct, a token is being sent back as a cookie.
And then it starts from the beginning and the client gets authenticated.
If it's trying that too often, of course it also gets blocked.
So this is the usual one. Ours is a bit ugly, so I just stole the look from CloudFair, which you probably all have seen before.
So this is the usual thing, what happens there.
They do some cryptographic operations, a proof of work in the background, and after a computer has done it, it's authenticated.
So what else can we do?
So we can use a control plane. So we have data in the load balancers about the people who are doing too many requests already.
Like there's a client doing 10,000 requests per second.
Why would we continue to serve it?
We just, from the load balancer, we push via an API to the firewall and we can drop the client on layer 3 already in the firewall.
So there's no load on the load balancer anymore.
And then also you can import blocklists for IPs, ASNs.
It can be hard or soft blocking.
So if you have very suspicious Russian AS which only sends bad traffic to you, you can soft block it and it will always go through verification.
Of course this all should be built with CI/CD, so you check if there's a net that's too big, sanity checks, all of that.
But the important thing is you can also block ASNs.
Because as we've seen, some of these layer 7 attackers, they have fixed ASNs they're using to attack.
So this is an example for abuse.ch for their botnet tracker.
So we just create a little bit of JSON, we have CI/CD pipeline, and then it just gets deployed into the firewalls.
Another thing that's a bit new maybe that we haven't really seen before is DDoS bugs.
So that's an unknown stability issue.
So your servers or your software is becoming unstable only under very high load reconnections and with certain configs which maybe wasn't tested by the software vendor.
So does this happen? Of course it does.
Otherwise I wouldn't have put it here.
So there are just two examples for HA-proxy.
So layer 7 requests weren't blocked.
So you didn't see all these requests that were being performed.
So an attacker would do this special kind of request.
You would look into your server log and there was nothing there.
Your CM, all your analyzer systems, they wouldn't see anything but this request still happened.
And there was another work that HA-proxy instances were getting crashed.
So reproducer is interesting, this is from the GitHub issue.
Around 200k current connections are doing a get slash at the same time.
What could that have been?
Well of course under DDoS this feature became unstable.
But all of these are fixed now of course.
So what else to do?
You need to do your homework.
You need to have all your data.
You need visibility, you need to know everything.
Your HTTP front end, error codes, backend requests, latencies, bandwidth, packets per second, Ttl.
From where is your traffic arriving?
What are the ASN sources?
You need to look into your systems about the load, pressure stall, disk or memory usage.
Just monitor everything so that if something happens you can have a look afterwards.
Super important.
So how to survive this?
Of course if you have all this data you do an excessive post mortem.
Even if it takes 10 people a whole week you do the post mortem because you don't want to get crashed by that again.
You analyze of course the whole attack.
You build test cases so you have a reproducer.
You give it the reproducer to the software vendor so they will fix it.
Of course you report that upstream because you want to have it fixed for everyone.
So this means we will add your technical distinctiveness to our own.
Your zero days burned if you attack us we will make sure you can't use it again.
So key takeaways.
Engineer a general solution for issues.
So layer 7 defense is essential nowadays.
Capture and use all the system data you have.
Like I said everything needs to have an API so different systems can talk to each other.
Of course do post mortems for DDoS and also test your systems.
DDoS your own systems or have someone do it for you.
There are some providers that will do that for you.
Well sometimes also for us sometimes you lose.
This is an old picture but sometimes you win.
We also monitor them and they said in the channel after they attacked us.
Why are all websites fast?
We won this time.
Thank you.
So just one more thing.
Well maybe you've already read it.
You can use DDoS-test.com to non-invasively test your website.
It looks at timeouts and things like that.
So it's a neat tool you can use to improve your own defense.
Thank you very much Craig.
So as you mentioned we've got some time for Q&A.
Are there any questions in the audience?
Okay we have a pretty lively microphone stand over here.
Please everyone who has a question line up over there.
So yeah you can ask your questions from a centralized point.
Make a nice queue.
Okay let's get started.
Can't hear you sorry.
Hello oh no it's working okay.
So mainly your talk was about HTTP.
Are other protocols more robust against or have other attack vectors or something?
And additionally what about HTTP 3?
Yeah HTTP 3 is really tricky because it's based on UDP which is annoying.
Because then you can't just easily apply all these filters
and there's still lots of reflectors out there right.
So there's different options you could do.
For example you could fail gracefully which means if you're getting attacked via UDP
you could stop serving HTTP 3 and just only serve HTTP 2 if that's what you choose.
Yeah there's some other ideas, some heuristics you can work on.
So yeah I don't have a very good solution for that too.
It's a problem many people will face.
And it will be good for Cloudflare and other providers
because if people get delost with this stuff they will move to the cloud maybe.
Hard to defend.
Next question please.
Yeah have you thought philosophically about what you're doing in terms of
you're building a system which encourages the competition to sort of compete in a Turing test
situation to differentiate its behavior from machine to a human who actually wants access
to your website so they're just going to keep getting better.
How do you deal with that problem?
Well for most attackers they also need to spend time on you right.
There are these butcher websites which is basically DDoS for hire
and they offer Cloudflare bypasses.
So but if we were using Cloudflare we would probably have been hit by that
but our solution is different.
We can't use the cloud because of the customers we have
so ours is different so people would have to invest a lot of time
to engineer a circumvention for that.
It's worth spending like a few weeks on engineering time on Cloudflare
but is it worth spending on us?
Like that's maybe an advantage but yeah of course they could try
using an interactive browser then we can enable button click
then we can do captures.
There's a lot of other tricks you can do like yeah there's lots of other client
verifications.
These are just some examples.
Yeah there's like a dozen other methods.
Hey Craig I'm wondering as an ISP now my former problem was my PEP were filled by the attackers.
Now it's going to be my server who is going to be attacked so should I go
and move my protection service to match those IP sources and so on
plus what the servers are doing and getting like with the HEParxy
because obviously we've got like two source of data who are useful for our metrics.
What's your thinking about that?
I think I didn't get that right.
I'm not sure you mean like if you're getting attacked by a legacy 7
you have attacker IPs or?
Well I'll still get my source ASN source IP sources and so on so do you match those
two informations to get a decent oh so you like like this metric for your protection?
So you mean if it would be useful to match maybe the link that traffic's coming from?
For instance and maybe also does the RPKI is good for this traffic?
Does the signing is good? Does they have ROAs and those kind of stuff?
Yeah that should be some things that you should do of course
a match on Bogons match on your own IP space why would you get that from externally
and stuff like that so yeah sure of course.
Next question please.
Hello so basically trying to compare your solutions one to three
versus using Cloudflare during crisis situations so basically you're saying
okay you don't have a cookie throw a capture but my concern with that is do you think that most of
the teams in different companies would be able to implement captures that can counter basically bots
in these attacks?
Yeah it depends on the team of course yeah if you have an engineering department that can do
something like this but you can use existing solutions already you can you could just use
Google recapture for example or H or H capture you don't have to implement and code the capture
and everything by yourself just present the page and then get the proper information the proper
verification back so that that would be possible to do also yeah.
Your turn. From the resource balancing perspective like you have your servers and you have all your
observability tooling your telemetry and so on and then I observed that some kind of request
used a high amount of observability output compared to others and in the end we were
our concussion was with the telemetry back line so how do you balance within
at one point decided just to cut off the telemetry because it was decided that it's more important
that some requests were at least answered but how do you balance this or is there a good
general solution or is this really domain specific?
You mean you had too many requests or maybe I didn't get that properly?
We got mail from mail requests and the request caused a lot of telemetry output and this telemetry
output then this traffic actually the damage internally this was the issue.
Well I guess our backbone is internal network is maybe bigger than yours then we're having
enough internal capacity to handle that well of course you see the spike and if you collect like
everything then you have several terabytes of data but I can buy a server that hosts that so
I have budget so sorry.
That's always nice to have yes and you have a question.
Yeah thank you that was a great talk.
Do you ever see issues with web sockets because I know that's becoming more of a
thing that web developers like to use because I can imagine people could then pipe data in.
Yeah web sockets are long lived and we really hate them so we don't have many customers that
are using them so and there's customers that attract DDoS traffic a lot and fortunately
the customers that are using web sockets they're not the ones attracting the DDoS so yeah so it's
it's annoying yeah.
Okay any more questions there? Yes please.
Thank you for your great talk.
REST APIs are more common now how do you protect them from DDoS without such methods as the
challenges you showed us? Yeah there's probably different methods for that maybe
maybe you have another endpoint for the first request to authenticate and then you feed that
back with an API to the other system like you have a login thing on one sub domain and if that's
verified only then you can access the other one because there's a layer four filter on it or
something like that. This very quick idea I don't know there's probably some other better methods too.
All right your question.
So if you're already redirecting these attackers to a capture service have you maybe looked into
doing any more offensive mitigation techniques like tar pits or gzip bombs
once the attacker fails the capture? With this anonymous attack back then weblog runs in the
browser. If you redirect the browser to someplace else it will go there so you can redirect it to
I don't know well it would be really evil to redirect them to your competition but
yeah you can redirect them and do stuff with them if it's browser based also if they use a
headless browser to do some stuff but yeah better to detect and block them. Tar pitting yeah it's
an option but yeah thank you. All right one more question yes please. So commercial products are
quite mature now so do you see which product commercial products harlock commercial products
do you think so well the vendors are saying that right do you see any limitations in them I mean
why would you go this route when let's say you have the budget and you can yeah take some I mean
what are the advantages on this one yeah so a solution you buy is often not scalable right so
then you have one box it can do 100 gigabits on prem or something like that and then if you want
to add another one they say oh no you can't just add a second box you have to buy this bigger box
it can do a terabit and it costs more like a house so it's not really scalable I think and also you
always need two because what happens if there's a software update of course you need twice the
capacity all the time or if you have three well you well you get the point and also I think some
of these mitigation solutions aren't really market ready because we had a solution and tested it and
we had another one in production and it had issues with it could serve captures but it can't look
into tls 1.2 traffic well yeah not very helpful anymore right and then it also would serve the
captures which with an attp 200 code which means if the google bot is coming it's indexing all these
sites and then the index is fucked so yeah and with our own solution we can just adjust it
whatever we want to do and have different levels of verifications for example or we can do
start with easy verification and it's like a maze that an attacker would have to go through like
start with a simple proof of work then some redirect javascripts and and all these and they
have to code something for that in a row and when they think they're finished they will meet they
will see the next thing all right there's one last question in the queue please so I've seen
h a proxy in your presentation so I assume you're using h a proxy as a load balancer
what would be your recommendation for for example for the firewalls or the other components
yeah well everyone likes open source so I would recommend using some some open source things that
you can debug actually for h a proxy you have the source code even if you don't buy the professional
version you have very very good support because they're really interested in getting having a
good product right so some of these fixes they're done in less than 24 hours which is really great
so I would always recommend using open source and not buy a very big box from a vendor but also
including api's as you recommended right yeah for for api's usually if you have a more complex
system it's very specific to your environment because for us it's web hosting I'll just block
udp who cares right but if you're a gaming provider you can't block udp because your games use it so
you need to design very differently so in the end it's a whole system that's being designed with
different components and of course this is not everything we have it's a lot more stuff that's
interconnected and giving and sharing information so yeah all right right behind you another
questions born you said about coding something for the proof of work protections are you talking
about like computing it on the attacker nodes or some form of bypassing it some somewhat by some
form of bypassing the protection formal form of form of okay well it could be anything if you have
a really big botnet and you have a computational proof of work your bots would have each bot would
individually have to calculate the the formula and stuff so you could code that of course yeah or
have a have a headless browser executed and then you have the result and feed that into the client
but that also requires you to code a lot of stuff so your botnet can attack and then we will just
use another mitigation for that all right thank you very much for the questions if you have any
questions later i'm pretty sure craig will stay around and you can ask him later but for now
uh let's thank him again for his great insights on ddocs thanks
[Music]