I think we can begin . Hello everyone. Hi people. I'm Luca. This is Matteo.
So today we're gonna show you a little talk, too loud?
A little talk on how to sync state lights with live music, right?
Now this is a super beginner friendly tutorial so please lower your expectations, right?
If you already did this before, this is gonna probably be too easy for you, right?
Especially because I'm a beginner myself.
This is the second time I'm doing stage lights, right?
So I kind of have no clue what I'm doing.
However, what I want to show you is to demystify stage lights, right?
It looks super hard if you look at the light controller console.
There are a million buttons. There are so many configurations.
It looks very hard, right?
Instead, today I want to show you that you can get
pretty far with just a laptop and an Ethernet cable.
Nothing else, right?
We're not gonna do any of the super fancy effects, absolutely.
We're gonna stick to the basics, right?
Basically the scenario is you have half an hour,
right, of time before your DJ friend is gonna play.
What can you pull off in this half an hour with very basic effects, right?
So I'm gonna do this live starting from a completely blank project, right?
As you see here, there's nothing and I'm gonna
call this from scratch and I will make mistakes.
A hundred percent. I will have bugs. Stuff is gonna break, right?
So please be patient, right? Help me debug if you can, right?
Hopefully it's gonna work, right?
Now this software you see here, it's TouchD esigner.
It's a pretty crappy software but it's free work, right?
I'm gonna try to avoid it as much as possible and
go as much as possible to actual Python, right?
It's gonna take us a couple minutes of boilerplate, of setup, right?
Now this is a visual programming tool, right?
That means that you have those boxes, you know, right?
And you link them to each other like this, right?
What we're gonna do is to start, the talk is gonna work like this.
We're gonna spend 20 minutes, more or less, figuring out how to turn
on lights, like how do we get these kind of bright lights, right?
Then we're gonna figure out how do we do bit detection in the simplest way possible, right?
It's not gonna be very, very good bit detection. It's gonna be good enough.
Something you pull off in five minutes, right?
With the tools we have available, right?
And then the last half an hour, Matteo will actually play.
He actually knows what he's doing, unlike me.
And we're gonna see how it looks, those effects with actual someone playing, right?
So stage lights, all of them follow this protocol called DMX, right?
TouchD esigner supports this and there is this, you see here, this software
is very complex, there's a million buttons, you don't need any of them.
You need like five of them, right?
So in this case we need this DMX out, which outputs this
DMX protocol, in which I can speak to the lights, right?
Of course, it's in a visual programming, you link like those, right?
The only configuration you need to do here, that you need to
remember, is that you need to speak this Artnet interface.
Artnet is a fancy name to say DMX over internet.
So my laptop can speak DMX, right?
You're also going to need to choose one of the universe, right?
Now, if you look above, there are many, many lights here, right?
Each light might be set up as a different universe. How do you know?
You need to be friends with the people who built the club, who set it up.
And they should send you something like this, they sent me, which
is, if you can see, it's an exact layout of the club, and there's a
PDF with all the addresses of which universe is which light, right?
So assume this is something you know, right?
However, for the sake of this experiment, we're going to start with universe four, right?
Because those yellow lights over here, those are universe four, right?
Now, as I said, DMX is the simplest thing possible.
You just need to send 512 bytes every frame, right?
And each light will interpret those bytes differently.
But these ones at the top are pretty easy.
They just interpret it as brightness, right?
So what we're going to do is that we're going to try to
send the value 255, right, on all 512 bytes, like this.
And now we just need to tell that this is something custom you will need to figure
out because you need to speak to the correct switch driving the lights, right?
Now the target, you see that they all turned on, right?
If I send less, right, 128 here, you can guess that
what's going to happen, they get half as bright, right?
What happens if I send only 100 bytes, for example?
Only a few get turned on, right? This is the basic.
As you can see, it's extremely simple, right?
What we're going to do now is to simulate bit detection.
Let's assume we already have it, right? We have this bit whatever node, right?
That just goes from zero to one every four seconds, right?
Let's add some interactivity to those slides, right?
So we feed this here, but it's only one channel.
So here we need to have a math node thing in which we do the multiplication.
I really hate visual programming. So go here, here. Oh, wow, right?
Now, yes, very nice, okay? Now, this is terrible.
We're going to throw away all of this and we're going to try to stick to Python now, right?
Because thankfully, thank God, the TouchD esigner people have a script node.
And look what's here inside. Oh, Python. Ah, yes, right?
Now here, you have an editor here. You can write here. This is atrocious.
I really, I really do not recommend to use this.
Instead, just press this little cross or control E or command D, whatever,
and it's open it in your favorite editor, which hopefully it's Vim, right?
Now, let me set syntax, right? Let me increase the font. Can anyone see?
Too little? More or less? Okay. All of this is trash.
We just need the on cook thing because every frame, I know this cooked, right?
Every time I save this file, it's going to update here, right?
So this is pretty comfortable to code here, right?
Now, what we're going to do is we're going to feed whatever our output is to the DMX, right?
And let's keep the simulated bit that we give as input, right?
Now, the only bulletproof thing you need to do is
that we need to create 512 bytes in TouchD esign.
Those are channels, right? So let me just do that real quick.
Append chan like this, right?
And then to write there, you just do scriptOp .chan, the name.
So let's do another for loop over here, right? And then you can set the values.
It always needs to be an array. Not exactly sure why in TouchD esigner.
But you have it like this, right? We have already our first bug, perfectly.
Ah, because yeah, the indentation is cursed. Okay, perfect.
So we have Python writing here, right?
And we have much more interactivity like this.
Okay, it's a little bit faded, right? Now, let's read the bit.
You can do that by looking at the inputs, right? This is a matrix.
I have no idea why. You just grab the first element each time, right?
And then I can put the bit here.
Now, the bit is from 0 to 1, so I need to multiply by this.
Oh, but finally, right? Finally, we can code, right?
Now, before we look at the other light, the other lights are going to be more
complex, because based on which byte I send, different stuff will happen, right?
And usually, you cannot really just send 255 everywhere, because for example,
one byte is the tilt, one byte is the pan, one byte is going to be the dim.
There is a lot of stuff, and sometimes you need
to make sure which channels are on and off, right?
So let's try to do a very simple effect on those lights here.
So what I would like to do is that instead of dimming like this all at
the same time, I would like a line that extends based on my bit, right?
Now, I will spare you the details of how those are mapped, right?
But there are basically, if you count, there are 14 lights, 14 LEDs on each
individual of those lamps, and there are 12 lamps on two columns, right?
So we write over here, number of lights is 12, and LEDs per light is 14, right?
So now our for loop is going to loop over each individual LED,
and we're going to do another one on the number of lights, right?
And here, I'm going to calculate how much of the bit it
is left, so I know how many lights to turn on, right?
So I calculate the maximum here times my bit, and I
subtract how many lights it already considered, like this.
All right, and here the light level is going to
be 255 times remainder minus i, I think, right?
Now, the channel here, I need to pay attention on how this
is, and here you will see stuff will go wrong, because the
way lights are mapped is almost never like you would like to.
Let's see, for example, so here I have this plus I, if I
do something like this, yeah, okay, somewhat there, right?
This is going to happen all the time.
Anything you do is going to look not really correct, but wrong.
So we need to, we need to flip the order, I would like the
line to start from here, because you know the DJ is here.
So here we're going to go backwards, right? Perfect.
Now, I think I need to multiply this by two. Okay, we have one column.
I will not tell you why we need to do this, because I have,
thankfully I did a dry run before, but one of the columns is
mounted on the opposite sense, so we actually need to do like this.
Well, kinda, you know. Good enough. I don't care. We have half an hour.
Good enough, right? More or less, right?
So, now that we see the slides, before doing bit detection, let's try the, those
circular lights you see over the head, those, they're called the spikies, right?
They have a lot of features, there's a lot of different things, right?
Now, normally, normally, to know which byte does
what, you need to look up the data sheet online.
So, those are Rob Robin spikes, you will Google the Rob
Robin spiky DMX sheet, and see what channel does what, right?
Now, I will save you this. I already did my homework.
I have boilerplate .py, right? Perfect. So, this is what each channel does.
As you can see, there is pan, tilt, pan control, tilt control.
Some of those, I have no idea what they do. I tried. I couldn't figure it out.
That's okay, right? I will show you. Most of them, we don't care.
So, what we're gonna do is, they live on a different universe.
So, we're gonna copy all of this, right? Yes, with the same bit detection.
That's okay, right? Now, the universe we want to speak is universe zero, I think.
Ah, something weird is going on. Okay, let's stop, right?
Now, you can see you're not supposed to send random value to lights, right?
Let's reset this light. Unfortunately, lights have memory.
They remember what you're sending them. So, let me reset by sending zeros, please.
That's okay. Let's pretend this was intended, right?
Yeah, we're gonna have this, right? Okay. So, let's edit our script now.
Sorry, edit this one. Yeah, so, we keep all of this, the bit stuff.
We just grab the spi key from the boiler plate, right?
Now, for example, now I send everything zero. Here. Thank you.
What I'm gonna do now is, what if I send 255 on the first pan value, right?
I don't know which one is the first lamp. Is there a lamp that move?
Oh, this is so annoying that the store, right? Okay.
Well, I mean, I know for sure that there are many of them.
So, I'm just gonna do a loop over, I don't know, 20 maybe, right?
Each one of those has 21 channels. So, we just add them here. Ah, there was a bug.
Yeah. In range, sorry. Okay. You see, we controlled only the first row.
That's because the first row is on the universe zero, right?
If we change universe, I should be able to change the second row. Yeah, third.
Oh, too much. And the third universe is the last row. Okay, perfect.
Now, I wanna control all of them at the same time.
I don't care about distinguishing the columns.
You can do that at home when you have more time.
So, I'm just gonna duplicate these and say this is universe
two, this is universe one, and this is universe zero, right?
Okay, perfect. Let's try to turn them on, right?
Okay, to turn those on, actually, you need to turn on a bunch of channels.
This is the dimmer, the master dimmer. But we also need to give them some color.
So, let's say red and white. And I think that's it. Not really.
We might need to turn on more. Ah, yeah, the shutter, of course.
The shutter also needs to be on. Maybe. Power could be, I didn't need it before.
It might be color mix, actually. Ah, yeah, they're on now. Okay, perfect.
So, I want them to point downwards. That's the tilt.
You need to give 128 to point downwards. All right, pretty cool. Are they all on?
Okay, perfect. So, here we have a lot of more stuff we can do.
I'm not gonna show you everything, but, for example,
prism wheel, whatever this is, does this, right?
Okay, cool. Flower effect. Zoom is also nice, right?
And you can have all of these going with the beat, right?
And you can do, it's very easy to do some weird stuff, like,
for example, you reduce the tilt, for example, like this.
And then pan control and tilt control, I think they go automatically, right?
So, if I put here, that's what's gonna happen. Zero. Yeah, stop. Okay.
Okay, so you saw those, right?
Now, last one I wanted to show you is what's behind us here, right?
Let's do that real quick now that we saw how it works, right?
So, copy and paste again. This is instead universe 5, I think.
And let's grab from the boilerplate.
Those are called the cluster B2 whatever, whatever. Right? We put those here.
No, sorry, I copied the wrong thing. Yeah. So, those are eight, I think, right?
So, and each one of those has 41 channels. Actually, there is a lot going on here.
I forgot the equal sign, forgive me.
You're supposed to do this beforehand, before the performance, right?
Not in front of the people. Okay. But, sure.
So, here we're gonna need to turn on a lot of stuff.
So, let's do 255 times the beat. The strobe also, I want 255, sure.
And we want red only. Red and white, maybe, right? Sounds good.
And there is also a lot of stuff here. I hope that this is just good.
No, we need to turn on more.
So, ah, yeah, because also each one of those has two
lights, you will see as soon as I manage to get them on.
White, white, where did you see? Ah, yeah, dimmer, of course.
Yeah, dimmer here and also dimmer here, I think. Okay, perfect. Right?
So, those are called the blinders. In fact, they are pretty, pretty strong, right?
Okay, cool. Now, we have all of these.
Now, we would like to make it go to the beat of the music, right?
Now, what's gonna happen is that Matteo is gonna put some very
light beat and we're gonna try to make the lights sync to it.
Now, um, don't, yeah, yeah, yeah, please go start.
Don't expect anything fancy, right?
We're gonna do exactly what touch designer has us.
So, first of all, there is an audio device in thing, right?
And I have a microphone here plugged in, right?
And, ah, yeah, this is the audio track, right? Playing right now.
Next, what we're gonna do is actually the whole reason why we're
using T ouchD esigner is that there are some pre -made components.
In this case, audio analysis. It's pretty cool.
You're gonna see a lot of people using this, right?
So, it's this block over here, right? I can feed the audio, right?
And, ah, here on the right, this box here, I have a lot of
options I can turn on, on detecting kick, snare, whatever, right?
And, ah, now, let's output this into a null.
A null node means only I'm visualizing, right?
And you can see it's trying to detect, ah, a lot of stuff.
Now, the secret to beat, to syncing beats, right?
You probably already know this, but you only need to look at the lows.
Virtually, every electronic music will have the beat on the lows, right?
So, we only care about this, right?
Let's do, ah, a select on only the low channel, right? Okay.
What you need to play around here a little bit is this low, right?
Low threshold, which is when lows are detected, right?
So, you see that I'm decreasing it, and now I get lows here.
Let's visualize it a little bit better.
There is this trail component that lets you see
much better when the lows are being detected.
You see, it's not the greatest to detect the beat. It's still pretty good, right?
If I sync the lights to those, it will already be nice.
However, let's explore a different, a few different other options to do that.
Now, there is not the correct way. There are many, many options.
Like, for example, um, sorry, some people do add a lag component,
which basically just takes the average of the last 0 .2 seconds.
If I feed this to the trail, you can see that the signal is cleaner, right?
It's not greater, but it's cleaner, right?
Other people instead take this, take the low, and they do
the slope, which is they calculate the derivative, right?
Now, this gives you an even different signal. It can be useful, right?
It's up to you to decide which one is better.
There is no, like, surefire way to do it, right?
And in fact, often, you will see people that when a music
changes, you might need to adjust the threshold, right?
Now, what I recommend in my experience is that the
yellow is the best one to do lights and visuals, right?
What we get out of the lag component, right?
I mean, usually you would spend a lot of time to
see, oh, 0 .2, maybe here, 0 .5, but I don't care.
I just want to show you a proof of concept here, right?
So, uh, we grab this in another box, right?
And what we're gonna do is we're gonna remove this and feed this to our scripts.
Where's the other one? Here.
Okay, you see, uh, the lows are kind of getting detected.
They are too small, though, right?
So, there is this gain parameter we can increase to just add on the effect, right?
Or you can play with the volume you feed to your laptop, right?
Now, if you pay a lot of attention, you will see
that, uh, there is kind of some offset, right?
TouchDesigner is not the fastest software at all, 0.
There is a lot of analysis going on in my laptop and
in the lights, and, uh, I'm not synced perfectly.
Probably, I'm even off by one beatt.
The secret is that, uh, the human brain doesn't care.
It sees patterns all the time, and it's gonna think
this is synced to the music, even though it's not.
There is actually a big delay, right? You don't care about this, right?
Maybe it's off by one.
If you stare at it long enough, it's gonna look perfect, even though it's not, right?
Now, um, the last thing, um, I want to do here is to fix those spikes,
so also the, um, their brightness, uh, is synced to the beat, right?
Let's do that real quick. In this case, the dimmer here, right? Perfect.
Now, you see, those are the basics. We only have three kind of lights going on.
Um, before we actually switch to, um, real music, the
last thing I wanted to show you is to add color, right?
So, for example, here are the blinders.
I would like to make them change color to the beat of the music, right?
So, how do we do this, right?
First of all, uh, we have this, uh, nice, uh, Python package called, uh, c olorsys.
If you know it, uh, yeah, it's a bit cursed.
So, what it's gonna give us is the, uh, HSV to RGB function, right?
Uh, in which we need to provide a hue saturation value, right?
And gives us an RGB, right? Now, in our case, hue saturation value.
Uh, let's set them all as one, right?
And we feed this into the red, green, and blue, right?
Now, if I change the hue, in theory, they didn't really change color.
Uh, am I editing the color one? Maybe it's the color mix. Um, not really.
Maybe it's a virtual color. It's changing color, but not when I ask him to, right?
That's annoying. I don't think it's changing color, right?
Maybe the issue is the... Uh -oh. Oh, what am I missing? Sorry?
I didn't use the beat for the color. It doesn't matter.
It should always be, uh, it should always be,
like, even if I leave it here, like this, right?
Um, no, I just wanted to turn on in any kind of color
possible without using the virtual color thing, right?
I have to turn off the white. Let's try. Not really. Color mix, uh... Oh, yeah.
Who suggested that? Thank you. Yeah. Thank you very much. Okay.
So now if I change this, does it change color? Maybe. Ah, yeah, yeah.
Because I need to multiply here, right? Yeah. Oh, finally. Perfect. Okay.
Let's also add the white. White is usually always on, right? Yeah. Actually, no.
Maybe not. Right? Okay. Now I want to change it with the rhythm of the music.
So this hue value needs to change with the beat. Like, I could put beat here.
It's not gonna look very well, right? Changes color too fast.
I would like it to change color only on the beat, right?
So another trick we have in touch designer here is
that I go into, um, whatever output of the lag, right?
Here. And I have this speed thing, right?
Uh, this is just a sum of all the values in, uh, for now.
You see, it's always growing, right?
So now in my script, I can grab the speed one block I just made, right?
And the channel is low, right? I think. Kinda, right?
You probably need to multiply by some constant, right?
To make it really change color on the beat.
But you see, now it's kind of working, right? With this speed thing.
Now, um, if you want to do this stuff properly, you would do effects
on the mids, on the high, the snare is pretty good, the kick,
but you need to play around with, uh, uh, the detection of those.
I'm not gonna do that, right?
Instead, um, what we're gonna do now that we have, uh, more
or less 20, 30 minutes left, um, I'm gonna, uh, shut up,
let Matteo play some real music and see how the effect goes.
And in the meantime, I will also draw some, uh, uh, use
Touch Designer to draw some visualizations that are gonna
use the same beat detection to be reactive to the music.
And you're gonna see it on screen, uh, live.
So, um, that's more or less it, right? Thank you very much for your attention.
We're gonna play a little bit. You're welcome to stay. You're welcome to live.