From Y2K to 2038: Uncovering Time Bombs in OT and ICS Systems with Pedro Umbelino

Aaron Crow (00:01.729) Hey y'all. Thank you for joining the podcast today. Protected all, got a guest on here, Matt at RSA. We had some fun, exciting stuff. I was in the sandbox at RSA with my OT wall and you came up and we had this great conversation. We actually played around with some of the HMIs and we'll kind of get into the details of what that is and kind of the things that you and I were talking about at RSA. But first of all, why don't you introduce yourself to us and kind of let us know who you are and what your background is. Pedro Umbelino (00:30.26) Absolutely. Well, first of all, thank you for having me here. My name is Pedro Molino. I currently work as a principal research scientist at Bidside. My background, you know, it's not just into hardware hacking, it's also software hacking and so forth. I guess I grew up with the beginnings of the internet and that challenged me a lot to figure out how things work in terms of... programming and computer virus and all those problems that when you're living in 1996 you end up having. And so you kind of accompany the start of the internet and the interconnected systems and we are getting at a very interesting place now that everything connected and everything talking to each other. But yeah, I digress. Aaron Crow (01:27.608) That's awesome. Yeah, I mean, you know, that's kind of my path is that computer you see back there, that Mac is one of my first computers, not my first computer. My first computer was a Tandy TRS 80, Trash 80, with the cassette player that you had to like fast forward to the spot and hit play to get the program to launch. That thing my dad got from the company that he worked for, they were getting rid of Macs and going, know, converting everything to PCs. So they sold all the machines they had in the company. I think they used them in the accounting department or something like that. So he bought that and brought it home. Well, it was like the commercial, all the commercial accessories. So it had like a SCSI hard drive that was about the size of a ream of paper. So was like, I don't know, almost three inches thick and pretty big with a big, the big, old school, big SCSI connector. I think it was like four megabytes or Maybe it was 10. I don't remember exactly, but it wasn't much in today's standards. And that thing sounded like a jet engine when you fired it up, how fast it spun and how loud it was on the desk. We go back to now and my laptop has two terabytes. We were just talking as we joined. I just got a new laptop. It's got two terabytes and 36 gig of memory and it's got a GPU and a CPU. It's just insane how far we've come from just when I was there. And that's not even the beginning of things, right? Pedro Umbelino (02:34.323) as well. Pedro Umbelino (02:50.836) Absolutely. And I remember the cassette. So in Portugal, we had the ZX Spectrum computers. So my first computer was actually 48 kilobytes of RAM. All it had. And then we had the cassette players so we can load the games. This was around 1988. And it came with a manual, like the programming manual. And I actually taught myself as a kid. I was eight. And I started reading. I was super happy because I already know how to read. Aaron Crow (03:00.852) Right. Aaron Crow (03:08.417) Yep. Pedro Umbelino (03:20.338) and I like to read a lot. So I started to read the programming manual and I taught myself how to program when I was programming like very simple things like, hey, what's your name? And you type in the name and hello Pedro, how are you? Things like that kind of challenge. So it wasn't actually a PC or what we call a PC. That came later, that came in 1994 for me, which was quite late. So I already had like a 486 DX 33 megahertz computer, which was an absolute powerhouse. I could run games without picking MS-DOS, like high mem and the memory settings. I don't know if you remember, have to the Outexact.bat for some games that were very memory-heavy. So you have to like unload everything so you can run those games. I didn't have to do any of them. But yeah, fun times. Aaron Crow (03:57.292) Hey. Aaron Crow (04:18.988) Yeah, I remember my, I think was freshman year of college, my, had a roommate that was a senior and he was an electrical engineer. Um, and he was doing a project. so he, as an 18 year old, you know, freshmen doesn't know anything, doesn't have any friends yet. You know, I just moved in, you know, he's like, Hey, why don't you help me with my senior project? So we, he found out that there were like three or four companies in the small town that the college was in that had, you know, kind of just discarded their old electronics, thrown them in dumpsters. So we went and got, you know, old 386 and 486 machines and we were, he brought all the motherboards and all the, all the PCs and systems. And we were de-soldering all the components off the motherboard and taking the CMOS chips and the, you know, diodes and resistors. and we, we just had a big fishing tackle box and I would put them into different places and we had them all labeled with, you know, how much resistance this one has or the ohms on this one and you know, capacitors over here and you know, all the different things. I had no idea what the heck I was doing, but it was a blast because I was just de-soldering things and clipping things off and organizing them. And I didn't know what the heck he was going to do with them. but it was, it was kind of my first kind of introduction into kind of hacking and, know, kind of figuring out, Hey, these things are dead. You know, nobody wants them to nurse value, but there is value if I maybe take those components and reuse them or put them in some other use case. so that's, know, it kind of got me. excited as a 18 year old kid, not exactly sure what I wanted my future to be. Pedro Umbelino (05:49.78) Yeah, scavenging for parts was also one of my hobbies. Even if you don't know what you're going to do with it, you might, you know, sometime need it in the future. Or even if you don't understand what it's for, I'm pretty sure this piece looks important. I'll save it for later. Then you learn how to use it. Aaron Crow (06:02.854) Exactly. Aaron Crow (06:09.353) Right. Aaron Crow (06:12.979) When I recently moved and we could dive into other stuff, but it's funny that I recently moved a couple of years ago and I went to my parents' house and was kind getting some stuff that had been left in there. And I found that tackle box with all the old components and things like that. And I finally allowed myself to throw it away after what, 30 years or something, whatever it was. Cause that was like 96 when that happened. it was a minute ago, almost 30 years ago. So it was just like, okay, I guess I could probably throw this stuff away. I'm probably not going to use it anytime soon. Pedro Umbelino (06:45.948) Yeah, I admit I have some issues with that. I have like, I'm just looking at, for example, an ultrasound machine that weighs 200 kilos that someone offered me and I cannot, it's huge. It's right over there. I'm going to do some sort of project with it, but it's sitting there for like two or three years and I cannot throw it away. I feel, I feel. Aaron Crow (07:00.434) Right. Aaron Crow (07:09.64) Right? I understand. understand. Well, why don't we, why don't we dig into, so you were at RSA. there, there's a lot going around at that time. not only were you there to talk, you, you came up to me in the, in the sandbox and we were talking about a unique issue that I'll let you kind of introduce. but you know, it, it's also during the time, that day was when we had the power outage, in the UK and where, you know, kind of Portugal and a lot of big. big part of the UK was hit with that. So a lot of things are happening. Like we were having conversations, was this a cyber attack? We didn't think so, but you know, there was just, it was down for a long time. So there was a lot that went on that day that I met you. So was very memorable. I always remember the day that I met you, sir. Pedro Umbelino (07:54.204) Yeah, it was crazy. At that time, I think I was still unable to talk to my family back in Portugal. And it was around the time that the telcos started to lose the UPSs on the cell tower started to fail. So even wireless communication or mobile communications were out and everybody was like uncertain. So what's going on? Why is it taking so long? It's the entire Portugal and Spain, the entire two countries in the blackout, it's huge. It's something massive. And we were talking about that. And then I was talking about some other side projects that they're not related with the talk that I gave at RSA regarding time issues. So I can go into whichever one you like first. Aaron Crow (08:30.739) Yeah. Yeah. Aaron Crow (08:48.593) You lead with whatever you want. So why don't we kick off with the, with the RSA discussion. We'll talk through that and then we'll get to the time thing. after that, if we have time, or if we have to, we could do a second episode and talk about the other one, depending. So let's just dig into the RSA talk. Pedro Umbelino (09:03.164) Absolutely. So as I say, I was talking, it was, I think, my third public talk about ADGs. And the title of the talk is blowing up gas stations for fun and profit. It's something that I drew from Aleph when smacking the stash for fun and profit, which is the memorable hacker paper. But the idea was, Aaron Crow (09:29.618) break. Pedro Umbelino (09:32.98) I was set up to blow up a gas station by exploiting ADG vulnerability. So that's my initial idea. When I started looking into that protocol, so a bit of background, we at Bitsight, we scan the internet kind of every day. We try to understand what's exposed and what's not. That's part of what we do. do more stuff, of course, but we see a lot of exposed ICSOT devices that shouldn't be Right? And you try to understand what, which protocol do they speak, how they are connected. So we started building our own lab. So we don't want to touch these devices in a random way. We want to be very, very careful when we probe them, when we scan the device, we don't want them to reboot. We don't want them to touch them if we don't have to. But still we need to understand what's exposed and what's out there. So we started building our own lab. And one of the ICSs that we have in the lab is ATG devices, because that protocol was quite prevalent. We could see a lot of ATG devices exposed, or thousands of them. At that time, like a couple of years ago, give or take, I wasn't aware of the ATG protocol. And I started digging in. And I figured out that in 2015, HD Moore and Jack Shadowitz, they actually published a paper exposing this kind of vulnerabilities. Like the protocol itself is vulnerable by default. It's one of those old protocols that look and feel like a serial protocol that was given the ability to speak or have an IP address and speak over the internet. So we all know how that ends, right? There's no authentication. There's no... security at all. The idea is to make sure it just works. So I identify some of those devices that were exposed. I started looking at the other issues that a protocol might have. Then I started looking at new devices. OK, so this was back in 2015. So what new ADG brands and models are out there? Are they also vulnerable? Are they not? So they were also vulnerable to this kind of Pedro Umbelino (11:57.384) remote manipulation, but they had more technology built in. So they have a web server, they have the ability to synchronize with an NTP server and send emails. And so the attack surface on those devices is actually bigger. you're vulnerable because you have the old protocol enabled and you're vulnerable because you have a bigger attack surface. Well, potentially vulnerable anyway. And then I started, okay. Aaron Crow (12:13.329) Bigger. Pedro Umbelino (12:26.184) Let's try to understand if these models have issues. So in the past, I was a pen tester too, so it's quite natural for me to start firing up burp and other tools and try to... And if there's basic vulnerabilities about... the web server or in the port that I had open. And there was a bunch of So last year, I reported... I published... 10 different zero days, there are more coming that are still being triaged. But the issue here is, okay, now you're an admin on this ICS system. What can you do with it, right? So it's an ATG. An ATG stands for automatic tank gauges. They are used to monitor Aaron Crow (13:12.966) great. Pedro Umbelino (13:22.194) big fuel tanks, they monitor the level of the fuel tanks, they monitor the temperature of the fuel tanks. They also interact with the environment. They are able to turn on emergency pumps or shut down valves. They can actually connect to the fuel dispensers. It varies depending on how the integrated installs the system on site. It really depends on the brand and model. But there's some basic features that those devices have. It's the tank level monitoring and it's a relay board that's used to control outside peripherals. So my goal was, so how can we use abusing an ATG make a gas station blow up? So you can either go to the sensors or the actual actuators, the relay board. Aaron Crow (14:13.978) Right. Pedro Umbelino (14:20.916) With the sensors, you can fool the ATG if you want to report fuel levels that are not real, for example. Or you can reconfigure the tank and tell the ATG, hey, you're controlling a tank that holds 20,000 gallons of fuel. But the tank... Yeah. Aaron Crow (14:40.326) When it's a 10,000 tank gallon. Yeah. Yeah. Right. So you're changing the volume. You can change it or say, Hey, it's saying it's half full when it's really all the way full, or there's a lot of different variables that you can attack and come at it. Um, because that open protocol, just assumes whoever's speaking to me, I'm going to do whatever, whatever you told me is, is truth. I don't have to authenticate. You spoke the right language. So I'm going to, I'm going to assume that what you told me is correct. Pedro Umbelino (15:06.034) Exactly, it's just like that. When I talk to technicians that go on site and actually install the systems, this is a scenario that they are actually very concerned because this increases the probability of an accident, of a spillage accident. When the refueling technician comes and starts refueling, the time that he has to react to an over spill, Aaron Crow (15:24.805) All right. Pedro Umbelino (15:35.834) is lesser. If you think about it, if you spill diesel, it's one thing, if you spill gasoline or plain fuel, the risk for it. And there were, like this is a scenario that this kind of accidents happened in the past, not by using ATGs, but the spillage, I mean. So they were really concerned about this being, you know, someone doing the set scale. That's one thing. But In my view, okay, so I can increase the odds of an accident happening. What about the relay boards? Can I do something with them? So I was testing the relay boards and I actually extracted the relays from inside the ATG because I didn't want to, I didn't know what was going to happen. I didn't want to damage the ATG. Turns out that if you turn a relay on and off really fast under load, it becomes a light bulb. It will eventually pop in smoke, right? which I did, right? So if you just start switching the relays on and off really fast, you can either damage the relays or most likely damage the things that are connected to the relays or controlled by the like a pump. Pumps are not very happy to be turned off really fast. In fact, if you go back to the Aurora test generator, Aaron Crow (16:49.228) Right. Pedro Umbelino (17:00.584) when that big generator blow up because it was out of phase, you can try and get to that state with a pump. It's not easy, but you can actually achieve physical damage. And that was my point. Are we going to start seeing gas stations blowing up left and right? No, but you can really cause physical damage and it's not very hard to do. And that's... Aaron Crow (17:07.022) right. Pedro Umbelino (17:28.498) That's one of my concerns. That's one of the reasons I've been trying to raise awareness into this topic and try to understand if something's changing or not because something has to change. Aaron Crow (17:42.711) Yeah, and I think that's the point is a lot of times what you see is the fear, uncertainty, doubt, know, especially from salespeople, they'll say, you have this thing or, you know, I've been you know, pen tester, red teaming or assessment type thing. And, you know, you can it's easy to point out all the things that are bad. Right. But it's more important to understand, OK, here's the bad things. What can I possibly do with those things and what are the mitigations and how how at risk? Am I with those things right? Those are the big pieces to look at. It's not just about is it possible? It's like how difficult is it to do this? What other things that can you do? You know what? What are the scenarios that would have to go into this? But you know when you talk about clicking on and off relays like yes, that's their job, but it's also the reason why you don't want your kid just sitting there flipping the light switch back on and off. Eventually it's going to cause damage, right? It's not designed for that, right? Yes, you know it's going to click on and off overtime. but eventually the mechanical things inside that switch are going to break beyond the fact that if you switch it on and off, it's gonna get hot as it's engaging and disengaging. And obviously that heat, especially next to something like jet fuel or something like that, that would be a really bad thing if that thing puffed and ignited and got hot enough that it could spark and to your point, know, blow up a tank. Obviously that would not be fun or a good thing from a lot of different perspectives. That's one of the things with OT that we, it's a fine line of communicating the damage and the potential risks that we're talking about while not trying to oversell the fear, because it hasn't happened yet. We're not saying that this is huge, but we also need to talk about it and be honest as this is a possibility and we need to make sure that this doesn't happen. And yes, it's very difficult and yes, it's not. super simple to get it to that space, but it is possible. And we need to understand that the risk is not zero. The risk is not anywhere near zero. It's not a hundred, but it's also not zero. And we need to really be clear on what those risks are at any of these types of things. And doing research like what you're doing is super important to truly understand the possible and also like how easy it is. And then, you know, what are the things Aaron Crow (20:06.212) to your point, just because you got a new controller and it has all the web stuff and all the other things, does not necessarily make it better or more secure. In fact, many times it's adding more vulnerabilities and attack vectors and things that I can do to access it without having to speak that protocol. If I can get to a web browser and then do something from within the device, that's a lot easier and having more surface area to attack than just that one insecure protocol that we know is insecure. And it's going to continue to be insecure. And that's okay, but when you involve these additional, you know, of opportunities, it's like having Telnet on something. Just like, for goodness sake, turn that off. Pedro Umbelino (20:47.284) And you're quite right, it's a challenge to communicate these things without... So in a way you want to highlight the potential damage of the things. That's why the talk is called blowing up gas stations for fun and profits, not by accident. I want to communicate that risk. And the fact that I'm trying and I'm aiming in the research for worst possible scenario, I do believe... As a security researcher, this is the most responsible thing I can do. I have to think like an attacker and I have to explore the worst possible scenarios and try to figure out if they exist and how can I cause them. Otherwise, how can we defend against this type of attacks? So it's not just because it's fun. Yes, it's fun, at least for me, but that's not the point. The point is you have to explore this kind of scenarios. And in terms of communication, so yes, In the talk, I have the video with the relay popping up smoke and it's very impactful for someone that's not from, from, doesn't speak security. it's very important for them to see and impactful when they see that thing like popping up smoke. It doesn't echo that that makes the gas station explode automatically. That's not the idea, but it gives people something to keep on the back of their heads. Okay, this is physical damage. So this is possible by using this protocol. It's not very hard. And the way you communicate that, especially for someone that's outside of the security, infosecurity world, makes all the difference. Especially if you start thinking about, know, policymakers, reporters, and folks that can actually help you drive and make the change that we need. You have to be able to communicate with those. Aaron Crow (22:42.005) Yeah. Well, and that's the piece, you know, I say this all the time on my podcast, but one of my mentors a long time ago told me all business is a people business. Like you have to be able to sell people on what you're doing and why it's important and why they should care. mean, ultimately, you're a salesman. You're selling the people that you're delivering, whether it's a talk at RSA. or it's a client that you're showcasing this capability to or whatever, you're selling them on why it's important, why they should care, and what the risk is. And they're factoring that against all the things that they've been told and, know, well, it's really hard or all the things, right? And you're trying to convince them. And this is where it can get teetered over. you know, I saw a post on LinkedIn this morning about, you know, selling fear in the OT space and how that's not a good strategy. I don't disagree with that. all of my sales shouldn't be you have to do something or a nation state attacker is going to blow up your thing, right? But that has to be a part of it as well, right? We know that's not the most common risk factor. You know, more than likely misconfiguration on these devices is way more likely than, you know, an attacker doing these things. But that doesn't mean it can't happen. And it doesn't mean I should ignore it or pretend that that likelihood is zero. Because it's not zero and the likelihood of those things happening, especially like you said, if you can see it on the Internet, it's a very high risk of your environment. Right. And obviously those things shouldn't be on the Internet. They should be on their own network and there should be protections and mitigations around those things. But this is the thing that most people don't get is they don't even necessarily realize that their device is directly on the Internet. Hopefully they didn't do that by design or they didn't really think through it, I guess. At this point in 2025, the fact that any OT devices live on on the direct internet is just terrifying and frustrating because it's just unacceptable. Pedro Umbelino (24:42.356) It is an acceptable, it's a trend that unfortunately seems to be growing slowly but growing and we have to deal with it. And I agree selling fear is not what we should do but at some point we shouldn't be, it shouldn't be needed for us to wait for something tragic to happen to make the change that we need. And I think, and I hope we don't get there, right? Because if you think about it, the level of the entry level for exploring these types of vulnerabilities and systems is getting lower and lower. My theory on why we don't have more types of attacks on ICS systems, quite frankly, is just because the track deckers couldn't figure out a good business model for this to make them money. It's easier to hack some server and encrypt the information, have a ransomware event and get money out of it than messing around with a PLC somewhere in a factory. But if you figure out a good business model around ICS attacks, I'm pretty sure they will just happen. and they will escalate really, really fast because everything that we see, not everything, but a big part of what you see exposed is vulnerable. Aaron Crow (26:19.218) Yep. Well, and there's also this thing to consider is you know, we have examples of insider threats, have misconfigurations, there's all these things that put it out there. And sometimes we're gonna get things that happen because there's somebody that's sitting around on a Friday night figuring out what's going on and they just wanna play and see what they can do. And, you know, especially to your point, you know, AI and the ability to, you know, put things in there and figure things out quicker and use resources like AI and things like that, that makes it more accessible to people that maybe don't have an OT background or don't really have any understanding of PLCs. The learning curve is so much faster to get in there and figure things out than it was even five, 10 years ago. Used to in this OT space, there was a lot of security by obscurity. Hey, this thing is so proprietary. Nobody knows what it is. Nobody's going to mess with it. That was my security model. Obviously that doesn't work, especially now because you can buy things on eBay and there's open source virtual versions of PLCs and most of these protocols and all this stuff is readily available to the good guys and the not so good guys to use in figuring things out and, All it takes is somebody like you with your skill set that puts on the wrong hat or somebody entices to do something. And that's really the problem is, you know, maybe it's a, it's a nation state. Maybe it's a disgruntled employee. Maybe it's a, you know, a competitor or whatever the case, all it takes is somebody that with a little bit of knowledge and somebody that misconfigured something or thinks that their risk is zero or it's not important enough to do anything with. And then bad things can happen to your point. Maybe they don't blow up the thing. Aaron Crow (28:19.73) But maybe they take it down. Maybe they damage a pump. Maybe there's things that take them out of service. They're losing revenue. They're having to send, you know, send roll trucks to go fix things. Like there's maintenance responses to that, you know, and, and that just overall, the ROI on this beyond just the risk of blowing up a fuel tank, it's also the downtime and, you know, the risk to human life and mechanical pumps and all of those things go into this factor of, you know, espionage, all sorts of things that you can think of that can be a risk for these things. Pedro Umbelino (28:51.39) Yeah, absolutely. You don't have to blow up an ETG to have a real, real impact. Like, think about this for a second. The colonial pipeline didn't actually affect, like the ransomware was on some computers and they shut down the pipeline just to make sure nothing happened, right? It was not an ICS. I don't consider it an ICS, like a level attack, okay? What happened was, so, and everything was fine in terms of gas stations, Aaron Crow (29:07.123) Right. Aaron Crow (29:10.696) Correct. Aaron Crow (29:14.248) Agreed. Yep. Pedro Umbelino (29:21.266) There was enough fuel for two or three days. So everything was supposed to be, everything was going to be fine. Right. But people panic and start rushing to the gas stations and start buying all the gas. And pretty soon in a matter of hours, a lot of gas stations run out. You know, have emergency services that don't have enough gas for the ambulances and so forth. this was just one, one example of nothing blew up. Aaron Crow (29:35.262) All Pedro Umbelino (29:50.81) And there was a major impact, right? If you think about ADGs, there are thousands of them in the US. If you put them offline, let's say 10%, what do you think will happen? Do you think, like, history shows people will behave pretty much the same way? They will panic. They will rush to buy toilet paper or whatever. And things will... Sorry, am I on? I saw... Okay. Aaron Crow (30:21.16) Yeah, you're on. Yeah, it said reconnected for a second. worries. Pedro Umbelino (30:24.5) You have to re-edit. So think about this for a second, right? People will rush to gas stations, they will buy all the fuel, they will buy all the toilet paper too, and there will be thousands of dollars of direct damage just by sales, but that's not going to be the major issue. The major issue is, you know, emergency services that are going to be heavily affected. Aaron Crow (30:26.76) That's okay. Aaron Crow (30:38.014) Mm-hmm. Pedro Umbelino (30:53.46) There's going to be like logistics affected. It's going to be millions and millions of dollars if this happens. And then after that, for this type of systems, the technician has to go to the location and recertify that this device is operational because it's used also to monitor for fuel leaks for environmental reasons, right? So a technician has to go there and certify. How many technicians are in the US? Aaron Crow (31:02.173) Absolutely. Pedro Umbelino (31:23.176) Can they rush to all of them at the same time? No. the recovery, the overall scenario doesn't look good at all. And you don't have to a single gas station for this to be super impactful. So yeah. Aaron Crow (31:26.151) No. Aaron Crow (31:39.906) And that's the thing that we don't always think about. And when I say we, I just mean the collective overall world, not necessarily you and I and people in this space, but that's a hard thing to think about because to your point, you don't have to blow up a gas station to cause really big impacts. And that thing dominos into other things. So to your point, if Colonial Pipeline or Pipeline shuts down, I'm not able to send gas or fuel or whatever the thing that I'm sending. And then gas stations, then I have a rush, then I have a panic. And then, you know, not only is the company impacted in their bottom line because they can't sell, but maybe the price of gas goes up significantly. And then, you know, that that's going to have a domino effect. And then, you know, all of these things impact people from, know, from a lot of these different factors. And to your point, and it's very similar to what we saw in like the the crowd strike issue when that crowd strike issue happened. It wasn't that it was really hard to fix, right? It wasn't unrecoverable. The biggest problem is this most of them, like, you know, at the airports, they had all of these computers that were all around and somebody had to put hands on those things to reboot it, get it into a right spot. And then it came back up. But it wasn't that it was really difficult. It was that they didn't have enough bodies to go out there and touch them all in a short amount of time. So they were having to bring in third party resources to help with this stuff. But to your point, like I talk about this all the time. If we had a, know, most people's incident response plan is I'm going to outsource that to vendor A, B or C. And there's only so many of that type of skilled person in the United States, let's say. Let's say there's some number. But if we had a really big incident that happened, whether it's gas stations or critical, any kind of critical infrastructure. And it happened across the country or maybe even across the world. Again, looking at the power outage that just happened that we were just talking about to kick this thing off. Let's pretend that that was a cyber incident and we had to respond and we had to do disaster recovery, incident response, that kind of thing. There's not enough incident response people in the world to respond if it's a big enough thing, because, you know, they're going to go to their highest paying customers, whatever. You know, maybe it's company A. Well, then company B is Aaron Crow (33:57.392) They don't have a resource because they're using the same vendor that company A is and that person is already at company A's spot. So if you're Delta and American Airlines calls you, well, the same resource can't do both things. So then they're having to bring in third parties. And that's where this thing gets exponentially bigger. And the cost of business, again, as we saw with CrowdStrike, it wasn't the cost of repairs that was the biggest thing. It was the downtime and response time that so many companies lost so much money. during that time because of this thing going down and resource limitations and expertise limitations and disaster recovery obviously didn't work too great for them and all these things just compounded on each other. Pedro Umbelino (34:38.856) Yeah. And then there's the stuff that you can't even think about in advance, like the power outage, for example. In Portugal, it was some hours. It was not a full day. In Portugal, someone died directly because their mechanical breathing device stopped and the battery didn't last as what it should have last, I guess. But then... Other indirect damage happen like people like think someone someone's apartment catch fire because they are using candles to lit up. There's traffic accidents because the the the the stop warning. So there's a lot of things that are hard to manage. But here we are appearing to be selling you know for when that's not the time. Just exploring that sometimes this these things have a much Aaron Crow (35:21.019) streetlights. Pedro Umbelino (35:34.822) much wider impact that you can immediately foresee. Aaron Crow (35:40.61) Yeah, and that's where, know, I've been part of in the States. have Idaho National Labs has a cyber informed engineering program where they really are leading the effort. It's been around a long time, I think, but I really like the branding that they're doing with it. I'm a huge advocate for it. I've been through their training, but it's really around considering when I'm designing a system and I don't mean a cyber security system. I mean, when I'm designing that fuel system and that tank system to consider. the things that you're doing right now, right? That, okay, if this relay, if I switch it on and off a hundred times, can it blow up? The probability may be low, but it's not zero. So that needs to be factored into how I design that system so that, again, so I'm not just putting it on the internet because I understand what those risks are and what the likelihood and the impact and all that type of stuff. So cyber informed really engineering is really just putting the cyber. lens from a risk perspective in when I'm designing this system, what is the possible things, bad things that can happen? And again, that doesn't always mean that it's a, that it's a nation state attacker. can be a misconfiguration and how can that domino down into my overall business process and really understanding that like that's something that we need to be thinking about. It's easy to do when you're doing Greenfield new design. It's harder to do with an existing system that was designed 40 years ago. How do I interject these things that we didn't think about? I upgraded the new, you know, ATG and now it has a web server. The guy that ordered that thing didn't think about that. He just got the newest one and it has these things in it. They didn't know about it. They don't know what risks it is and why it's a problem. So it comes from the factory. It's got the web server. that makes my job easier. Like they don't understand. They're not doing it maliciously, but they don't understand. This thing was on the internet. Now I just added a web port protocol on it. And now it's even more vulnerable than it already was that I didn't even understand the risk of. Pedro Umbelino (37:34.888) Yeah, absolutely. And I keep thinking on all the actors that need to play together so we can actually drive some change. So on one side you have the manufacturers providing all these abilities and sometimes shipping stuff with insecure by default configurations or supporting legacy protocols. On the other hand, you have the integrators or even the end users trying to make their lives simpler. by being able to remote administrate something that should be online or integrated. They just deployed stuff with insecure configurations because they change it to be insecure because it makes their life easier to your point. So all these players have to kind of come together, know, manufacturers, integrators, the end users, the policymakers. And what I do think right now, I'm also... kind of focus is the ISPs. I do think the internet service providers, they are not on a mission to monitor the internet, sure. They can have a role in fixing or at least being helpful when we are talking about critical infrastructure. That's my goal. They are in a unique position. And they have a unique perspective of what's out there and who owns it. So they have a unique view on the problem of exposure. So I'm not saying it's their responsibility to fix exposure, of course not, but they can be helpful, they can be a bit more proactive instead of the contrary, which sometimes... Aaron Crow (39:24.974) Yeah, 100%. Pedro Umbelino (39:26.866) Sometimes you find a bunch of different devices that are exposed. You want to contact the owner because just to send a note, right? Saying, hey, this shouldn't be online. Like you're risking something here. Like I found a chlorinator on some water treatment plant somewhere. And by the way, there's a bunch of them out there exposed. And I cannot even reach the owner of the asset because I don't know who they are. The ISPs know. Can they? provide a way to fast-track this kind of context somehow. I'm not saying they should give us at least their customers. That's not going to happen. It's not good also. But I think somewhere there's a role that they can play, especially when you're talking about critical infrastructure. Aaron Crow (40:20.889) Yeah, mean, the point here, I think what I 100 % agree with and we have to have all parties engaged, right? All of the things and all of the steps along the way. And if we're going across an ISP, which most of these are obviously, you know, and there's exposure, like we need to involve all of these folks and share, right? So, you know, if you're using AT &T or Verizon or whomever your ISP is, like we should be tagging, hey, these are, this is a critical site. Like the, if you see this type of traffic other than normal, you know, XYZ, then, then those things should be alerted on. Right. And, but the bigger problem I even see is most of these organizations don't even realize the assets that they have because they don't even know that they're putting them in on the internet or that they even exist because some engineer is putting them on trying to do his job, his or her job. And they're not, again, they're not malicious, but they just don't know the risks. So they just put it on because it's easy. It gets an IP address and they had to download an update and get the new firmware or whatever the thing is. they, it's easier just put it on the internet, right? And they don't really understand. So I think some of it is awareness from all the parties. And that's where the CIE comes in with the, know, cyber informed engineering. but also to your point, like we should be monitoring this stuff at all these different levels and saying, Hey, there's some things that look. This looks hanky. This looks weird. This shouldn't be like this. mean, the fact that you can find it, I can find it. I you can go to SHODAN and you can find OT stuff on the internet as long as you want to play with it, right? From water to power utility to oil and gas manufacturing, know, kiosks, you name it. It's all out there. if we can find it, obviously bad folks can find it too. Or even just gray folks that... you know, we're playing around and not necessarily intending to break anything, but oops, something broke and I don't even know what that was or where it was, right? Pedro Umbelino (42:18.322) Yeah, completely agree. You might be just scanning and not trying to do harm and just with an IP that has that protocol and you're trying to interact and it stopped responding. Maybe it rebooted. You don't even know what you're touching and something that was critical somewhere in some factory or in some system, right, rebooted and there's a huge problem and you're not even aware of it because you're just... playing around with an IP port pair and sending them invites. Aaron Crow (42:53.972) It's terrifying when I go into these places and it's not that there's old stuff and that's not my concern or necessarily that there's insecure protocols. It's more so that they don't realize it. They don't realize what they have. They're not protecting against those things. I'm not gonna walk in and say, you gotta replace all of your old devices with new devices. You have to patch every week. You have to have all secure protocols only. That's unrealistic to go in and. replace all OT devices to newer things. And again, as we've already said, sometimes newer isn't necessarily better, but you definitely need to understand, the colonial pipeline example is a great one, right? Is it wasn't an OT, technically an OT attack, but it impacted OT. And I get people so mad at me about it. They're like, well, it wasn't an OT attack. Okay, so what? Because at the end of the day, it impacted actual things in OT. Yes, they took it down. It wasn't a bad actor doing it and ransom wearing it. I agree, but it doesn't matter at the end of the day. All that matters is was were physical things impacted. Yes. So the, the attack, although it wasn't on an OT system, it impacted OT and that's something we should care about like that. We shouldn't just care. Was it an OT attack? Did I directly go to that relay and make it blow up? No, I went to a site. some ancillary system, but that caused somebody to turn it off. It's the same thing. I was doing an assessment in a power plant and we were, we were literally, we'd done a pre job brief that morning. We're sitting in the break room eating donuts and we'd already told the team, Hey, we're going to be doing this and blah, blah. But none of us had left the room. Like we haven't plugged anything in. Our computers are still in our bags. We're literally drinking coffee and eating donuts and the plant trip. So what you think the first thing that happened? They came in to us. What did you do? You broke it. What did you do? And we're like, whoa, time out. We didn't do anything. But for about 30 minutes until they figured out what the problem was, we were their primary expectation of we did something, right? So we had to prove that we didn't break something. And really the only way we could do that, because we couldn't prove a negative, is they had to figure out what the actual problem was, but they basically wouldn't let us leave the break room until they figured it out. Pedro Umbelino (44:50.484) was the don't. Yeah. Aaron Crow (45:17.236) And I don't necessarily blame them, but that's the point is you don't realize the impact that you have and just the overwhelm, the fear, you know, they only care about that thing being safe, reliable, available, you know, providing electricity. When I turn my light switch on, I want it to work. And me doing things in that space, have to be really, anytime I'm an OT, I have to be really careful with the things that I'm doing and how we even just communicate what we're trying to do, right? Pedro Umbelino (45:45.234) Yeah, absolutely. That is true. Aaron Crow (45:50.538) Yeah. So real quick, and I know we don't have a huge amount of time to go into it, but I'd love to kind of get a high level of the time issue that we talked about and even test it out in the lab at RSA. So why don't you give us a brief overview of what that is and why it's the next Y2K-ish type issue that we both remember. Pedro Umbelino (46:10.772) I don't think. Yeah, I only mention White Cookie because it's easier to explain in... At least in folks that have grey hair, like I do. Aaron Crow (46:18.293) Yeah, exactly. Aaron Crow (46:24.53) And me, mine's right there, it's there. Pedro Umbelino (46:27.604) So on January 19th, 2038, around 3.14 in the morning, the Unix 32-bit timestamp that's used in many systems will overflow. So there's no more space for the timestamp to be saved and it will roll over, which means the systems that are using the the 32 bit timestamp, when they ask the operating system, which time is it, they will be teleported back to 1901. And it's a bit different than Y2K, which was more a way to interpret time from what the zero zero meant. Aaron Crow (47:18.838) Two digit versus four, right? Pedro Umbelino (47:22.376) This means that the actual 32-bit integer will roll over and be a negative number, depending on how it's interpreted. And the problem is, it affects a huge amount of devices. And when we start thinking about this problem... The first time that I gave it some serious thought was I was at Hackloo, it's a security conference in Luxembourg, last year. And Trey Darley was doing a lightning talk on it. And what really triggers my mind around this, I understand the issue, but he said, okay, Y2K was 25 years ago and now we have 13 years. that something clicked like, okay, already 25 years ago, right? And we only have 13. So time goes by a lot faster than we think. And... Okay, so I started wondering, what are we doing about this? And I'm sure like this isn't an issue on newer systems, right? I'm pretty sure. But then I was wrong because when I came back home, I had some systems, was, you know, playing with some ICS systems. And in this case, I changed the day and I immediately got locked out of that system. I couldn't log in. couldn't remotely control the device. because the date change broke the authentication mechanism. So the authentication mechanism that's used in this device had a token that was always expired. was 1901. It was always expired. So this was just one example. And I started messing around with other stuff. So I had this smartwatch. It's a new one. And I changed the date. And it entered a boot loop. And I was like, OK, so this might be a Aaron Crow (48:56.563) Right. Pedro Umbelino (49:19.656) bigger problem than anticipated. And we start thinking about which type of devices could be affected. Think about it. 32-bit controllers, where are they? Probably controlling some part of our infrastructure, buried somewhere, controlling some traffic lag or some pump or emergency valve or whatever. That's dependent on time. So... Aaron Crow (49:33.531) Yep. Pedro Umbelino (49:47.784) There's millions, I was going to say billions, I think I'm safe if I'm saying billions, let's say millions of devices buried in our critical infrastructure that are designed to last for a lot of years because that's the goal. And they will fail to properly understand time in 2038. And that might mean nothing happens. That might mean a boot loop. That might mean... that might mean you cannot administrate those devices. It might mean that the data that they produce, like if they're recording flows over time. So the data seems to be valid data. So the amount of different scenarios here is quite concerning. So we started this project called the Apocalypse Project. what we are trying to do first is, well, Aaron Crow (50:34.356) Yep. Pedro Umbelino (50:45.864) Try to understand how widespread is this issue because it's not going to be like Y2K. Like it's going to be like at least two orders of magnitude greater. And it's not going to be like, okay, let's throw money at the last minute problem that you can fix because you can't. You talk about before, there's not enough hands and there's not enough supply of given systems that we won't be able to last minute start to unburied all the things that are going to. Right. Aaron Crow (51:05.428) Mm-hmm. Aaron Crow (51:14.164) Correct. Yep. Pedro Umbelino (51:15.422) So understand what's out there that can fail and try to also understand classes of failures. I spoke about some of them. If it fails, how hard will it fail? So we have to test some systems at scale. We have to characterize how they fail and hopefully try to devise if there's some mitigations that you can go on some of those systems that you can apply. This is also an issue because for a skillful attacker, sometimes you don't have to wait 13 years. You just have to manage your time. actually, there's a vulnerability that's being triaged right now. In some protocols, you can control time. And if you can do that, you can trigger this vulnerability remotely even. So I would call it more a vulnerability than a bug. Y2K bug was a bug. Aaron Crow (51:53.83) Right. Right. Aaron Crow (52:09.542) Yep. Pedro Umbelino (52:15.508) I like to see this as a vulnerability, first of all, because it provides us a different framework. If you think about something that fails, and so that the CEI gets compromised, the availability gets compromised. So you can think about this as a denial of service vulnerability in the case of the authentication lock off, for example. So you can actually apply some some of the frameworks that we use to study vulnerabilities, and then you can prioritize and choose the most important that you have to fix first, right? For example. So I think there's value in looking at this problem as a vulnerability problem than just a bug. Some cases will be just plain bugs, but there are real vulnerabilities that were already found, right? So it's a huge, huge problem. When I talk to some folks, Strangely enough, the younger folks get this problem quite fast and they understand it's an issue and they want to help. Maybe because they are a bit more naive, I guess. Older folks, if they are over 60, the answers I got was not my problem. It's not a malicious answer. It's like I understand the issue. It's so... overwhelmingly difficult to deal with because there are so many potential systems that can be impacted by that. I don't have the bandwidth right now and I'll be retired by then. So it's your problem to fix, not mine. And we keep thinking there's somewhere in some dark room, there's like government people working on this problem nonstop. We dig and dig and we talk about this at several levels with from the UN, European Parliament folks, folks at different companies, I'm not going to name here. But they understand that there's an issue and they understand there has to be some kind of joint approach to this. And our part in the Apocalypse Project, Pedro Umbelino (54:38.078) to start with, let's start raising awareness, characterize stuff, figure out where those things are and how can they break. Because you have to know the problem first before you can fix it, right? You have to figure out where it lives. Yeah, in a nutshell, that's... Aaron Crow (54:56.497) Yeah, I mean, it's a huge undertaking if you think about how many, know, because what we're really talking about here is that 32-bit, you know, time source library, how many devices, some of them even to your point, your newer watch, some new devices may still be using it. Even if it's a 64-bit operating system, doesn't mean that all the libraries it pulls are 64-bit, right? So... that can be in all sorts of things that we don't think about. Like we think it's okay. Like you and I tested a couple of Windows machines at the RSA and they were fine. So they were used in the 64-bit and they didn't experience it. But obviously some PLCs, some other devices will be not using that, right? Even if they have 30 or 64-bit architecture in their design does not mean that all of the libraries are 64-bit. We've seen that across Windows for years where Yes, it's a 64 bit operating system, but we still have 32 bit components within that and the libraries that are used, whether it's drivers or firmware, et cetera. So those are the bigger picture. again, if you think in OT, how many of those devices are out there, it's unknown, obviously. So working with the vendors themselves, the source of the devices, but also like the S-BOM would be a great. you know, resource some SBOM vendors that are out there looking at those libraries and saying, Hey, this firmware and this Rockwell PLC or this Siemens S7 device or whatever, these are the libraries that it's using that library is vulnerable to this, this thing. how would it respond? At least where to start? Like, so there it's a big effort to your point. And 38 sounds like a long time from now, but it's not. When you think about how big of the effort it is to fix it, it's, it's not that much time. Pedro Umbelino (56:40.916) It's really not Pedro Umbelino (56:45.78) It's not, and who's just talking about like OT, ICS and mostly things that are very impactful if you think about critical infrastructure. But if you think about IoT, like then it blows, it blows over, right? Like think about your, you know, security camera back home or whatever you have, your toaster. Aaron Crow (57:00.474) Yeah. Aaron Crow (57:08.516) This camera I'm using right now, the Sony, you know, digital camera. I would not be surprised if it was vulnerable to it and it just stopped working or, you know, my TV that I'm looking at the monitor and, know, any number of things that have that in there. And they didn't think about it. It's the same thing, you know, very similar to your point. I like the analogy of the, of the Y2K thing is it was just the way that everybody did it. They used a two digit, you know, year. And, you know, for, for a hundred years, it wouldn't have been a problem until we hit. the next one and then we rolled over to 1900 and they're like, oh crap, that has a domino effect. Like it didn't necessarily shut down the system, but it definitely could cause problems. Right. And they kind of, maybe they overwhelmed or overreacted. I don't think they did. I think we just did a lot of work. But to your point, I remember working, you know, being in the data center on new year's Eve, 1999, I wasn't partying. I was in a data center sitting there watching to see if everything was going to start smoking. Like we thought we'd fixed everything, but we just weren't confident that we did. So Me and a few others, we had a SWAT team of folks that were just kind of standing there going, is everything going to work? Like we're watching our clocks and, and, and, know, even after midnight, it was like waiting, you know, in case somebody was out of sync or it took time. know, ultimately we didn't have any issues, but again, we did months of work overnights and outages and, and maintenance windows and updates and patching and vulnerable, you know, firmware and. and all the things to get to a place where we thought we were okay, but we still didn't know. Pedro Umbelino (58:36.05) Yeah. And when did you start? Right. Last minute. So. Aaron Crow (58:39.917) Yes. Yeah. A few months before it, right? We kept thinking, you know, not a big deal. And then it just came in and yeah, was, it was, you know, the last few months leading up to it that we actually came up with, you know, this is what we're doing. And part of that was because we were waiting to hear from our vendors and our, you know, the, solution, you know, the owners, the bars, et cetera, because we didn't know, know exactly what the fix was. And so we waited till that, you know, we had fixes to do some of the stuff that we had to wait till the last minute for. Yeah. And it was, it was hectic for sure. Pedro Umbelino (59:10.022) Absolutely. And that's why I was mentioning like that approach won't work for sure in this one. And by the way, we're not the first ones to raise this kind of issue before other folks in the past talked about this. It kind of comes up and goes down this topic because I think most folks that raised awareness or tried it in the past got kind of the same answer like, that's too far ahead. We will worry when we get there. And, you know, folks didn't take this issue very seriously. I hope this time is different. But, you know, we know that for sure we at least raising awareness is a start to get things moving, in my opinion, I guess. Aaron Crow (59:58.691) Well, so all that we talked about today always always end with this question. So I ask everybody this. It's in the next five to 10 years. What's one thing you see come up over the horizon that may be concerning? Which maybe just what we just talked about and then maybe something that's exciting that you see in that in this space. Pedro Umbelino (01:00:18.708) everything is moving so fast that making any kind of prediction seems really foolish, right? I'm sure I'll be wrong. In the next five years, maybe very soon, one of the things that I have some concerns is... So the war, the Ukraine-Russian war is going to end eventually. Aaron Crow (01:00:25.188) Yeah. Pedro Umbelino (01:00:44.302) And if you remember back then before the war, there were some ransomware groups that were very active and there was a lot going on. And when the war started, some groups break off, some very famous groups. And those folks that were highly skilled, some of them eventually got recruited to both sides of the war and they are... actively working against each other, attacking ICS systems and getting new skills in attacking ICS systems, trying to take critical infrastructure offline, because they are part of some cyber army on both sides. When the war is over, I'm concerned what those folks with new knowledge and very, very sharp skills now are going Are they going to return to their normal day-to-day lives as before the war? Meaning, are they going to return to a criminal life or will they change their previous stance? I don't know and that's really concerning because we will see hundreds of folks that are highly skilled at attacking ICS systems and OT systems in the wild. What will happen? I'm actually concerned about this. That's in next five years. In ten years from now, I don't know, the AI overlords are ready to go for it. If you don't tell Chet GPT thank you at the end, you're probably going to be eliminated in the first round. I don't feel... Aaron Crow (01:02:25.013) Right. Pedro Umbelino (01:02:30.132) I don't feel we are going to be out of jobs soon in terms of, or fortunately, I don't know. I keep seeing more unsecure systems being deployed and rather reachable from the internet and things like that. So I don't feel we are getting to a point where we're feeling this inflection in the curve that are we doing a good job or are we... not doing a good job because, you know, vulnerabilities keep stacking up. We have this challenge, like our job, if we are super good at our job, we are out of job, right? So we fix all the vulnerabilities so we are no longer kind of necessary. If we get there, you know, for society at least, it will be a good thing. I don't think... Aaron Crow (01:03:03.502) All right. Aaron Crow (01:03:24.205) Yeah, well, and yeah, I mean, you know, that's another struggle that I think we have in OT that we don't necessarily have in IT. In IT, if you look at the all the possible vulnerabilities over the last, let's say, I don't know, 15 years, 10 or 15 years, and you think about that like in IT, they kind of roll off the end and they stop being a risk because we don't have Windows XP machines. We don't have DOS. We don't have Sun Microsystems really running in these spaces most of the time in most organizations, right? But in OT we do. So we're getting all of these new ones that are coming out every day. And as I bring that new, you know, PLC online that you just talked about or the ATG, et cetera, and it's got now it's got a web browser and you know, it's got Bluetooth and it's got all these other things. we're adding those vulnerabilities, but we're not getting rid of the old ones. The old insecure protocol is still there. The old Telnet is still there. Like all this old stuff is still around. So we're just expanding the scope and adding more vulnerabilities, but we're not really removing them by attrition, I guess is what I'm saying. Like obviously from a cyber perspective, we're disabling things, we're mitigating things, but it's a different problem set than what we have in IT. In IT again, if I bring a Windows XP machine and try to plug it into an app, a fairly decent IT environment, it's not even gonna give me an IP address. It's probably gonna say, yeah, you're not putting that on my network, right? It's gonna isolate it, it's gonna kick it off, I'm gonna get a notification, it's gonna block a port. It's gonna do any number of things. I can't do that in OT, many times. And those devices exist and they're critical. I'm not, the vulnerabilities just keep stacking on top of it. And I'm not patching, I'm not doing all these things. A lot of these systems don't have patches available. And these risks are just, the vulnerabilities are just stacking up and will continue to stack up. Cause I don't see a time where we're just going to rip and replace all of these old devices across all of our OT environment. mean, look at the time thing you just talked about. Like, are we really going to think that the answer is we're going to go replace everything with brand new controllers that has 64 bit everything? No. So what are we going to do about it? Right? So these are the things and the difficulties and the struggles that we have in OT. Aaron Crow (01:05:39.114) And it's why I agree with you. I don't think we've got any risk of not being in a job. It's just a matter of how do we continue to fight these fights as people get tired of it or hearing about it and, you know, the fear, the FUD, all that type of stuff. So it's, it's a, I enjoy it. I'm passionate about it. As you know, I mean, you and I sat there and talked for, I don't know, however long we talked at RSA and I wanted to have this conversation to get it out there. So more and more people are able to hear these things. and, know, obviously folks that came to your talk at RSA, but, know, I want to expand that and that's why I do this, right. Pedro Umbelino (01:06:11.036) Yeah, and it's something that's very needed nowadays. To fight the good fight and raise the world. Aaron Crow (01:06:18.486) That's right. So how can how can folks find out about you? Your next talk? Like what's the call to action? The organization around the time thing? All that kind of stuff kind of shared out. Will also put it in the show notes, but just kind of you know, kind of talk about that a bit. Pedro Umbelino (01:06:32.192) If you want to reach out directly to me, can do it using LinkedIn, for example. I'm there too. My alias is Cryptor. You can find me on other social networks. If you are interested in the time thing, you can search for the Apocalypse project. Not Apocalypse, but HEPOC-olipse from Epoch, Unix Epoch. Maybe we can find an easier name too, but we didn't invent it, but it's called like that. the apocalypse. can join in. There's like we have minimal infrastructure so far, but there's a wiki and there's a mailing list and we are also looking for volunteers to manage the infrastructure. We all have day jobs too and that's a problem. So we do it on part time. But yeah, we are going in our numbers in terms of subscribers and folks that are interested, folks from, you know, manufacturers that are interested and they want to learn more. I already had conversations with manufacturers that they were like, okay, this is interesting. Now what? Now, the guys realized that maybe we have to change the hardware, maybe we have to design a new board because, you know, we're not going to port back to whatever platform we have now. So, yeah. Aaron Crow (01:08:03.787) That's awesome. We'll put all that stuff in the show notes. So definitely reach out, learn more. If you run an OT or your company has OT, this is something we need to start looking at now. Cause again, 2038 is not that far along. And to your point, somebody could change the time, which is what you and I did at RSA is I could change it today to 2039 or the day after. I could do that today and potentially cause an impact. So. There's GPS time attacks, there's NTP attacks, there's all sorts of vulnerabilities that exist that are not gonna have to wait 13 years to get there, right? So it's something we need to start thinking about now. So I appreciate you coming on, talking about it, sharing more, happy to have and do a deeper dive on it and dive in. But definitely folks, these are types of things that we need volunteers, we need people to dig into. if this is up your alley or something you're interested in, definitely reach out. to them and, you know, get involved, start helping, advising, you know, being part of it, sharing the knowledge, et cetera. So thank you for your time, sir. I appreciate it. Glad you have electricity today and that everything is well. And I thank you again for your time and coming on. I really appreciate it, Pedro Umbelino (01:09:14.708) Thank you for having Aaron Crow (01:09:16.862) Yeah.