Episode Transcript
[00:00:00] Speaker A: Yeah, you definitely gotta have different look on it. And I've seen a lot of tools get sold or clients. I'll talk to CISOs, I'll talk to that. Hey, we've had this XOT tool for three years, five years, seven years. We haven't got one actionable alert. We haven't changed anything because of this visibility. So that's where, you know, we come in and really try to help make sure they're getting visibility. Most of the time it's just never got fully deployed typically.
[00:00:25] Speaker B: You're listening to Protect it all, where Aaron Crowe expands the conversation beyond just OT delving into the interconnected worlds of IT and OT cybersecurity.
Get ready for essential strategies and insights.
[00:00:41] Speaker A: Here's your host, Aaron Crow.
[00:00:43] Speaker B: Hey, thank you all for joining me on another episode of Protected all podcast. I'm super excited today. Patrick is joining me. If you have been an aot, you've been around the industry, you probably know this man and just all the things that he does, Guidepoint does, etc. Etc. So thank you very much, Patrick, for joining me today, taking the time and why don't you introduce yourself for anybody that doesn't know who you are and a little bit about your history and how you got here and all the fun we have in ot.
[00:01:06] Speaker A: Yeah, for sure. Yeah. Appreciate you having me, Aaron. Yeah, I definitely enjoy your podcast and see a lot of other familiar faces on it as well. So, yeah, glad to be here. So, name is Patrick Gillespie, if you can't tell by the accent. I live in Arkansas. I've been here about 15, 16 years. I've been at GuidePoint for the last three and a half years. So I am the OT Practice Director at GuidePoint. If you're not familiar with GuidePoint, we're essentially a VAR, which is we resell cybersecurity only products. And my team of OT engineers are hyper focused on securing OT environments and IoT IoT devices are everywhere too. So we definitely are heavily focused on OT and helping our clients, who are typically CISOs, understand what they can even do in OT, you know, even though they don't own the assets or they don't own the remediation, you know, trying to get them to start working with their counterparts in engineering, plant managers, controls, engineers, you know, those kind of things. So yeah, I kind of, I guess go back to where While I'm in OT, I didn't come to GuidePoint to do OT. I actually probably exact opposite, I was came on to manage the threat and attack simulation team for Offensive Security, essentially doing penetration testing and red teaming for very secure networks, you know, for large companies, you know, those kind of things. But GuidePoint has a lot of clients that have OT and it's now starting to get attention, especially since COVID all the remote access needs and things. So it got bounced around, I guess. About three years ago, guidepoint somebody asked, hey, does anybody know industrial Control systems? And told my boss it came around a couple times. Nobody responded. And I said, well, I'm not trying to volunteer for anything, but I worked, right? I worked in manufacturing for 11 years. I don't mind meeting with them to see what they even wanted, you know, I said, because I was like, I had. I've been out of it for a while. But back when I was left OT and what we called OT now in 2016, I built freight trains for eight years. So I left the freight train rail manufacturing industry in 2016 and got into office and security. But I do remember there's cleartext protocols, no authentication, Internet lines all over the place. So trying to figure out, hey, what do y' all like, what are you wanting to accomplish in this environment? So just started meeting with them. We just kept getting asked more to where we just started building out OT services under that TAS team. So again, kept growing and growing. Two summers ago, the GuidePoint partners asked me if I would consider building out a nationwide practice for operational technology. And since then we've been doing that. You know, I don't do any real work anymore, of course. All my engineers do all that. I say I'm just a pretty face for ot. I'm either on Zoom or on the road, you know, enjoyed hanging out with you in Vegas at Black Hat and seeing a lot of the community there. But, yeah, so now we do the whole gamut. Everything that Godpoint's done for 15 years in IT, we're now have built that in OT, essentially. But we also know that there are IT devices in OT, whether that's an HMI or historian. You got switches and routers and firewalls. So essentially we've built joint services with all the FireWall experts of GuidePoint, the network architecture, the identity team, the PEN testing team. So we do a lot of ot. PEN testing, OT is your response, those kind of things. So, you know, trying to help our clients, you know, get started in OT security and kind of mature that program. Right. And then also since we've started this practice, Guide Point is joined the OT Cyber Coalition in dc. So I represent guidepoint there. We're actually. It's pretty Cool group. I think there's like 15 members. It's a nonprofit that essentially we advise Congress, Capitol Hill, the House, the senators, White House, all that or critical securing critical infrastructure. You know, since my background was manufacturing, I actually started out as a CNC programmer building elevators a long time ago, around the time I was in the Army National Guard as an intelligence analyst. I got into manufacturing that way and then ended up, like I said, doing. Working for a company from the 1800s. You know, trains have been built, I think, since around 1812 was like the first locomotive somewhere around there. So a little bit before the Internet, of course. So the culture of a company from the 1800s was poor for it, much less for security. There was no appetite for security whatsoever. So. So I was responsible for building automated. The infrastructure for automated facilities to build parts for trains. And then of course, the freight trains themselves and the systems that supported all of that and then all the repair facilities. So again, my career, you know, wanted to go more cyber, so I went offensive security. So I didn't touch OT from 2016 till 2022 and now. Love it. We didn't have cool tools 20 years ago. I had Wireshark and PRTG. And I was actually talking to somebody the other day that they still use prtg. And so, yeah, it was, you know, it's the only way I could get any kind of visibility on the WAN side. WAN links. And so, yeah, it's. I wish we'd have had all these cool tools 20 years ago for sure. I might not have left manufacturing, but it would have been a little more fun during incidents instead of watching wireshark for two hours. So that's kind of. I know that was a very long intro of that through kind of how I got into OT and why I'm doing what I'm doing. I, you know, I didn't say not yet, I guess.
[00:05:35] Speaker B: Yeah, well, and you said, you know, you raised your hand and that's. That's the number one rule that you're not supposed to do, right? If you don't want to volunteer, you don't raise your hand.
[00:05:41] Speaker A: Never.
[00:05:42] Speaker B: Which is why no one else did, I'm probably, I'm sure. So, you know, it's so funny hearing I've had so many people like you sit in that seat and have this conversation. And so many of us did. I came from an IT background and my dad was in power utility for 45 years. And so I grew up around power plants and all that kind of stuff, but I wasn't really working in it, I wanted to do my goal. I wanted to go work at Microsoft and Google and all that kind of stuff. And it wasn't until I really got back into power because I had some early part of my career. I got out of IT for a while and then I got back in. And to your point, like, you know, I've loved it ever since and I've dove in. I still dabble in the IT thing and work because, you know, you and I were talking before we recorded this morning. No, you go on an assessment. You go on a site and a client needs help. And a lot of times if they don't have a mature environment, they don't. There isn't a clear delineation of where is ot and it. Is this system, an IT system or an OT system? Is this an OT firewall or an IT firewall? And is this Windows machine, is this it? It's sitting on the manufacturing floor, but it's plugged in the IT network because they need to be able to connect to the ERP system and they need to scan barcodes and they're connecting to. They're printing to a zebra printer, but it also has a serial connection into the PLC that they're running, you know, the skid on or, or the lathe or whatever. The thing is that it's controlling. So it's this hybrid, right? And so we're having these conversations with IT and with OT and, and it's being able. And, and for me, I think it's that superpower of being able is no clear line of delineation. There is no. This is always an OT device, this is always an IT device. Most of the time it's black and white and it really just depends on its use case. But also it really almost doesn't matter. It's like, how can I do this securely to make sure that they can do their work? Because it's very easy to come in from an IT perspective and say, I'm going to lock this down, I'm going to kick this off the network, I'm going to put a firewall around it. But at the end of the day, they need to make those widgets, whatever they're making, and if you make it so hard, they can't do it, they're going to find a workaround anyways. And then the business shuts down. They're not profitable, the business closes. Like, so we have to keep this in mind. And that's not a mindset we have on the IT side, but on the OT side, we have to have that. And that's one of the bigger things I see from my perspective walking into an OT space trying to help. I can't just walk in with a hammer and hit people over the head with security tools and expect them to respond.
[00:07:46] Speaker A: Well, exactly, yeah, because it's definitely different priorities because you know, even like when I was going around I did a lot of projects in those manufacturing and repair facilities. So we would, you know, if I replaced a. I actually replaced all my Cisco switches with fortigates. This was like 15 years ago. They were the support for Cisco. They wouldn't, I couldn't even get budget to approve support for the switches at each site or routers. So I did get approval to put a fortigate in because then I was able to do SD wan, multiple links and all that stuff. So did that. So each time I did a project I did it at all the sites but I would find cable modems in OT or you know, and it's not that they were being malicious, right, because most attacks that are successful and detrimental, especially manufacturing are accidental or you know, a developer art incidents in American railcar were more developers accidentally wiping out the Unix server from root because they had root access and accidentally, you know, used the asterisk or whatever. So wiped out complete ERP systems which took things down for days. What's called, you know, then that's millions and millions of dollars, all that fun stuff. But again, you know, when I found those things it wasn't like, like you said, I can't go bash them the plant manager in the head for that because their intent is to save the company money, keep the plant running because you know the, some of the automated facilities that we built, you know, we had vendors from all over the world like we worked with probably Google primarily on the automation side. We had a lot of Siemens, we had some rock, but primarily Siemens back when I did a lot of that. But we had people from all over the world. We had heat treat furnaces that come from built from Canada. We had a huge forge that built out, you know, the, the double axle for trains and that came from Austria. So like you know, when they're doing these things plugging an Internet connection into a PLC cabinet to a device that has a port that does not require authentication, has no encryption. So now literally clear text protocols, no authentication, public ip. So like you know, for security that's worst of the worst of the worst. Right? But for them it's now they don't have to wait for somebod to fly for two Days to come, fix something. They're keeping the business running, they're saving the money. Saving money. They're increasing profit, profitability. So again, it's not like they're doing, yes, from a security perspective that's bad. But their behavior was not ill intent whatsoever. So yeah, you definitely got to have different, a different look on it. And I've seen a lot of tools get sold or clients. I'll talk to Sisos, I'll talk to that. Hey, we've had this X OT tool for three years, five years, seven years. We haven't got one actionable alert. We haven't changed anything because of this visibility. So that's where, you know, we cut, come in and really try to help make sure they're getting visibility. Most of the time it's just never got fully deployed typically. And then showing them how to use it, you know, all that fun stuff. Yeah, definitely. But yeah, definitely great point on that because yeah, it's definitely all about availability because you know, it could be the most secure business in the world, but if the power shuts off for a week, it doesn't matter, you know?
[00:10:28] Speaker B: Nope, it doesn't. So how I've had this conversation so many times, I know it sounds like a broken record for you by listening this podcast. Like these are problems that we have in OT and it's not going like we're not going to wave a magic wand and replace all of the equipment in an OT manufacturing facility or power plant or something like that and be on the bleeding edge. And honestly that would probably make things less reliable. Like how many times do we, you know, patch a Windows machine in it? Blue screen of death. Right? You know, we don't want that in an OT environment. We'd rather just leave it over in the corner and don't look at it and protect around it instead of having to patch it. Like patching is just not something you want to do. So how are you having these conversations? How have you been successful and kind of flipping that script in the conversation that you're having with these CISOs and how they look at risk differently in OTNI?
[00:11:12] Speaker A: Yeah, it really just depends on the locations. So working with companies and I met with a company in Dallas this week that was, I think sounded like the 1850s. It was not even, it was not even manufacturing. But again, so they have a lot of very old facilities, but they also have new facilities. You have new data centers being built, new automated distribution facilities with robots and self driven forklifts and they're really cool and. But that's very different than supporting a manufacturing plant that has all Windows 98. Right? Because those legacy ones where it is too expensive to replace every hmi, you know, every machine just to replace the hmi. And a lot of times those software and hardware vendors from the 90s and 2000s are out of business or the people who wrote them or retired, you know, no longer working, those kind of things. So patch management, you know, if it's a brand new facility and the devices are supported, you know, if you come in Windows 11 or you know, Linux, whatever operating system it is, and you have supported, you know, Rockwell, Siemens, Emerson, whatever equipment, then that would make sense to prioritize patch management alongside segmentation and micro segmentation. But for these older facilities, you May, less than 10% of the devices may even have patches. Right? So it's, you know, that's like you said, that's where the mitigating strategies come into play. You know that because it is just so easy, patch, patch, patch every day or whatever. Even if there were patches in ot, like you said, you can't just shut down the facility to do something that may or may not ever even be needed, you know, depending on the situation. So, yes, definitely, most different strategies in our, and we have to help our CISOs understand that because if they come in saying, hey, you know, we're going to lock down this firewall, we're going to add encryption in ot, we're going to add authentication. When there's a down, there's an outage, even if it was a year ago when you did this, the first thing they're going to do is shut off all security, get it back running, because they're going to blame that 20 years ago the network was always the blame, you know, always got blamed for everything. Even if it didn't matter if there's a database application, whatever, but, but now it's definitely security that gets, you know, kind of blamed for anything extra latency which can hurt OT systems, you know, legacy things. So there's just so many things that it's like you just got to have different controls, you got to have a different look on it.
[00:13:17] Speaker B: So, yeah, I feel like a par paramedic or you know, a first responder showing up to an accident and you're having to triage. They've got a broken arm, you know, their nose is bleeding, you know, their, their clothes are ripped off, but you know, the car's on fire and you know, they're in the middle of traffic and there's more cars coming at them. Like you have to be able to look around because there's always going to be anytime you walk into some place, and obviously maybe it's more glaring in an OT space, in an IT space, at least the way that we're talking about it. But there's always going to be things that you can find. But it's about, oh yeah, you know, not fear selling and not. The sky's not fall. Sometimes it is, sometimes the sky's falling. But you know, sometimes it's just a matter of, okay, calm down, deep breath, you're here, you've been running this way for 10 years, 20 years, 30 years, 40 years. So you know, just because we see it today, it was there yesterday and you were fine. Right? So let's figure out a plan and let's prioritize and say if I have five minutes to spend on something, this is what you should focus on. And then after you get that thing done, then this is the next thing. And then this is the next thing because you're not going to, you can't boil the ocean, you're not going to solve all the problems overnight, like prioritize the things in order. And that prioritization, I think is the big, biggest difference between IT and OT of where I focus my time and energy.
[00:14:26] Speaker A: Exactly. Because yeah, if the sky is burning, a fire falling from the sky and there is an incident and OT is shut down, if the power facility is not distributing power, the water, wastewater treatment facility is not cleaning water, if the nuclear facility core is going unstable, you get OT operational, you unplug any Internet connections, you let ransomware run inside, you let any malware go, you get it running, then you stop the malware, you stop the ransomware and then recover from there. Like you said, it's, it's, you gotta prioritize the availability over that confidentiality for sure. Like, because the data in ot, like one, it's one thing for data in IT where proprietary information or sensitive information. Yeah, yeah, all that fun stuff, credit card information. So that's data that needs to be protected for a long time. But data and OT is real time. Like the temperature of that heat treat furnace. It matters right now. It didn't matter what it was two seconds ago. What is it right now? Where's, what water level is that? You know, cooling stack at, you know, all that. So it's just used differently. But that's also, I think it's called now cyber physical because the data there, if changed improperly, you know, up or down, left or right, whatever could hurt somebody. Could physically hurt somebody, kill somebody. You know, accidents happen all the time and especially large operational facilities and things. So, yeah, you definitely don't want things turning on when they shouldn't be or turning off when they shouldn't be, you know, those kinds of things. So, yeah, prioritization is it. Because when we hook up a visibility tool, we often find, hey, this device is, has this virus, you know, but it's like, like you said, it's probably been there for 10 years. Years, you know, or unpatched firewall for 10, whatever. You know, there's just. Because see behind me a lot of my buyback, I'll be a manufacturer, you know, some of that PLC equipment, I mean, very old, like SLC 500 and we got Windows 98 back there because we have a lot of clients still running Windows 98. So yes, this is very much a different, different strategy. And, but like you said, when you get to the wreck or the OT environment, I want to equip my team, my engineers or, you know, paramedics in that situation to go in and be able to quickly assess the, the damage and then have the tools in their kit, Flyway kit, assessment kit, whatever you want to call it, and have an array of tools that no matter what they see, they have it. They don't have to ask for an Internet connection to the cloud. It's all there on prem. You know, they can help resolve if there's malware running around or provide that visibility or, you know, start integrating things. There's a lot of great data even sometimes about OT in a lot of the IT systems, like a cmdb, for example, or, you know, things that somebody had to purchase these things and they may be in the purchasing system. So a lot of, thankfully, a lot of OT tools now have integrations with these tools to help pull a lot of that asset data and start doing traditional things like asset management and kind of, because it's hard to secure if you don't know it's there, you can't secure it properly, you know.
[00:17:04] Speaker B: Right. Yeah. I mean, many times when I'm walking into these places, all the way back to, you know, I was CTO of Industrial Defender and we talked about that stuff, you know, working at ey, working at all these places. Even when I was an asset owner at a power utility, right, that was one of the main things that we focused on was I have to understand what's in my network and then I can start defending it. Right. It's hard to defend what I don't know is there. There I know about these things. But what about the stuff that is over in the corner that nobody told me about? Right. How do I. And those are a chain's only strong as its weakest link. We're throwing out all sorts of puns today, Right. But it's so true. Like if, if I have protections and that doesn't mean that I have to remove the Windows 98 thing, it just means I need to know about it so I know what, what vulnerabilities it has so that I can protect it. And that doesn't mean patching. You can't patch Windows on ea. But I could put it in a segmented environment. I can put firewall rules on and I can make sure that, you know, I disable USB ports and things like that that make it risky. I can reduce that risk. It's never zero, but I, it, I know I'm not going to plug that Windows 98 machine into the Internet. Like that's a definite no.
[00:17:59] Speaker A: Well, you can, you know, you can.
[00:18:01] Speaker B: I don't recommend.
[00:18:03] Speaker A: You don't want to. Exactly. So yeah, it's. Yeah. But again, like to me this stuff's fun. Like it's, you know, you're dealing with retro. So guidepoint has a lot of crazy smart people and we're cyber security only like 1100 employees. And when I go to his GuidePoint conference or any other conference where GuidePoints at, a lot of times it's the huge thing right now is AI. Do we use AI for our business? Do we, do we block all the AI? Do we, you know, does AI protect our AI and defend our other AI? You know, and then I'd jump in, it's like we're going to go a little retro here and you know, I'm not going to talk about AI. So you know, it's just, I don't know, I don't know if I'm, you know, definitely older now than, you know, all the younger folks, but more traditional command line and pre cloud services. You know, definitely traditional active directory windows servers all over the place and got a couple of Right. Servers back there too, but yes. So that's kind of how we help a lot of the CISOs is just understanding how to talk with the asset owners. And we have to keep it simple. Simple. When we started building this out, you know, two or three years ago, we used a lot of things like 62443, Cisco plant wires, converge Ethernet and all, of course, all the guidance that all the different government agencies put out, fda, Food and Drug Administration puts out Cyber guidance for medical devices, epa water, wastewater, NERC ship for power. You got TSA for an FAA for air, TSA for rail. You know, cruise ships are in their own international waters are whole different beasts. But so we made it so complex like it was so, so much information that like it was a hundred plus pages per site. For these assessments, you hand that to a CISO who doesn't own any of the assets remediation, you hand that to a plant manager. It's like, here you go, this is all the things you need to do. Go find your priorities. You know, it's. Nothing happened. Like nothing. Like we would then meet with the clients a few months later. It's like, hey, how's it going? It's like, well, we never did anything. It's just we don't know where to start. So we dumbed it down. I don't say dumb it down. We simplified it to essentially the SANS 5 critical control controls for industrial control systems. So we've grouped our vendor partners, our OT and IoT vendor partners into the four of the categories that you need technologies for. And we grouped all of our services into each of the five categories. So now when we do an assessment, we give them a crawl, walk, run roadmap based on those five controls here. So if you don't have an OT asset inventory, a lot of times CISO doesn't, especially a full one for hardware software, you know you're not even crawling right to crawl. We're going to build you an OT asset inventory. Then you can do your policies and procedures and asset management and you know, so on and so forth. If, do you even have an OT specific hour plan? Because if not, if you put in a visibility tool, doesn't matter which one it is, you're monitoring the span traffic. The SoC gets an alert that it's on an OT subnet. It could be a Windows box, it might be Windows XP. Even if it's Windows 11, that SOC person or whoever's responding to that incident cannot SSH to the OT switch and shut that port off. Even if it's a modern IT OS inside ot, your playbook has to be different. Again, we're talking about physical safety here, you know, that trumps everything in else from data. And if you don't have those separate playbooks, you're going to have outages, right? So again, our plan, like you already, you know, brought up like defensible architecture, micro, it's gotta be heavy micro segmentation. And I also include OT secure mode access and defensible architecture. Because of the quickest way to reduce so much risk is by treating your IT network like it's the fricking Internet. You know, if you're in ot, you want to treat it. I know they zero trust. We'll throw that one out there too. But you won't don't want to trust anything coming from IT it so. But you can't just shut all that off. You have to have visibility and know what's communicating. What should be communicating. Is that business critical? You know, is that engineer remoting into a PLC over BNC with no password? Is it a ERP system pulling data from a historian and then going over to billing? You know, once you figure out hey, these are legit things that need to happen, you do them the right way. You set up OT remote access for your engineers to get into the OT, OT, HMIs or workstations, whatever. And you do the same thing for your third party vendors. You set up industrial dmz. Then you shut off all the, all the IT to OT connections, right? Because now you should know what's connecting the two. So now when that accountant, I pick on accountants a lot in my talks but clicks that link in that phishing email, that and ransomware, you know, hopefully is stopped pretty quickly in it. It's not going to spread over to ot, right? That open SMB share that an engineer decided to, hey, this would be easier for me. Instead of walking out to the plant and sticking a USB drive in, I'm just going to share a folder with no password, right? So that happens ton so and then to. So we do a lot of like our team, my engineers do a lot of mentoring like you mentioned, entry level. So godpoint has a guidepoint security university and a lot of us help other people in the community, right? OT security is very niche, a lot smaller than the whole cybersecurity or IT umbrella. So for those that are entry level, you know, a lot of people say, well you can't be entry level in ot. It's like you can know how to secure something. It's the same thing when like offensive security and pen testing was new like 15 years ago. You're like, you can't be an entry level pen tester. Well, you can figure out what routers a vulnerability and know how to exploit it without knowing how to configure the router. You know, it's, there's different things. There are always things. If you go into a, at a visibility tool and they got a million assets, you can't pay a senior engineer to go classify every one of those, you have to have people learning ot and you got to prioritize the younger generation. If not, you know, when we're all put to pasture, then, you know, there's these to be people behind us ready to take the helm and keep this stuff running, you know. So from an entry level perspective, if you're entry level, I know entry level means, you know, let's say you're inexperienced doing OT, you've never done OT security, you may have been in it for 10 years, you may have been in the military the last 10 years, doesn't matter. You may just graduated college. So if you're doing it, Help desk, network, admin system, admin cloud, whatever, talk to the people who are running ot. Don't do what I did when I started. So I learned a bit the hard way on that OT is sensitive to things like port scanning. So when I did the first OSCP course, this was in Backtrack Linux, if any. Anybody remembers that. This is before Kali Linux, ever before Cali.
[00:23:49] Speaker B: Yeah.
[00:23:50] Speaker A: So I did it with Backtrack Linux and Open Boss was kind of the tool to use then. So of course I had my IT computer on the IT network and I also managed these physically separated O. It was just a plant network. It was just one or two ones to say, not one. Everything was on that. And then of course it was like a/16. And they just kept adding more stuff to it. But so of course I plugged a second NIC into that network. So now I gotta do a home nick. Hey, I can play with this older stuff. So I run Open Boss on the OT network and. And of course weird stuff happens. But the craziest thing, Help desk calls me. I was like, hey, there's a lady in shipping freaking out the her printer won't stop printing the message, are you dead? So, so open vos. That's like one of the messages. Like it's in. So like it sent this message, I guess instead of peeing it was like, are you dead? I don't know why you wouldn't do are you alive? But she thought somebody was like stalking her or something, like she was freaking. So I had to there and Apollo profusely apologize, but. But then I learned like, oh, you can't really do. You know these things are not the same, right? So, so don't do that. So if you go meet your OT asset owners, people who help with the network because again, they have firewalls out there. They got switches and routers. They're not firewall Experts. I wouldn't even call my OT engineers here that are cybersecurity focused firewall experts. That's where we bring in a firewall expert along with an OT engineer and together they make sure that's working properly. When we do an OT pen test, there's always it. There's oftentimes active directory and ot. So we have to do it very differently. We don't even send broadcast traffic whatsoever. But it's a somebody from TAS or inside simulation who's active directory expert and all their misconfigurations and weaknesses. There's somebody with windows privilege, privilege escalation expertise because nobody can know all of this. It has to take both sides working together to have a valuable pen test. And I don't even recommend our CISOs do an OT pen test until they have an inventory visibility because I can tell you 10 findings right now without charging you a dime that you have an ot. So it's like. But some, a lot of times that's what it takes to get leadership buy in is a tabletop exercise, a pen test, whatever. So we'll start getting to know the people who run your OT assets just to start learning terminology. And there's great training now, like Mike Holcomb, his whole YouTube, I mean, great YouTube training, his util sec channel on YouTube. There's, you know, COMPTIA is putting out, I think we're in the world, they're calling it industrial control roles. Yeah, OT whatever, plus whatever they're calling it. And then of course, sans. I know sans is expensive, but there's a ton of great training out there. But continue to, you know, improve your IT security skills while also learning ot. Because I think the career growth is going to be great for the next three, five, ten years. Because as legislation happens, you know, I think it was 45 states enacted like 75 cybersecurity bills. A lot of the states, a lot of the administration in D.C. are now focusing on critical infrastructure, power, water, road, how, you know, commercial facilities, you know, all these things that our country relies on, airplanes, you know, how much media attention does it did I get when the, you know, a system is down on a, you know, for a plane or for traffic or air traffic control. Like now you got hundreds or thousands of people in a miserable state. It's different when it's rail. You know, rail goes to a passenger rail car. You can go pick them up on a bus, it's usually not that far. But for planes it's a much different scenario. But yeah, just start and start Reaching out to other people in the community, you know, because I always tell people I mentor a lot of veterans in the military to get into cyber jobs. It N O T and like you need to have a mentor to, you know, progress yourself. You know, find somebody that is doing or has done the role that you want or you think you want. Because to be honest, before COVID I started meeting with mentors because I thought being a CISO was the logical path for me to become in the C suite, you know. So when I started meeting with ciso, I was like, I don't want to do that. So one of them, one of them suggested start mentoring veterans to get into cyber from a technical perspective. So I started doing that with several of the veteran mentoring platforms and like, that's really when my career really took off. From a leadership perspective is helping other people. But, you know, have a mentor, but also be a mentor. There's always somebody behind you as well. Even if you just finished college, you got four years of college students behind you. You know, if you just, if you've been help desk or soc for a year, you got people who've never done that. So be open to helping others and you know, be kind of course, because not everybody's going to know as much as you, but you know, on where you're at. So, you know, I know a lot of times in cybersecurity it can be pretty, you know, hard to ask questions because you feel like you're dumb. But yeah, definitely, definitely need to, you ask questions to learn.
[00:28:19] Speaker B: So yeah, put the ego down, be willing to ask the questions. You know, I also think that's one of the big things with OT to double click on that you talked about there, you know, there's a lot of things. And so I've been doing this since oh, 2010 when I started building teams in power utility and I had, you know, a team of OT people and my team, I had six employees and another 10 contractors that worked for me or something like that. I don't remember the exact numbers, but around those Numbers we supported 45 power plants across the state of Texas, including mines that associated mines and you know, other stuff that kind of went with that. But the main thing was, you know, that this, the actual power generation, we had some overlap into the substations, but substations were kind of ring fenced off with encore in the state of Texas and they weren't really my responsibility. But with all of that, again, especially back then, there was no such thing as ot. We didn't have that term. Yet, like we did, it was controls and automation and INC and pnc Production and Control, Instrumentation and Control, dcs, like all that type of. Type of stuff. But I needed a team of people that had cyber skill sets, but I also needed people that had the understanding of the systems. So I would bring in people that had never been in a power plant, that never even heard of ot. They'd been in law firms and, you know, data centers and places like that, but they had networking capabilities. But then I would get. One of the guys that worked for me had been a control engineer at a power plant for 30 years at this company, right? So I brought him out of the INC group into my group because it was amazing how much putting him with a, you know, an it, you know, background person and how much they could understand and find a solution. Because the IT guy's like, oh, we'll just patch it. Oh, we'll just reboot it. Oh, we'll just lock them out. And he would be like, the hell you are. And this is why you're not going to do that. And this is why they'll kick you out of the plant. But it was amazing how much authority and ability for us to walk into these places with his credibility. Because when we walked in, we were. He was able to credentialize us even if they didn't know us, because he'd been there for 30 years. He'd been in the, you know, he'd done the control system upgrades and so would I, but the rest of the team hadn't, right? So there's a lot more to learn in OT than just firewalls and networking and asset discovery and threat protection. You have to know those things too. But you also like to really a great. I think the difference between a good and a great and an OT person is somebody that understands the functionality. When I walked in, I told you I was doing an assessment. I do them all the time. Whether it's a power plant, manufacturing facility, it doesn't really matter. I'm more focused on understanding why this. This equipment exists and why they have this thing connected. I'm not like, oh my God, you have this connected to the Internet. Like, I'm not doing that. I'm just, what are you trying to do? Yeah, why do you have an SMB to this up to your corporate network? What is the business function you're trying to get? And obviously the reason you set that up is because you had a pain and this was the only way you could solve that pain. Let me understand how it is. Before I just go in and write, you know, the, the letter that has all the red markings on it because your, your essay was awful. Right. You know, it's the 100 page report you talked about. That doesn't help anyone.
[00:31:11] Speaker A: Yeah.
[00:31:11] Speaker B: It's better to come back and say, hey, here's a better way that you can do that. Not that you can't do it. Not that the answer is to patch and upgrade and you know, have Fort Knox around the thing. That that's reasonable. What is this? What is a better way that they can do this? But you have to be able to listen to what the operator, what the engineer, what the plant manager is telling you, understand the business process, and then come up with a compromise, a better solution that helps improve security, but it also keeps them able to do their job. Because if you make it so secure, they can't do their job. They're to your point, they're going to unplug it as soon as you leave and go back to the way they were doing it before because they have to do their job.
[00:31:45] Speaker A: Exactly. Yeah. Because. And to the IT folks and you know, traditional IT and security folks, a lot of times it's easy to think of your business as a technology business. So like, even when, you know, I was in IT doing rail car, like nobody in our IT department made a dime for that business. We were supporting the people that were making the trains and making money. So if, if it comes down to, you know, the CISO telling the CEO, oh, you got all these S and B shares, we need to shut them off to the plant manager who's making millions for the thing, who's going to win. It's like it's going to be that plant manager every, every time, like without question, won't even matter. Like you're never going to win that batt. Well, don't go in there and call them dumb because they plugged in Internet lines and got all this stuff everywhere. Right. Like you said, they have pain points, you have new pain points now that you've opened your eyes to all this stuff. No T. So yeah, you have to have a good plan and a good relationship with them. They need to trust you. And oftentimes they don't trust people in suits and ties or from the corporate office. You know, get steel toe boots, get you a hard hat. Like go out there. Like a lot of IT people want to sit at their desk and remote in to wherever they want to do. They don't want to get out, out of their cubicle on in their office. Like, go out there. Like, you know, it's. It's really cool stuff, like seeing an automated facility. Like, you know, if there's 16 critical infrastructure industries. My favorite is transportation. I like things that move. Planes and trains and cars. Like all these automated cars now. And cruise ships. Like, it's. They're just really cool. They're like cities to themselves, you know, especially the cruise ship side of things. And. But you know, there's other things if you don't. If you could care less about that, like, yeah, do it, you know, go to power generator. I mean, power. Electricity is cool too. Just don't touch it, you know, so there's a lot of really awesome things. Our mining. We have a lot of mining class. I love our mining class. Like, mines are cool. Like, and they're not really. They're not even listed as one of the. They're more like manufacturing split. Manufacturing split. You know, several different critical infrastructure industries, and they're thousands of feet below the ground, which is crazy to me. So very much safety, you know, it. It wouldn't. It doesn't take much at all to create a very hazardous situation down in a mine. So. So, yeah, it's good. Good stuff.
[00:33:51] Speaker B: Yeah. So if you're interested in getting into ot, like, these are great things to think about. Or if you're already in ot, if you're the ciso, that's new, or you find yourself in a place you don't know where to begin. Like, it's very easy to. And I've been the consultant that. That's done that. Right. And it sounds like you have two, Patrick, where we beat them over the head with a hundred pages of. Of evidence of how smart we are and how bad they've done their job. And that. That doesn't help anyone. Right. You know, I remember walking into a facility and they had 80,000 endpoints that were out of date patching. And they were like, there's no way we can keep up with patching on this. What do we do? Yeah. You know, so just saying that you have to patch all this stuff. It's impossible. Yeah, impossible. So, yeah, how can I prioritize that? Right? And that's the conversation we need to have. And you walk into, you have this conversation with your IT people, you know that you have, you know, Windows 98 and you're never patching, and you have SMB and, you know, you have remote access and like, all of these problems, their heads explode because they just don't. They can't imagine doing that. In an IT world. And they're right. But the flip side of that, you know Sun Tzu, Art of War. Use your weaknesses and strikes as weaknesses, right? Things don't change very often to not work. So if you do it correctly, if you can monitor, you add monitor and you have an asset inventory. You begin monitoring your environment, you should be able to notice differences. These devices don't just randomly talk to things. They talk to the one the HMI talks to the PLC and usually nothing else. And if it starts trying to talk to something else, that should be a red flag. Because it's not like it just randomly, it's not trying to go the Internet, it's not connecting to Starbucks, WI Fi, nobody's installing anything on it. And anytime those things happen, it should be an obvious red flag that says, hey, something's going on.
[00:35:22] Speaker A: On, look over here.
[00:35:23] Speaker B: So that's the way that you can start protecting these environments is you just have to look at it from a different mindset than you do on the it. You're using the same tool sets, you just look at it from a different lens, understanding that I can't just go change everything. But that also means that you know this. A lot of these systems haven't been updated. Hell, many of them haven't been rebooted in three years. And not touch, not change, just normal operations and just sits there and does it, right. It's slow, but it does its job. And as long as that old saying again coming, my dad living working power utility, if it ain't broke, don't fix it. Right? Literally, that's the mantra of ot. I'm not going to just replace an HMI because you have a new one. This one does everything I need it to do. Why would I replace it? Microsoft was for a reason.
[00:36:01] Speaker A: Yeah, Microsoft wants you to. Every two years. Yeah, exactly. Yeah.
[00:36:05] Speaker B: But you don't have to like, you know, there's. Yeah, there's so many ways around that. And there's so much value to understanding because at the end of the day, there's so much equipment ot. There's so much out there that we have to be. Be able to protect it. And know when you walk in, you're going to find old stuff and you're going to find things that are crazy. You can't imagine, you know, things directly, the Internet, any, any rules you access, no passwords or if there is a password, it's literally written on the desk or on the monitor. You know, all that type of stuff. But that's okay. It's not okay. And it's okay at the same time, definitely something we need to continue to improve. But having conversations like this and helping you understand those things, when you walk into these places, you're looking at it from a how can I solve this problem in a way that the OT team, the plant manager, the operations guys would be on board with. Because if you can get their buy in and the other piece to this really quick, and I want to get your, your feedback on this is if you know that going in and you know that the plant manager, their. But they own the budget, own rights. And a lot of times I hear, well, we don't have budget in it, they won't approve anything. If you can find something that makes the plant manager or the operations people's jobs better, it makes the environment more reliable or easier to troubleshoot or easier to onboard or whatever those things are. If you can make their lives easier, they have the budget.
[00:37:15] Speaker A: Yeah, yeah.
[00:37:15] Speaker B: And then they'll pay for it. And you get to have segmentation, you get to have secure remote access. You have to sell it as a value to them as an roi. Because if it's just nobody wants cyber, nobody wants a firewall, they just to want, want the outcome.
[00:37:28] Speaker A: Exactly. So that's where like Godpoint, you know, we've been selling to the CISOs for, since the beginning of Godpoint. And that's where, you know, we're having to kind of teach our sales teams that, you know, if it's an OT asset owner that wants this. Because a lot of these OT tools not just provide the visibility in where risk is, but those anomalies like you talked about, like why is this IT IP now downloading PLC files, like that's just not normal behavior. So that tells you the anomalies and helps you if a third party vendor's doing stuff outside of the change window or you know, causing issues and they're saying no, we never touched IT type stuff. But the shared goal, if you're on the IT side or security side, whatever your shared goal with OT is to reduce risk. Patch management is a tool that you can use to reduce risk. It's just not as much of a priority in OT and it shouldn't be. You know, again, you have other tools you can use. You just gotta, you know, learn what those are.
[00:38:21] Speaker B: What's the craziest thing that you've seen or the thing that surprised you the most when you walked into one of these places and saw something way it was configured or a perspective of a, of someone, whether it's it or ot anything come to mind?
[00:38:34] Speaker A: Um, man, just all the Internet connections like you know I would did an pen test when I was a pen tester a long time ago and saw traffic. I don't even remember going to what it was going to but essentially I saw stuff coming from IPs in Africa through to multiple endpoints inside the network. Like I don't know if they had any, any allowed or what, but I saw all failed authentication attempts directly to a domain controller from Africa and directly to something. I don't remember where it was in ot but like communication. They were talking SNMP with public or private whatever two devices in OT and that Internet line came through it. It was, that part wasn't even configured appropriate like you said. Any, any stuff. It's so easy to misconfigure a firewall but even in it you're never 100% secure. You never have zero risk. Like there's always going to be risk. Yeah, it may be, you know, seem worse when you get into ot, but it's all about making, getting started somewhere you can plan for 10 years on how to secure OT and never do anything. You get started, pick your highest site, typically it's close to your headquarters. If it's not at your headquarters, pick your highest priority system there that's making the most money or will cause the most damages. From a financial perspective your CFO knows what system this is if you need to ask. So you learn what you need to do to secure that and then you follow your, the rest of your priorities. Right? Follow the money. Like if you're on a patch, a PC that gets used once a quarter because it's running Windows 10 and it has patches but you're ignoring the 30 year old stuff and you're not segmenting like you know, that's where you know you're going to be spending a lot of time and money on, on something that's not valuable. But you know, mentioned the budget, you know, so that's where we've had to teach our sales teams to if OT wants it, they can get it from it, they can get it for maintenance. I got most of the budget for plants for the maintenance manager because I'd go to these switches that were out in the environment, they'd be caked of dust. The fans wasn't running but as soon as they get rebooted or try to patch them, they never booted back up. So like you need new switches or we need new wireless access points so they could use Motorola scan guns to log in you know, time clocks or whatever. So, you know, so yeah, definitely. If you can get an OT person, they will always get budget because they're again, they're making the money. You know, it's not like, you know, being ciso and I don't even remember what accountants call like IT and security teams, but they're definitely not like revenue generating, you know, functions at all.
[00:40:53] Speaker B: So no, usually cyber is a, is a cost center. Not bad. Yeah. And but when you can find those OT problems and help them solve those, they have the money. Right. Most of the time. Right. If it's a healthy business, they're going to have the money to fix it. Especially if it makes them more efficient. Especially if it makes them, you know, if it saves them five minutes, that makes them more productive. It makes their ROI better and they'll find a way. Especially you know, if you're looking at safety systems, if you're looking at, you know, availability, you know that you talked about it earlier, the CIA triad, it's flipped upside down. Right. Availability is key. Nobody cares about confidentiality because there's no confidential information in it for the most part. In most scenarios, obviously that's, you know, the secret recipe of Coca Cola is secret. Right. But most places at a power plant, the steam temperature coming out of, of the boiler doesn't really matter, right?
[00:41:36] Speaker A: Exactly.
[00:41:36] Speaker B: It doesn't matter. It's the same at every steam turbine everywhere. So. So it doesn't matter. So awesome man. So I always ask this question to everybody, I kind of tee it up to everybody but you know, next five to 10 years, what's one thing you see come up over the horizon that may be concerning and maybe one thing that's exciting that from your perspective concerning.
[00:41:53] Speaker A: Is just probably just, you know, continue to get awareness. Like I seem like there's a ton more awareness in the last five years and especially my Stuxnet was like what, 15 years ago now. So yeah, like concerning to me is are we ever, you know, going to get head the investment in securing ot, Right. Or businesses going to realize, hey, a lot of private sec, private industries own critical infrastructure. So that's definitely a concern. And another concern is CISA and nist. You know, all these controls put out great things but they don't always apply to OT or not all of them. Some there's always pieces that apply to ot, but like, you know, like secure by design, for example, that if they wanted clients to buy or get their manufacturers to add security, add encryption, add authentication, like basic, basic things.
But even if you could Get Rockwell, Emerson, Delta, V, Johnson Controls, all of them to 100% have secure devices. They're still going to be vulnerable Firmware, you know, there's going to be vulnerabilities, but you have 20, 30, 40 years of legacy risk behind you. Like you can't just, you know, move forward and then, you know, hope, you know, there's nothing behind you. Right. So yeah, it's great to do these, you know, great things to. But that really never happened and I think there's. Now it's secured by something else. I don't even remember what it is now.
[00:43:04] Speaker B: But yeah, I just talked about this the other day. The Nava National Lab.
CIE Cyber Informed Engineering.
[00:43:11] Speaker A: Yeah. So yeah, it's. So there's so many things like that that like the government is great about planning these things and creating these things and research matter and Idaho national, all these labs like do these great things and then it never really gets picked up by industry, you know, so. So those are concerns. I'm great. The things is there's a lot of interest in people wanting to get into OT security. Like I mean we see. So I had a couple senior level jobs I posted a few weeks ago and over about three or four days we had about 110 people apply. I think we interviewed like five maybe, you know. Cause a lot of them were not really OT or more grc, whatever. And we were needing more engineering folks and. But then I posted a OT security analyst role like more entry level up somebody. But I hire people that are prior electricians, prior mechanical engineers, prior military. So it's. It's entry level in that they don't have OT experience or that maybe they don't even know network network architecture or security. That's what we want to teach them is those things. 1100 people applied first day. So it's like, it's like dang. So I had no idea. So if you follow me on LinkedIn and I didn't reply to your message, I got a thousand messages that day and connection requests. So I used to be good at replying on LinkedIn but I just.
[00:44:15] Speaker B: That's overwhelming.
[00:44:16] Speaker A: It's very overwhelming. It's like, man, I think it's like maybe, you know, I get some sleep every night but I'd have to sacrifice what I did get if I started on that. So. So yeah, there is. There seems to be a very heavy interest in. Because if there wasn't that nobody would want to go secure this. And then when the people who are running it retire or pass away sadly like there's nobody to run it, you know, so. So then that's where like industries or, you know, nations start crumbling is for the infrastructure. Like Rome had some great, great things and just things just fall apart, you know. Yeah.
[00:44:45] Speaker B: We have to continue to maintain them. That's so very true. So what's. How do people find out more? See, obviously follow you on LinkedIn if maybe they've got a job coming up and wanting to get into ot, reach out to you. It sounds like you. If you're a veteran and you're looking to get in the space, definitely reach out. Patrick has some, some connections there. You definitely the. One of the things I love about the OT space is, you know, technically Patrick and I are competitors. I'm a OT cybersecurity consultant. He's an OT cybersecurity consultant. I don't care. I see there's more than enough needs in this industry that needs good people, needs good companies that have ethics and integrity and capabilities. And I don't see us as truly competitors. Yes. If we're going on the same proposal, sure. We're a competitor. The rest of the time, we just happen to be in the same industry. And what I love about this industry is it's a very small niche industry and there's a lot of really, really amazing people in it. Right. And there's a lot of people that are willing to answer those questions and mentor and help and advise and guide. It's why I love doing. It's why I do the podcast, why I enjoy having conversations like this today with you, Patrick. So thank you so much for your time. How. How can people reach out? Find more anything you want them to know about with Guidepoint or yourself.
[00:45:49] Speaker A: Yeah, yeah, LinkedIn definitely the best you can. Reach out to others on my team and then guidepoint security.com of course, for our company. But Gut Point also has a university gpsu. So if you're active duty military, you're looking at Skillbridge. Look, you know, you can Google guidepoint security GPSU gapport security.com GPSU learn more about Skillbridge. It's a great way to transition. Transitioning from the military to civilian life is very hard. Don't do it alone. And I do a lot of the things I do to prevent or try to help prevent veteran suicide. But the hot. Sorry to end this on a kind of a downer, but the highest risk of veteran suicide is the first 60 days when you get out of military. So whether you're transitioning from careers, even if it's not to Military, like don't do it alone. Like if you apply for a hundred jobs and you don't get one phone call, like that's like it's, it's demeaning. Like it's painful to go through that. So have mentors. Don't be afraid to ask a mentor for help. Review a resume, you know, be a reference, you know, don't, don't do it alone. And then, yeah, so, so yeah, reach out to gpsu and then also if you're in college or recently college grad, we have paid internships. Like I have a paid intern starting next week. Actually two. I got two OT security interns starting Tuesday. So that want to do ot, you know, prior military want to or once prior college one's PR military. So you know, kind of grow up that force people that know how to do this, you know.
[00:47:04] Speaker B: So absolutely. That's awesome. I appreciate you doing that. Got a big heart for veterans and definitely want to pay that back. One of my best friends, gunnery sergeant in the Marine Corps and he does very similar things in data center industries. So there's a lot of great organizations. There's a company here called Overwatch that does data center manufacturing and works in that industry and they do similar types of things. So there's a lot of skills, bridge type environments to get into and you know, state of tech expectations to increase the capacity of generation by 50% by 2030. So it's a huge growth in a very short, I mean it's 2025, halfway through 2025. So there's three and a half years to spin up more than 50% capacity than what we currently have. So that means a lot of generation being created and most of that is because of all the data centers that are coming in because of AI, because of, you know, all of the cloud and all the things that are coming. So thank you so much for your time. Patrick, thank you for giving back and all the Guidepoint does and if there's anything I could do for you brother, just don't hesitate to reach out.
[00:47:58] Speaker A: I appreciate your time, sir. Yeah, thanks brother. Yeah, it's great to be here.
[00:48:01] Speaker B: Thanks for joining us on Protect it all where we explore the crossroads of.
[00:48:05] Speaker A: IT and OT cybersecurity.
[00:48:08] Speaker B: Remember to subscribe wherever you get your podcasts to stay ahead in this ever evolving field. Until next time.
