Episode Transcript
[00:00:00] Speaker A: If you think about it in a couple of ways. One is this kind of full stack component of the types of people that we interface with on pipelines, on water plants, on building control systems. They are people that are capable of this.
There is no shortcoming in their intelligence. It's just giving them things they can use.
[00:00:24] Speaker B: You're listening to Protect it all, where Aaron Crowe expands the conversation beyond just OT delving into to the interconnected worlds of IT and OT cybersecurity.
Get ready for essential strategies and insights.
Here's your host, Aaron Crow.
Awesome. Hey, thank you for joining me today again on another podcast of Protect It All. Steve, thank you for joining me. Taking time out of your day, we've been kind of tangentially connected. As I mentioned before, I've played around with your product a bit and all that. So why don't you introduce yourself, tell our audience who you are and kind of your journey into this cyber OT type space.
[00:01:05] Speaker A: Sure. So, Steve Kiss I'm founder and CEO of a company called IP Meter. We provide appliances and tools for testing OT in an operational setting. I got into this business years ago, let's say 25 years ago. I got into this business out of college doing network engineering. I worked for a network equipment manufacturer as an engineer for those folks. Did a lot of protocol analysis and low level activities, routing and switching, ospf, bgp, that sort of thing.
And as I kind of worked through those activities, I really found myself gravitating towards large scale projects.
And those projects led me to train, airport, casino, hotels, health care, that sort of thing, just building large construction pieces.
So I ended up in the field in different capacities, either at a manufacturer or integrator or customer, you know, building infrastructure in. In that journey it became really clear that there was starting to be a separation of operational networks from IT networks. The operational networks that were out there were serial based, did a lot of work in that arena. But as we started to build networks that were higher speed, greater capacity in different ways, shared medium, we needed to have some network expertise in that arena. So I lived in that for years and years and years and just built anything that I could get my hands on. Work for a lot of general contractors and electrical contractors and so forth, doing those low voltage systems, everything from scada, access control, video surveillance, that sort of thing.
That's kind of the first kind of part of my journey.
Kind of along the way I started to realize that I needed to do a lot of pre configuration, pre testing. The general contractors were really pushing for me to do what's called factory acceptance testing in cybersecurity, in performance analysis, that sort of thing. So I started building labs for these projects and as part of those labs I build tools in that environment. And about four or five years ago I started to realize I could downscale those tools.
Me working as a little cottage industry, you know, making my own little quilt while there were other quilt makers making their quilt didn't really scale out. So it was time to, to build tools that, that can be used by a wider audience. Right. So not cybersecurity experts, not network people that can basically plug something in and get results out to know what the status of their environment is. And that's kind of the journey of IP meter.
[00:04:12] Speaker B: I love it. I mean my career has been similar and coming from working in network engineering and architecture and systems administration and all that kind of stuff, really starting out in the IT world and then kind of transitioning into this space. Most of the people that we work with in these OT spaces are not technologists, they're not network engineers, they're definitely not cybersecurity people. But they are the ones usually that they take the control, control system engineer and they give him the hat, him or her, the hat that says you're now the technology person. You are responsible for the switches and the servers and all the things. Even though they don't have a CCNA like I did and, and a CISSP and an mcse and I'm dating myself with all the certifications I've had in my past. But, but all of those things are, are there, they're responsible for it, but they don't fully understand it. So they may have tools, but they don't even necessarily know them. So having tools that are super simple that I can take out, plug in and just not have to be a doctorate in cybersecurity to be able to get value out of it. Right?
[00:05:14] Speaker A: Yeah. Look, there's a couple of ways to look at this. One way is to say there's other parts of our industry that do this, right? Like, like it's really simple to take a fluke meter and measure cable and you don't have to be a near end crosstalk expert in order to, in order to know that the cables either pass the green button that says test in some models, auto test in other models. Those two pieces kind of drive that. I think I've always had a little bit of a slight against the IT side frankly, because it's this specialization where in the OT people that I'm around in my world, they're full stack people, right? Like in the development world we call them full stack. They do the hmi, they do the cabling, they, you know, even in big plants there's kind of a, there's not so much segmentation. So to take a person that's got a chemistry background or a biology background in a water treatment environment and, and assimilate them into our world is not as big a challenge as it should be. The challenge I think is that the tools that they get to use are incredibly complex to configure, set up and operate. You know, it's not so much that you have to be a network expert, instead you have to start out being a UNIX expert in order to get the tools off the ground. And that's a, that's a challenge. That's, that's, you know, that, that is a challenge if you, if you think about it in a couple of ways. One is this kind of full stack component of the types of people that we interface with on pipelines, on water plants, on building control systems. They are people that are, that are capable of this.
There is no shortcoming in their intelligence. It's just giving them things they can use. The second part is we've got to get serious about the fact that we don't have the staff we need. And there are, you know, there's a huge shortage, there's, there's just this massive shortage of folks. There's a quarter million people short in the cyber security industry. And I'd argue that in the OT side of that the problem's worse because you also have to have these expertise, this expertise in whatever operating facility you're running, baggage handling or whatever they are. So we don't have a choice. We've got to convert these people, we got to assimilate them into our world of cyber. And I think we're starting to see that happen.
[00:07:50] Speaker B: Yeah, I completely agree. You know, I worked at a power utility and, and I was the OT cyber security manager responsible for. And all that meant was anything technology at a power plant was my responsibility. So switches to your point. We called them multi skilled, but it's the same thing. I first have heard the full stack, but that's exactly what we were. My team had to Support everything from VMware to Windows servers to patching servers, the backup server, the antivirus server, the firewalls, the switches, the all, all the layers of the O OSI model. Every single layer we were responsible for from end to end. So my guys had to be an inch deep and a mile Wide like they weren't firewall experts, but they had to be good enough in firewall that they could make sure that it worked. And I say this all the time, the IT side of the business. So this is working for a power utility. Right. Their entire product is power generation. That the company that I worked for. Right. That was their only product. And, and the IT organization had hundreds of people, teams dedicated to networking another firewall team, another, you know, VMware team and all these huge groups. And I had six people on my team, plus some contractors, but let's say 10. And I was supporting almost 50 power plants across the state of Texas.
[00:09:04] Speaker A: Wow.
[00:09:05] Speaker B: So it was, it was so much more responsibility and difficulty. And my thing was the thing that was directly correlated to the business making money. Yeah. And it was completely inverse of in my perspective, how much. Not that I should take. And I don't want to say that I should take away from the IT because that's important too. That's our first layer of defense. But, but the fact that they had such large teams and such large budgets and such mature processes and we had a team of six and we were just flying by the seat of our pants.
[00:09:36] Speaker A: Yeah.
[00:09:37] Speaker B: Says that it's missing something.
[00:09:39] Speaker A: Yeah. I think that, you know, but I would argue that, you know, the kind of NERC FERC component of the business provides the structure to, to incorporate performance testing, to incorporate, incorporate cyber security into it. So we have these very mature processes that deliver water or power or, or movement elevator up and down without someone falling through to the pit. All of those things have already, have already laid the groundwork for all of the operating technology systems to bring in cyber and performance analysis into the environment. Right. The only thing that's really missing is tool sets and a method to basically bring in capabilities. Right. Like to bring in the ability to actually execute it.
So it is interesting that we're kind of at that space. We don't have a choice to get our way out of IT by doing it with just consultants. We don't have a choice by bringing in IT to do IT or bringing in IT tools. We've got to create tools inside of the OT workflow that, that can be run by non.
They're technical people. It's very insulting to say that someone that's got a degree in chemistry is not technical, but they may, they're very technical, clickety clack IT VM experts. But we got to basically build that out. We got to, we've got to be part of that, that solution.
[00:11:05] Speaker B: Well, and I love what you said so those of us that worked and have worked at OT obviously understand the factor acceptance test. Fat or side acceptance test. You know, both of those are in the vernacular that we're used to hearing, but many people in the, in the IT world may not hear that or understand what that term means. But, but the whole idea behind setting up that factor acceptance test, doing that lab environment. So I did very similar things at a lot of different places, but both as an asset owner and as a consultant, building them for other clients, building that staging floor, that factor acceptance test where I'm going through and I build it and I test it before I ever take it to production. Right. And it's very obvious why we do that in an OT world. Because the, the, the, the impact of it not working is super high. And the really bad things happen if this thing's got, you know, power plant or a train or these things. Like there's, there's real adv hacks to this thing not working as expected. Whereas on the IT side, I kind of do it on the fly. I have a maintenance window. Yeah, test it. But I, I did virtualization. Like there's all these ways that we can do it in an IT world safely, but it's different in an OT world.
[00:12:15] Speaker A: Yeah, absolutely, absolutely. And the way I think that we can enter into this in one, one, you know, it's, it's. How do you, how do you start to build this workflow? Right? It's, you could do it per vertical, you could say, I'm going to do this only in dams and water, or I could do it in, you know, transportation only. But I think the right way to build it in is on a, is on a project basis because you already have the tools you need. In the construction arena, in the construction workflow, through a hundred years of, thousand years of building things, you already have a workflow. And it's just a matter of using the same workflow that's used in that factory acceptance test, in the system and unit level test in that process. And then when you turn over to the owner, to the operator, they already have a system and they can frankly use your same methodology to continue forward. Right? So where there may be some reluctance to build out facilities by the construction company, they now are under liability reasons. They've got to start looking at cyber security. They got to start looking at the performance. They're having shared networks with all these different subsystems that need to have a shared network. And the end result is they're under the gun. They're putting the process, they're holding the manufacturers and integrators heat to the fire. And then by end result, we can start kind of bringing in the, the local operational teams during our handover phase, so kind of builds from there. That's how I see this going down 100%.
[00:14:01] Speaker B: And to your point, we've been doing factor acceptance tests in, in these spaces for decades. This is not a new concept. This is not a cybersecurity concept. This is a concept that we've been doing forever. Right. The difference is, is how many people. Because again, I started doing this back in 2010 when we were doing fats and I was incorporating all the cyber and technology side into those fats that they weren't really in scope in the beginning. Like I had to force those into the scope of the project because I wanted them to test their cyber tools and validate that it worked before they shipped it to site. Yep.
[00:14:36] Speaker A: I mean, I'm in the same boat. We're still in that same role. Like, let's, let's just take something really, you know, nuts and bolts. Why are we still talking about Telnet in these environments? Right. Why do we still have, I don't know, 80% of our traffic and OT still is, is not. The transport's not secured.
There's a disconnect there. Right.
The, the specification needs to include these documents and it needs to be tested so you can't have Telnet into these devices. And then at the same time, you, you have to hold responsibility and say we're going to test for that. And if you don't meet that requirement, back to the drawing board. You're not getting paid to list things resolved. Right?
[00:15:18] Speaker B: Yeah.
[00:15:19] Speaker A: So.
[00:15:20] Speaker B: Well, it comes back to, again, coming back from my, my experience firsthand was the vendors want to do the right thing, but they have, you know, you look at. And I'm not, I'm not calling on anybody by name just to be rude, but ge, one of the largest control vendors in the world, everybody knows who GE is, Right. GE has amazing stuff I am not dogging at all. They've spent a lot. They do great things. I love ge, but GE has to be, they have to come up with a standard and a process because every, they're, they're trying to standardize and make it where they can cookie cutter this out for all of their customers as much as they can. So they standardize on a certain thing. And sometimes that may mean the, the, the, the lowest common denominator may be Telnet, because this one client in wherever can't support whatever ssh for whatever reason. So they just say, okay, we'll just standardize on Telnet because that works across the board everywhere. But it takes a customer to have a standard and say, nope, you're not doing telnet in my space. I don't care what your standard is everywhere else here we're doing it differently.
[00:16:21] Speaker A: Yeah, yeah. And I, you know, I think we've gotten a, we've, we've had a pretty good Runway over the last four or five years with CISA and some kind of pieces from the federal government. And this is not at all to be a political discussion, but as we start to think about downsizing the federal government, I'm getting more phone calls from municipalities. Right. Like I know I'm going to have to take care of this. I don't have EPA likely doing some of these things and making enforcements. How do I, what am I going to do? And my, I have had a significant uptake recently in municipalities starting to actually take ownership of their own cybersecurity and not saying, hey, I got this EPA test I'm going to pass. What, what's the minimum requirement I can do? Because that shift back to the municipalities is going to create a really different environment for us. And I'm already seeing it, I'm already seeing those, those local water guys, water and wastewater, start to say, I'm not really as concerned about your, your audit tool capability. That's great. Stop talking about that. How do I actually get the test to run and what are the things that I can do once I find things? What are your tests look like that I can, that I can. Then how do I action these things? And so we're sitting in a really weird spot, right? We don't have the labor to do the job. We don't have as much federal oversight that we're going to have and we have to employ tools that operators can use and, and, and execute. You go back to that kind of full stack comment that comes out of the development world, right? Like this. When you talk about since, since we have kind of a development arm to do our software, you look for these 10x developer, 10x developers, these people that can code better than the next guy. But really what they are full stack guys, they just have the capability to go top to bottom. And when you find those folks inside of the operational environment, baggage handling, water, electrical, your background, you know, these sorts of things, they can take this work on, they can add this to their workload as long as they know, as long as they have a mechanism. And we talked about one half of that, which is making the tools easy to operate. Right. Like, my appliance is headless, no keyboard, no mouse. The design is to power it up just like you would a flute meter. But on the other side of that, we need baseline scaling on the output that tells the user, good, bad, indifferent, log, don't care, whatever. And we have mechanisms for that. We have international and nationally recognized standards that, that can be used just in the same way. We can look at, we can look at edge cases for water.
You know, the water has, you know, too much of this chemical in it. You can turn around and use the same model to say we're right at the level that we got to take action on this thing. And, and that's, I think the other half of the story is getting solutions that are easy for people to consume the data. Right. Like, this is not good. You need to fix it. Why is Telnet a problem still? Well, yeah, you know, we can solve this. Right? Do you, you know, and, and the manufacturers, to your point, I think are ready for.
They have the stuff built in, right? From the, from the board to the operating system to the system level and wider. The manufacturers have the parts. It's just a matter of turning them on. It's just a matter of configuring them in the proper way. Right?
[00:20:17] Speaker B: Yeah, absolutely. You know, it's, it's, you know, it's like I, I tell my kid to clean his room and he's going to clean it to a certain level that he thinks he can get by with, but he's not getting down on the ground with a toothpick or a toothbrush and scrubbing the floor. Right. That's what his mom would like, but that's not what happens. Unless she, she makes that a requirement. Then he's going to do just enough to check the box. And sometimes that's what compliance is.
[00:20:38] Speaker A: Right.
[00:20:39] Speaker B: You know, the power utility world, NERC SIP is a great. You know, if you look at the critical infrastructure, 17 critical infrastructures in this country, I would say that unarguably, NERC SIP is probably the reason why the power utility is a, is a leader in cybersecurity in the space. And on the flip side, there's a lot of companies that just check a box to be compliant and skirt around actually implementing security policies because they've done nercxip. What else do I need? I've, I've done all that I have to do, and I'm not going to do more.
[00:21:10] Speaker A: Yeah, I Think it's really, you know, you kind of hit something that's a nerve for me because you know, the great thing about, about power utilities is there is the central binding in the transmission, so there is some relationship. There are other critical infrastructure items that have much more siloed where there's no connectivity. Building management, you don't connect to the building down the street and therefore you have a group that has to have a certain standard across the board. If you're running a water utility in one city, it doesn't affect what happens in the other city. So you know, I, I am most careful with those critical infrastructure components that have the least amount of interface with other, other activities because there's no like, you know, what is it Rising tide raises all ships?
[00:22:04] Speaker B: Raise all ships.
[00:22:04] Speaker A: Yeah, yeah, Nerc SIP raises all the ships. Doesn't exactly work that way in building management. Right. You know, your elevator control system which is now connected to the Internet for my, for a number of reasons is one, that is one where it's lowest cost and minimum configuration and great, we're complete. Let's get back in the truck, put the ladder back on the roof and go. Right. So you know, it's, it's, it's, it's a challenge. So you know, and that's going to be building managers that, that, that, that put that into their contracts.
[00:22:40] Speaker B: Absolutely. And you know, I'm actually an advisor for Building Cybersecurity, which is a nonprofit organization.
Lucian is the CEO and founder of it and he really, he came out of, out of government and he built this organization because there were so few building management folks that really understood the risks that they had in their environments other than what their vendors told them, etc. But he really understood that they, and they don't know where to start. Like many of them, even if they understand they need to do something, they don't know where to begin. So, so he started this to help that conversation. Right. Like give them an easy way to start. Here's a, here's a nist, you know, kind of a framework type thing that you can use in your environment, you know, a step by step process of what to do. But to your point that they don't really understand what it's there and they may have six buildings and they're not interconnected and they have different stuff in them and all that kind of stuff. And, and again, they're real estate people, they're business people. They're not to, to your point though, it doesn't mean they're not technical doesn't mean they're not capable of understanding it. They're super intelligent, capable. It's just not what they. They grew up and they don't have 25 years experience like you and I do doing this, right? So they need people like us to help them understand. What do I need to do? Most. Most of the people want to have a safe, to have the elevator work. They want to have their tenants and the people in there to be safe and the building be reliable and them to be able to lease it, like all that type of stuff. And they're willing to do the work if they know what to do and how to do it.
[00:24:06] Speaker A: The way that we've penetrated that, frankly, is we know there's all these different systems in the building, right? Conveyance, cooling, all these different systems. But the group that really has a definite understanding of the overall cybersecurity vision is the physical security folks. So it's, it's interesting. You know, you might start out with a physical security person and they kind of push you back a little bit. And they're like, oh, this is really complex. There's a lot of nuts and bolts and clickety clack and buzz, buzz, buzz. But it's the same concepts, right? It's perimeter, it's center, it's separation. It's all the things that they do that happen in physical security. We've copied into cybersecurity. Physical security was long, or, you know, putting a rock in front of your cave was around a long, long, long time before, before we had cybersecurity and we've copied those folks. So when I've, when I've done things like I've embedded my software into a physical security company into an access control system. But that conversation started by me kind of learning that language and making sure that I was using their language. What I found in physical security is they use the same. They, they invented that language, right? They invented the, the, the terms that we all use in cyber security. So kind of.
[00:25:28] Speaker B: Absolutely, right. It came from, from those concepts, came from physical security, layered defense, defense in depth, you know, perimeters, like all we, we. Firewall, like all of these things came from them. And these are not new concepts. We're just putting digital spins on them and trying to use. But the good thing is that is it's easy to. To your point, it's easy to communicate those terms because we're. And that's the way I look at frameworks, right, Is I don't look at NIST or 62443 or any of these, these standards is which one's better? I hear that all the time. Which standards should I use at the end of the day, per my perspective? And people will probably, you know, want to throw arrows at me for saying this. I don't think it matters. Whichever one that you're gonna do and use and be able to use and communicate and document and actually follow. It's just like a work, a workout plan or a diet. Which diet should you use? I'm not a doctor, but whichever diet you'll stick to is the diet that will work best for you.
[00:26:23] Speaker A: Yep. Yeah, I couldn't agree more. I think that, you know, it kind of bends a different point too, which is we know we have all these shared systems that are going into buildings and other, other facilities. We're kind of staying on this kind of building management component. But if we can, if you as a system operator of one of those systems or a, or a system integrator of one of those systems can become the expert in that building, you become the expert for all of the systems. So it's self serving. So one of the things that I, that I convey as I try to get people to kind of shrink the Kool Aid is to think about the fact that, you know, there's 12 other companies that are servicing your customer in other disciplines. You're being asked to be on a shared network. Do you want to be the one that runs that shared network or do you want to be a victim? Do you want to be, you know, at someone else's, you know, beholden to someone else? You know, and I've definitely seen enough of that, you know, where, you know, from a performance. We talk a lot about cyber security and protecting that. But cyber security to me goes hand in hand with performance analysis too. And as you start blending these networks together, people get stepped on. You know, networks get out of control. And, and that's important.
[00:27:42] Speaker B: Yeah. And, and to that, you know, because, and this is very common in ot. It is, it's, it's very standardized. I've, I'm using, you know, you're using my hardware. It's in the cloud or it's on my iron. We're using VMware. I've got a standard template of, you know, things that when I spin up, I'm turning these services off, I'm disabling telnet, I'm dis. Enabling, you know, snmp, you know, all that little, little type of stuff. We've done this for so long. We have these playbooks, but, but these OT Spaces, building management like you just talked about. There's, there's 10 or 15 vendors, maybe there's five vendors, maybe there's three. However many there are, every place is different. But all three of them, they only know about their spot. So when, when you do a factor acceptance test with the, with the elevator controls, they're only testing their system. They're not testing their system connected to all the other systems you have at your building. Right. There's no way that they could be an expert in those things. So when I did factor acceptance tests, I would force all of my vendors to bring all of their gear to my, to my party and I do a factor acceptance test in my lab and I'd have all the control vendors, all their cyber security tools connected on the network and then make them play together because that's how it's going to be when they plug it into the building.
[00:28:54] Speaker A: Yeah, for sure.
[00:28:56] Speaker B: That's the real test.
[00:28:57] Speaker A: Yeah. Look, it is a 20 year relationship with three vendors. You have your back end vendor that does your servers, you have your operating system vendor that does the operating system and you have your transport vendor, that's your network vendor.
OT is a 50 year relationship with 30 vendors. And there are huge organizations that don't ever, they're never going to, Never's a long time. It would be very rare for them to get in each other's shorts. So if you make elevators, you probably aren't going to make air conditioners. I mean there's multinationals that do that. But, but within the product lines, you're not going to have a combination elevator cooling system. It's not going to be combined. So that in and of itself changes the dynamic of how to approach operational technology from a non IT perspective because you have way, you have more embedded, bigger vendors, like no more by number that have longer contracts and they are not providing technology, they're providing equipment that provides something else. Right.
[00:30:10] Speaker B: They're providing a system, not, not a server, they're an entire system. Right.
[00:30:14] Speaker A: Yeah. And so it makes the situation kind of interesting because you know, when you do those factory acceptance tests and now part of the factory acceptance test, let's, let's take video surveillance at scale. You're going to do license plate readers in the cloud.
Are you going to, are you going to measure the cloud? Because that's going to be a big component of checking that box that makes sure that the license plate reader works. Otherwise you're going to have a disqualifying component on your test. And so it's an Interesting model that you bring up about bringing everybody together. Not just everybody together, but you need to identify that you're taking this out. One of these systems is going out to the cloud. Now I've got to create in my lab environment a test to get to the cloud. Because you're doing lpr, doing license plate reading in the cloud. You now, are you. I now need to make sure on behalf of my customer that we're testing that. Right. That that's going to work. Right. So opens up a can of worms.
[00:31:13] Speaker B: Yeah. So it is, you know, another, another thing and, and you may have run across this as well with the, with these control vendors is in it. If I install Windows, I don't have to install it exactly the way that, you know, Microsoft says I have to install it to get support.
Right. Whereas if I buy a GE Turbo control system, the switches that they deploy and the HMIS and all the things, if I change a component, if I add anything to it or whatever, the control vendors, their initial response, which you can push back on. I'm telling you, if you have this pushback, you're the customer, push back on these things. But their initial response and all of these OT owners are concerned that their vendor is no longer going to support their product. If I make a change and I, and I change the switch configuration or I disable Telnet, then the control vendor is no longer going to support me.
[00:32:04] Speaker A: Yeah. Do you think some of that I'm going to turn this around. Do you think some of that revolves around the fact that the owner relies on long term contracts from the installer and integrator? Like in other words, I've got a 30 year relationship with someone to operate the SCADA system on my water plant.
This is the way they've always done it. Like do you think part of that is due to that relationship versus the quick turnover of I need another MSP who's got a pulse.
[00:32:32] Speaker B: Bring them in here 100%. And again, going back to, you know, the GEs of the world. And again I just bring up GE because they're probably one of the largest, you know, that everybody will have heard of and probably dealt with in one way. And they cross all verticals that obviously they make trains, they make turbines, they make all sorts of stuff. So they're kind of, their hands are in everything. You know, one of the oldest country or companies in our country. Very, very old. Anyways, they, they, they have to support power plants and trains and all these different things. So it makes sense that they want to standardize on an architecture and that they can't support an infinite number of configurations. Now you and I understand that, that it, it worked in 20 years ago, right, when they're, when they had proprietary systems and they knew exactly how their thing worked. And if you, if you mess with the secret sauce, it doesn't work. It'd be like, you know, the same reason why you can't go throw a supercharger on your car and expect the vehicle manufacturer to, to actually, you know, warranty it, right. If you blow your motor because you put aftermarket parts on it, it's not your, your, your, your, your warranty is, is null and void. So that's kind of the mindset that they've had this entire time. But we're not talking about putting a supercharger on it. We're talking about adjusting how fast the fan within tolerance runs on my air conditioner. Right. I'm not changing it, I'm not replacing it. I'm not, I'm having it run in nominal places. I'm just wanting it to, I want to disable remote access to my, my car while I'm driving down the road. For instance. They're a prime example of I don't want anybody to be able to remotely control my car as I'm driving 90 miles an hour on the highway or 60 miles an hour on the highway. I drive fast. I live in Texas.
[00:34:15] Speaker A: So yeah, absolutely. Like, I, I think your, your point is well taken. And so some of that comes from the documents in procurement. It comes from the procurement path.
[00:34:28] Speaker B: Yes.
[00:34:28] Speaker A: That the owners need to take. They need to take the standards and the, and the work that's the good work that's been done in the standards committee and standards committees and feed that into their documents and see what comes back and see if they, they will find that people will accommodate it or people will ignore it until the time comes for non compliance and then someone's going to eat it. Right? And that's kind of how this all, to me, how this all started is that all of a sudden we people like you and I started shoving things in, in the procurement stage. And you know, some, you know, some poor estimator somewhere is like, yeah, yeah, whatever, that's subsection G. We're gonna go on. And all of a sudden, wait a second, I said that this was going to be, you know, that this was going to be stored securely and it doesn't appear that you have this stored securely. And then all hell breaks loose, right? Then they're like, oh yeah, yeah. So those that's kind of a way to start that for those kind of operators that are looking for a way to enter into this and figure out how to do it is to figure out, like you say, what standard is going to work. But to kind of use some basics doesn't matter. In the IT world, it would be, you know, PCI or HIPAA or whatever. It really doesn't matter. How do you prevent unauthorized access? How do you get notified if there is unauthorized access? How do you secure the transport? How do you secure at rest? These things are very, very basic concepts that all systems operators should be asking of their integrators, partners, manufacturers, that sort of thing. And we wouldn't be left with 97 or 98% of the IoT traffic or the OT traffic being open, being, you know, you throw a sniffer, my rack run. Throw a sniffer on an environment. It's, it's kind of appalling really. There's no, there's no excuse for it, you know, and I get it.
A turbine is a very expensive piece of equipment. And if it has a daughterboard on it that has an ethernet port that was built years and years ago, it might not have the compute power to do encryption. And so, you know, we talk about these pie in the sky things about, you know, each device having all of its own security and being secured. It's unrealistic. That's a million dollar piece of equipment that's not going away. You'll be dead before that thing moves, you know, and that, and that's the reality of it. So now we got to come up with what kind of interface are we going to put in front of that? What kind of firewalling are we going to do? How are we going to segment that out? What can we do to kind of mitigate that? Because, because that, that thing's going to outlive you. You're going to be dead in dust long before that, you know, that device comes out of the environment.
[00:37:26] Speaker B: So yeah, and you, you hit something really important to think through there. Right? And a couple of points I want to bring out a, you talked about the fact that we put it in, in the sourcing, right. Supply chain. As we're, as we're signing these T's and C's, as we're, as we're, we're doing this statement of work as we're buying these things. Put that langu. When you have the power, right, when you've already signed the contract and you're two years down the road, it's really Hard to go back and add things in because they're going to charge the heck out of you or they're just going to say, no, this is the contract. You agree to it, but it works the opposite direction as well. Like if you put that stuff in there, they have to follow that contract too.
[00:38:01] Speaker A: Yeah, for sure. And that's kind of why I lean on that. And, and your operational teams don't have the power to do that lift themselves either. So there's kind of three places you could enter in. You could do it in the, in the document writing, in the scoping portion of the business. You could do it during installation, or you could do it as part of your operational piece, but your operational team doesn't have the bandwidth to be able to take that on.
[00:38:24] Speaker B: So.
[00:38:25] Speaker A: So really you're talking about trying to do as you're doing plant upgrades. You're talking about starting to add this in on a piece by piece basis. People ask all the time, how do I do this? Like, right, well, you do this one bite at a time. You do it by taking. The next time you're doing a plant upgrade, that thing's coming down for a bearing replacement anyway. This would be a great time to look at the comm system as part of that activity. You're. You're coming up on this maintenance window anyway. And the maintenance windows are so different in, in OT than it. This isn't like Thursday night's a good night because less people for two hours. Yeah, this, these maintenance windows are.
We're taking down. You know, it's winter and so one of the cogen facilities is coming offline so we can do eddy current testing. Yeah, you have a great window right there. Right. It's a matter of aligning that. Aligning that window. Yeah.
[00:39:24] Speaker B: Anyway, well, and, and the other piece to this, and I love this right. Is it's just like with everything. OT is no different. Cybersecurity is no different. It's people process and technology. I can't just buy a tool and expect. You know the analogy. I say this all the time. People probably get tired of me saying this, but it's like I can, I can have the nicest woodworking tools in the world in my garage, but I can't just put wood in there with the tools and expect that I'm going to open the garage door and there's going to be like a cabinet or something beautiful that's built. I have to go in there and do something with that and actually do the work to make it happen. And if I don't know what I'm doing. It doesn't matter that I've got great tools, I still don't know how to use them. So I'm not going to be able to create anything beautiful either unless I get the knowledge on how to use the tools and the overall process of that. So the understanding that. And putting yourself into that mix is an important piece too.
[00:40:15] Speaker A: And that circles back to kind of why I built this IP meter product, right? Like, when I get down to it, it's because I needed a tool that was not, first of all, bringing all your, all of your pieces to work and building a tool out of components is foolishness, right? It's just not reproducible, it's not scalable. So if you, if the first thing you did to build a car was to like start assembling a socket wrench by hand and, and molding the metal and making, you know, building tips for your screwdriver, that would be foolish. No one would find that acceptable. But yet in, in other parts of the industry, that's been the method. So what can I build that I can just, what requires power, like just plug it in, Ethernet, what's the minimum that I can do? And why don't I have a reproducible appliance that does that? And kind of along that same lines, why am I only looking if I'm gonna, if I'm truly gonna be forced into this shared network medium which creates all these opportunities for feature set, I'm gonna share all this data and then I can, I can use my modeling to, to determine things. So you're, you know, I'm. The benefit of converged systems is feature set increase or performance analysis or whatever the case may be. But the downside of that is I got to have a clear control, control plane. I got to have quality control plane activities. And so I've now got to start thinking about not just whether or not the tool is reporting the right temperature, but whether the control plane that sits below that is operating efficiently, securely, highly reliable, those sorts of things. So there is a trade off between new feature set, buzz, buzz, buzz, iot, and all these great things that the vendor is telling you you're going to get when you install this new, you know, blower motor. These things are going to be great. But the downside is you're going to have to have better control of the control plane that's independent of the manufacturer's ability to look at their own data. Right? Because they don't. They might not even look at whether the control plane is operating, they just know that it stops sending data. Right. So to know whether these low lower level control planes, I look at the world, it's control plane, data plane and in that control plane world we've got a, we've got it, we've got a job to do there because we're asking to, you know, we're being asked to share the data and we're being, or to share the plane and we're being asked to secure the plane. And that's, it's not a big ask. Right. At the surface level you get benefit from having these new features reduced, you know, increased efficiency, reduced staffing. Whatever the reason is increased feature set. But there's a, there's a trade off. And the trade off is you have a shared medium and you have medium that is now less secure than when you started. So you gotta, you gotta, you know, you gotta start putting that stuff into those pieces. And that's, you know, that's been the, that's been the model for our business really is just going into those facilities and creating an appliance that, that people can plug in, operate and then get data out of their environment on supply chain performance and cyber simple.
[00:43:39] Speaker B: So, so your, your typical. Let's talk about a, I won't say typical, let's talk about the, the building management or a wastewater individuals. These are, these are small and the reason I want to focus on those is because they're not, you know, the Duke Energy of the world that has 150 million dollar OT cyber security program right there. It's not that. Right. It's an individual site that may have one guy and he or she is, is the multi skilled, you know, full stack person that all. And they're not trained in any of this stuff. Doesn't mean they're not capable. It's just they have five jobs and this just happens to be one of them that they have to do. So what is the, what is the use case for them? What are they looking for? What is your product doing for them? And, and how quickly can they start seeing value by plugging it into the network and starting to get some of that data?
[00:44:26] Speaker A: Yeah, so we only offer through partners and resellers because we want to be in that food chain. And the partners and resellers are not, not reselling our product necessarily. They're already in there doing the systems work that they're doing.
And so that's the first kind of differentiator is we're coming in through the normal channels. The second piece is I kind of alluded this because I kind of backhand that group all the time that says, oh, we'll just download this code, create a vm, stick the stuff. You know, it's just, yeah, I'll do that at lunch. Right. Like there's just no capability time. Right. So ship them something that's usable, like that's a usable component that they can start collecting data immediately.
And in that environment, make sure that the device is ready to go, it's hardened environmentally. Like, it's foolish to think that you can put this all in a laptop and go out to a can in the field and.
[00:45:25] Speaker B: With no air conditioner, with no air.
[00:45:26] Speaker A: Conditioner and get it to run for 24 hours. You know, go to a rail yard in the middle Arizona and expect that this thing's going to go into the case at a crossing and you'll come back 24 hours later and it won't be slagged piece of plastic. Right. So, so dust protection kind of environmental. Environmental protection kind of those pieces just being, just being respectful to the industry that, you know, it's insulting, frankly, if you're going to show up as with, with something that's not even going to work in the, in the environmental environments that, that people are used to working in. So then once, you know, once the unit is installed, which basically plug in, you connect to it via a browser and you set a few things. What's the serial number of the device? So we get some authentication. What, what, what IP address area do you want to scan? Right.
And, and when do you want to do it? Like, pretty much that's about it. We don't do a lot of custom configuration because there's no reason to. We'll throw the data out that the customer doesn't want. Right. So then we run that activity and then we securely send it either to something on site or up into the cloud and then we process that data and display it. So soup to nuts, like from the beginning of that install to when we start saying you have a critical vulnerability, here's the CVE that you need to read. This is the firmware update that you should be doing on your system is three days, right? It's three days that they can start using those activities. And what are we doing to get there? The federal government's already provided. They've for years have been trying to get operators to do things. They've poured billions of dollars into activities that allow us to, to help, you know, say, hey, this is a Rockwell device. I know what this is. And this is the problem with that thing and this is how to fix it. So we have kind of a whole sleuthing mechanism as well as a remediation component. And, and, and that's the core of the product. Like just how hard is that? It's just understanding the workflow that the OT folks do need to have in play in order to enter into that facility. They're not going to sit there with a bunch of nuts and bolts and a, in a welding rod and put something together and, and do that. They just don't have this time, skill or patience. And they shouldn't, they shouldn't be insulted. Right. They should have a tool that they can put in. You know, nobody expects them to build their own crescent wrench.
Yeah.
[00:48:07] Speaker B: They need a tool they can easily pull out of the toolbox, plug it in and it work reliably every time they need it to work, and then they put it in the toolbox, you know, in the crescent rich example, and they put it back in the toolbox until they need again. Right.
[00:48:20] Speaker A: Yeah. And I love when our appliances like, like it's going out because someone's already going to a lift station. And so while they're out there, hey, would you plug this thing in while you're out there? And I love it when it's just sitting in the back of the, you know, on the back seat of the pickup, on, in the crew cabin. It's on the back seat. Right. Because that's how it should be treated. We should have tools that do that and you treat your, you know, it ships in a Pelican case for obvious reasons, but beyond that, it should be treated like another tool. And we should, we should, we should really, we should really allow the operator to have those capabilities to do that.
[00:48:57] Speaker B: I love that. And you know, it's, it's so funny, you know, I've been doing this a long time, as we've said. And you know, and I've been in, I was a CTO of a product company in this space. So I've been the, you know, on that side of the business, I've been the asset owner, I've been the consultant. I've worn a lot of those hats. And there's a lot of, there's a lot of need and a lot of gap in our space, especially in ot, in the tool space. And there's, there's some really amazing, super complex, capable, buzzwordy things that'll, you know, do all sorts of fancy stuff. And then those are great. And there, there, there is A space for those. And you know, if you're a really mature environment and you're trying to get to that next level, those are good. But many times when I'm walking into places as a consultant, I'm, I'm talking to people that have done little or nothing and they're looking for basic foundational stuff. It's not sexy, it, it's not fancy. It doesn't need bells and whistles and AI and all the things. They just need to understand what's on their network and what's communicating and the basic level type stuff that's you and I take for granted, but they don't have.
[00:50:03] Speaker A: Yeah, I, I think it's.
I brought a salesperson on board a while back and he comes from a giant manufacturer and we were going through the product features the first time as I was displaying the product to him, he said, well, what about inventory? I'm like, well of course we have to have inventory because we're fingerprinting everything. What are you talking about? It's foolishness. He goes, no, no, no, no, no, you don't understand.
So many people don't have a grasp of their own inventory. And, and it really caused me to kind of rethink a little bit of the software, making sure that I at least mention. And I didn't do it in this. Which just shows that like when you get around other ot people, you miss the fact, you often miss the fact that people might not even have an inventory of their own environment that they can work in. Now energy, probably they do nuke. They do. Like there's certain places where that probably is not as applicable, but, but building management, you know, hey, we just plugged in that RF system so that we can get a data because we're doing ICE management on the roof. Right? That just was five more connections you added, you know, four years ago to do, you know, to do ICE measurement on the, on the roof. That's great. You know, let's. Maybe that should actually be documented somewhere. Right? So yeah, your point is valid. Sometimes it's the simplest things that people need and it's about creating that environment for them.
[00:51:35] Speaker B: Well, and even in, you know, even in these high. So I supported a nuclear power plant and I supported, you know, coal fired power plants and wastewater plants and critical manufacturing and kind of everything in between. And even those power utilities that had NERC SIP compliance, their NERC SIP documentation was perfect. But not everything is NERCX applicable. Meaning some of their assets were documented in their asset inventory for that and others that Weren't required to be. Weren't. And they didn't necessarily know about them.
[00:52:05] Speaker A: This is the greatest thing. This is always how I got drug in. When I was doing protocol analysis and just doing sniffing early in my career, people would do a PCI audit. And the first question was, what could we get off this PCI network and put somewhere else so it's not part of it? It's like you're fundamentally missing the point. Right. The point of this isn't to, like, reduce the set so you pass. It's to be inclusive. And it was always comical to me to watch. Who in this, in the chain of command would be the first to say, oh, this is definitely out of scope? Like, that was my favorite. Oh, that doesn't matter how insecure that thing is. It's out of scope. Right.
[00:52:47] Speaker B: 100. And it happens all the time. And, you know, I've seen it enough that it's just normal. But when you tell especially IT people that don't. That don't have that experience or understanding that, you know, there's a Windows XP machine that's running, you know, the nuclear power plant and, and a lot of these critical systems, and it's going to stay there. And it's fine. You have to. You have to segment it and mitigate it differently, but it's going to stay there because it's millions of dollars to replace that whole entire turbine to get that thing out of there. Right?
[00:53:17] Speaker A: Yeah. And that's exactly it. That is the. That is the crux of the whole thing, you know?
Absolutely. You know, I'm a pilot. There's.
There's navigation equipment that is in aircraft that is so hard to pry out of that equipment that the only time it's starting to be changed is because they can no longer get the screen for it because the manufacturer's not making the screen. And that's the time it gets changed. Right. And those, those. That is the story of ot.
That is the story of ot, Right, Is that, you know, you are the tail of the system. You are not dog of the system. And you need to learn that lesson day one, because you're providing, you know, services that have nothing to do with moving zeros and ones. They have to move in water and power and whatever else they do. Right.
[00:54:08] Speaker B: So 100.
[00:54:10] Speaker A: That's the crux of it.
[00:54:12] Speaker B: Well, and it's just a different mindset. Like, it's different again in an IT world. If I try to bring a Windows XP machine and plug it into a corporate network network, they're not even going to let it on the network. It's just going to get booted off, you know, whatever. Lockdown. And the answers would be like, tough. You're not bringing that onto my network. Here's a brand new laptop, use this one. Right? I can't have that perspective when I walk into an OT space. I can't just expect, oh well, you should just block that. You should just not do it. Let's just send patches because Microsoft released them. We should patch it now. It's critical. Yes, but if you break something, it's more critical that you just crashed the control system and the power plant or the, or the plane than it was some hypothetical, hypothetical thing that hasn't happened that I have other mitigating factors around. It's better to not patch many times in OT than it is to patch for obvious reasons.
[00:54:59] Speaker A: I think that it's even less dramatic than that. There's also the situation where there's one vendor that makes that thing or two vendors that make that thing and they custom build that for your environment and you, you operate that thing forever, right? And that, you know, there are, there are lots of OT environments that have that critical component of operation that is not often well understood, I think, by the wider community is that, you know, you're not, you have this risk to wastewater.
I actually love wastewater in a weird way because it is a waste, a bellwether for the whole critical infrastructure world. It's very, it's not interconnected, it's run often at the municipal level. It doesn't have a lot of pizzazz. And nobody thinks it's sexy to work in wastewater, except for when you go to flush the toilet, you're like, hey, that worked all right. Right? But what is the downside, what is the downside for when those plants go offline and they do go offline from time to time? Well, well, bigger reserve tanking, right? Better storage. But at the end of the day, when it becomes a cyber problem, it's about shipping that material somewhere else and letting another plant take it. It's a huge risk. And you know, and who's helping those folks out? You know, you're right. I'm a little more downscale in some of the things I talk about. I love to work in big, big, big industrial systems. Super sexy. But you know, your local, your local wastewater guys are doing a full stack deal every day and every time you do a deal, they're taking care of your deal, right?
[00:56:49] Speaker B: Correct. 100 so, so speaking of that, like, so we've Talked about a lot here the next five to ten years. What's something that is concerning and maybe something that's exciting coming up over the horizon with all this problem, right?
[00:57:03] Speaker A: There's a huge staffing problem, quarter million shortage in the US alone in cyber security staffing. Gotta get tools that are not more complex but are simpler to use so they, so it can be more inclusive to people that aren't cybersecurity experts to give them the tools. That's the first kind of, that's the first thing that I think in the, in, in the horizon that we're seeing. The second thing is I'm just continually, I see myself as a plumber, right. I mean I'm a pretty sophisticated plumber, but I'm a plumber. I'm putting, putting networks together, I'm putting infrastructure together for people. But it is great to see what the people, when I stick my head out of my hole, what people are doing with the, with the stuff that I'm building that, that low level infrastructure for. The way that the thing, the way that things are integrated, make them more efficient, they make it easier to use is it's just amazing to see in buildings the reduction in power in, in, in, in power requirements. You know, we talk about, you mentioned like coal fired power plants, nuclear power plants, solar, like we have all these different ways to produce power. But when we start to think about consumption related savings that we can offer, it's, it's dramatic. It's absolutely dramatic. Right. And so you know, those pieces really get me excited. Both the feature set capability of getting all this data to work together and doing these great things so that when I have proximity that when I come up to the building, it's already set for me when I, when I go to my workspace, I already have, you know, it's already set inside of my environment and my environmental variables are controlled. All of these things are great. I mean I'm just super excited for it and the fact that we can do it efficiently and, and with, with less waste I think is, is, is what I see on the horizon and I just want to keep building the plumbing to do that.
Right.
[00:59:08] Speaker B: I'm okay being a plumber.
[00:59:10] Speaker A: Yeah, I'm great being a plumber. Right. Like that's what it is. It's, you know, being in support of that infrastructure allows great things to happen. Happen. Right. So that's my, that's my outlook. We're going to continue to do just awesome great things. But we've got to tighten up that infrastructure to allow Those things to happen.
[00:59:29] Speaker B: I love it. Yeah. So how do people find out about you call to action? Like, how do they find out about you? Where do they find your spot? All that kind of stuff.
[00:59:37] Speaker A: Find me on LinkedIn. IP meter.net is our website for our product sales@ip meter.net reach out to me, reach out to my team. We're, we're into this, we're into this deep. We like, you know, we like to solve the problem. We like to ship things to people that want, that they can plug in and operate and use. So that call to action is reach out, tell us about your operational environment and we'll tell you how you can stiffen it up, how you can, how you can make it more reliable and safer for the communities that you serve.
[01:00:12] Speaker B: Awesome. I love it. Guys, everybody, the, the. Everything will be in the show notes. So links to, to get a hold of, of Steve, links to the website, all the things that we talked about in here. We'll, we'll make sure that we, we post in the show notes. Definitely reach out all of those small municipalities, the, the wastewaters, the, the building managements. Like those are, those are the unsung heroes that, that show up every day wearing all those hats or they're the full stack developers that are, that are trying to find the tools that they can support and not have to or to. I love the analogy of they're not expected to build their own tool set. They're not expected to build their crescent wrench. They should be able to pick something up off the shelf and have it be value add to them, not just something they stick in their toolbox and don't ever use.
[01:00:54] Speaker A: Yeah, for sure. Absolutely. So, yeah, so reach out to us. We're, we're, we're ready to help.
[01:01:01] Speaker B: Awesome. I love it. Hey man, thank you so much for your time. I really enjoyed the conversation. There's a lot of value and folks will really enjoy it. So, so thank you. I really appreciate your showing up here and talking with me.
[01:01:12] Speaker A: Aaron, thank you so much. It was great spending time with you.
[01:01:15] Speaker B: Absolutely. Thanks for joining us on Protect it all, where we explore the crossroads of IT and OT cybersecurity.
Remember to subscribe wherever you get your podcasts to stay ahead in this ever evolving field. Until next time, sa.
