[00:00:00] Speaker A: Hello, and welcome to the Consulting Specifying Engineer podcast. I'm your host, Amara Rosgas, and we're talking to Daniel shepherd with Dewberry. Today, we're talking about cybersecurity as it relates to building control systems. And this isn't a topic we talk about frequently, so thank you for joining me, Daniel.
[00:00:20] Speaker B: Thank you, Amari. I really appreciate the opportunity to speak to you today.
[00:00:24] Speaker A: Good, good. Well, let me officially introduce you.
Daniel shepherd is the control system cybersecurity discipline manager for Dewberry's Mechanical, Electrical, Plumbing, and Structural Practice based in Huntsville, Alabama. Daniel is a control systems, cybersecurity, and building automation professional with more than 20 years of experience in designing secure control systems for DoD and other federal facilities.
He's a member of several professional organizations, including Project Management Institute and ashrae.
So thanks again for joining me. Let's dive directly into this topic. What are control systems in the context of cybersecurity, and why is it crucial?
[00:01:12] Speaker B: Well, Amara, I mean, control systems in relation to cybersecurity, if you think about it, every building you're in has some form or fashion ability to control the environment where it be through heating, air conditioning, and ventilation, through access control systems, security systems that address physical security elements of the building, lighting controls. So all of these sophisticated systems that help operate a facility to its intended design purpose, though all those systems now communicate over networks, and they've been communicating over networks for quite some time.
But as it relates to cybersecurity, those networks and those devices have matured in terms of. Of capability, what type of data is collected, and they all operate over.
A majority of them operate over IP networks. So with that being the case, in today's advanced and technologically advanced facilities, where there's a network and where there are components that have the ability to store, process, transmit data over a wire or wirelessly, you've now exposed an additional element of complexity, and that is how do we protect it from cybersecurity. So you have to think in the context that the building is really not much different than your home with its home network and devices that connect to that network. So there's a lot of interconnected systems within the facility. So from a cyber stance, we have to make sure that we design these systems to be able to be resilient from potential cyber events, whether it be through vulnerabilities of software, whether it be from outside individuals wanting to employ some form of diminished capability, through hacking. I hate to use that word, hacking, but it's a reality. There's a lot of bad folks out there. And there's a lot of ability to shut down the normal day to day operations of a facility and the people that inhabit that facility, whether it just be general workers doing desk work or in a manufacturing environment or critical infrastructure. So there's a lot of elements that come to it. So from a cyber protection standpoint, that's what we're looking at. Those things within a facility or structure that enable the structure to meet its intended design state.
[00:03:41] Speaker A: Essentially any building can be hacked. So what are the typical cybersecurity threats faced by control systems?
[00:03:49] Speaker B: I think first and foremost, if we look at a control systems from a control systems lens, control systems have been designed to be in the environment for a very long period of time. They are not on a refresh life cycle or replacement life cycle on the same glide path as say your laptop or your iPad or your iPhone. You know, those things come out every two years, you update and you get a new one. Right. Because it has the latest, greatest bells and whistles. Control systems have a much longer lifespan, you know, some, some up to 30 years because they're tied to real, you know, real tangible equipment in the field that they, they oversee the, the actual process control of those elements, you know, valve sensors, actuators.
So first and foremost that is the number one threat that they are on legacy computing operating equipment. So old software platforms, old operating systems, code that was not at the time it was developed, did not think of cybersecurity as a forefront thought it's either been bolted on after the fact or it just never has been addressed. So I can't tell you how many facilities that I've been in or plants that I've been in. And you go and you look at the workstation terminal that's, let's just say that's monitoring and controlling a process line or the H Vac.
And the first thing you have to do to log in is just hit enter, no passwords. You know, imagine that it's because it's been an afterthought. So I think that is like one of the number one greatest threats or vulnerabilities facing the control system environment is just its legacy hardware and software.
Another big one is they don't have defined roles and responsibilities to the access of those systems.
So I'm sure many people can relate to having a workstation or a computer or laptop that's managed by your corporate IT people.
And you can't do anything on that workstation, right? You can't add software or add a print driver. You have to call the help desk. And they have to do it because they have the administrative privileges along many process control environments or building operation systems.
That aspect is completely wide open there. It's the single user to do all privileges and it may be multiple users that have all privileges. So they haven't consolidated those down to role based or permissions based activities to do what is needed to perform their job at a minimum level. So those are some big, big, you know, challenges that face the control system environment.
[00:06:45] Speaker A: Yeah. So can you give me a really specific example or a real world incident related to this?
[00:06:52] Speaker B: Yeah, I think you're seeing it a lot in today's environment. And some of it is not necessarily the control system itself is the vulnerable piece. It's the software that enables the control system or how that works. Workstation or piece of equipment is exposed to the Internet here. Recently, in past, like couple of years, I want to say two or three years, there was a massive ransomware attack on a wastewater treatment facility in the Hillsborough, Tampa, Florida area. And essentially, you know, without getting to many specific details, it was because that workstation was on the Internet and it was accessible through the Internet, so there was no protection. Right. So phishing emails occur and somebody clicks the wrong link and the next thing you know, you've, you know, you've promulgated a ransomware attack.
So that, that is a challenge and understanding of like if you're on the Internet, you are, you are exposed to a lot of threats that are out there, no different than your home computer. Right.
So, you know, that's a real world example of where operations at a wastewater treatment facility were down for a long period of time due to ransomware.
Now of course, you know, not sky isn't falling because control systems are designed to, with sound mechanical and electrical principles where you can have the equipment can be manually operated, so to speak. It's a good way to speak to it in layman's terms so the process can continue. But it's not efficient. Right. That's why we have automation, is to gain efficiency and have precise control over an environment.
Another great example was the Colonial pipeline attack.
That was a oil pipeline that was under attack a few years back also another ransomware attack.
So it's critical that these systems are protected and isolated and not wide open to a public Internet, you know, exposure.
[00:09:09] Speaker A: Yeah, I remember those particular incidents.
[00:09:12] Speaker B: Oh yeah, they made major news. I mean the Department of Homeland Security was involved.
It's a big deal. And municipalities don't have hundreds of millions of dollars or tens of millions of dollars or a million of dollars to go and pay these ransomware attacks, you know, restore their system. So that's another thing that has to be in consideration. When, when, when facilities or plant operators or real property owners, when they have their facilities, how are we going to design a network to accommodate our systems and reduce the exposure threat by placing them not wide open on the Internet? There's opportunity cost there because you don't have the ability to access a little bit, you know, easily access it from outside of the environment.
But what's that risk? You know, risk, reward, trade off. And those are things that facility owners and plant operators, they have to take that into consideration. And as a consulting engineer for design, those are things that we have to really sit down with our clients and stakeholders and really explains those trade offs. And what can we do that manages our risk but also enables the facility operate efficiently?
[00:10:29] Speaker A: Well, Daniel, that sets me up for my next question. What regulations or standards are there for securing these control systems?
[00:10:37] Speaker B: Yeah, there's so, so many different frameworks and cybersecurity professionals, I mean, everyone operates on different things. I know I do a lot of work for the Department of Defense and in my former lifetime prior to joining Dewberry within the Corps of Engineers, we have unified facility criteria of the Department of Defense.
So building code, and that is one framework that we use to put in from a specification standpoint and design specification is leveraging unified facility criteria, developing project specifications that are aligned to the NIST cybersecurity framework. So that's one element that the Department of Defense is using to incorporate cybersecurity code or specification into their building projects.
If you look out into the commercial world, there's a lot of other frameworks.
There's the ISA, the International Society of Automations, IEC 602443 standard, and of course NIST with their special publications 853 and 882, which, which is really aligned to protecting industrial control systems. So those standards are, you know, readily available. They're frameworks, they have controls, security controls, not necessarily controls as for automation, but they're tasks that tell you the system must employ a password protection mechanism. Okay, great. So now I've got a requirement and now what am I gonna do on that control system to make sure that those things are enabled? So those frameworks, they have a lot of different steps and tasks that delineate requirements that can put a system in a secure state.
So the reason I'm very intentional with saying in a secure state is because not everything is 100% risk, avoidable right. You can't have a 100% secure system. There's always trade offs. They have to go in setting a risk posture. But those frameworks, if used correctly and incorporated into project specifications and you know, and as consultants sit down with project owners, you can come up with a good strategy that can reduce the risk exposure exponentially than just field deploying it without taking any considerations. So those are some of the things that can be done using standards, building code to address cybersecurity for the control system environment.
[00:13:17] Speaker A: Okay, so stepping back then, how can an organization or a building owner assess risks associated with their system?
[00:13:26] Speaker B: That's, that's really, that's really cool too because not only do those frameworks tell you how to lock it down, they tell you what you should do. They give you guiding principles that you can use through the NIST has assessment protocols, IEC62443 standards have assessment protocols. And there's a lot of other open source resources that are out there that organizations can use and have folks, whether that be internal organic personnel within the company or facility management facilities engineering that they can use to assess their own, or there's a lot of consultants that are out there that can take those frameworks and come and partner with an organization and go through and target and kind of do a gap risk analysis, what is your risk exposure and start highlighting those things so that they can build a punch list. Some of it could be very low hanging fruit. Like let's just put passwords on our workstation, let's do role based access control on our workstations. Hey, let's not expose our control system it side to the Internet. Right? Let's segment that off into its own dedicated network or dedicated infrastructure.
But then there's other things that are much more capital intensive.
So by doing that gap and risk analysis and finding out what your posture is can help you develop capital investment plans to get after it down the road. And this is one thing I think all facility owners need to recognize is this isn't a one shot, one kill type, you know, event. This, this goes on in perpetuity. Right?
You're only as secure as the day that you employed security. The next day, if you don't maintain it, you become weaker and weaker over time. So it's, it has to become ingrained into the business processes of an organization to ensure that those systems are protected. And it takes money and it takes resources which are many, oftentimes very thin because these are new requirements for most. So it's, it's a Delicate balance between the people that own and operate the equipment and their, and their budget. Right. So there are a lot of very, very practical things that can be done that are very low cost, that can limit risk exposure.
[00:15:45] Speaker A: Then what factors should be considered when you're selecting a vendor? I mean obviously the cfo, the chief financial officer comes into play, like you just said, but what else it's really.
[00:15:56] Speaker B: Important for those that are going to have a consultant come in and do an assessment is there's a lot of firms that have a lot of experience in the traditional information technology world. Like the things that you do to protect your corporate network, right. Your day to day business, laptops and email servers and the such, those things are important and that is very good, sound technical knowledge. But employing that to systems that manipulate or control physical equipment, valves, actuators, gates, dampers, those types of things take a special skill set because you can't employ the same IT cyber strategy to the control system environment because those pieces of equipment are not necessarily as robust as your laptop. And the mechanisms and means that you use to protect a corporate network for IT and day to day business is not the same. So if I'm looking for a vendor that's going to come out, a consultant that's going to come out and do that assessment, I want to look in their portfolio and I want to see their experience of working in that type of environment, working with facility engineers and not just it, pure specific folks, because many times, often not there is a, it's almost like speaking two dialects of the same language. Right. They don't all speak the same. And so those, those personnel have different goals, objectives and they have different missions. Right. In the IT world we're really concerned about protecting the confidentiality of data, our data, our intellectual property. In a control system world, we're worried about safety and availability of the system.
So the data is important, but just not as important as the availability and the data integrity. Right.
So those folks, if I'm looking at IT and I want to have a consulting engineer that's going to come out and do an assessment, they've got to have a lot of experience in the control systems environment and in applying cyber to control systems, because it's not one in the same.
[00:18:11] Speaker A: Okay, well then looking ahead, how can organizations stay ahead of these evolving threats?
[00:18:17] Speaker B: I think it's really incumbent for the facility engineering community to have those hard conversations with their, their cio, their CFO and sit down and come up with a game plan because you can't get after the future threat Because a, you may, you don't really know what the future threat's going to be.
But you can't just flip the switch automatically and make this happen. Right. It really does take corporate leadership, corporate, corporate buy in communication of each other's mission, requirements and responsibilities to come up with a very secure and sound game plan to address control systems. The budgets don't stay the same. So what used to cost me X is now going to cost me Y because I'm now folding cyber security into my business processes.
The other thing is don't overload your technicians and try to make them cybersecurity professionals. There's a lot of collateral duties that you know, mechanics, technicians that manage and operate the plant, that they are responsible for.
Cybersecurity should be shared amongst them, but not their primary duty. Bring in cybersecurity expertise that can augment your mechanical and electrical engineers and your technicians and your, and your mechanics within the plant or the facility to work together. It can't be done in a silo, right, because there's network requirements, there's the actual physical equipment requirements. Then you've got this third element of cyber and they all have to harmonize together because if you over cybersecure something then it's not working.
You've created your own self inflicted denial of service because it's so secure it can't function in its intended design state.
So those trade offs between those three disciplines and or three areas of responsibility are paramount to make sure that you have a full born organizational structure, a funding stream and goals and objectives that you want to get shored up before you can even thinking about addressing future threats. You've got to kind of take care of the foundation, so to speak.
So that, that would be one thing that I would, I would highly advise. The second thing is know what you have.
If you don't know what you have from an asset perspective, how do you know what your risk is and how do you know how to prepare to defend what you don't know you don't have.
So a full asset inventory, knowing what operating systems your equipment is on, what firmware you're going to controllers actuators are at software version. All of those things are important in looking at from an asset inventory because that's what's going to lay the foundation and the blueprint for what you need to defend and what your risk exposure is. So those are the two big big elements. I think I would advise any organization who's not got a mature cybersecurity program.
[00:21:20] Speaker A: For their control systems well, Daniel, I can hear the dedication to this field in your voice.
How did you get into this? What made you pursue it?
[00:21:31] Speaker B: Well, it's kind of interesting. It all started during my time when I was in active duty in the United States Navy. I was on board a nuclear submarine. So I did almost 8 years total service time in the Navy and aboard the submarine.
It's a very complex environment, right? We make our own water, we make our own oxygen.
We submerge a ship, it stays underwater for long periods of time.
And so it's like a city under underwater. And everything on that submarine has some form of control. A reactor, Right?
So we have a nuclear reactor on board. That's all has some form of automation. And so that just kind of sparked, engaged my. My passion for the built environment above ground because I got to see it firsthand in a very compartmentalized and small space. How, you know, things are operated and controlled and, you know, and then you apply that cyber aspect to it is to me, that's what just really sparked my interest is the cyberpiece to the real world, mechanical, operational, type of equipment. And seeing it, knowing how those things work, it just led me to want to learn more about, well, how do we protect it, how do we secure it, how do we make sure that these things are integrated and we can see real time data and not just in stove pipes, but in a wide spectrum, multi, you know, multi integrated system environment.
And so that just really intrigued me, and so I just carried it on, and here I am today. And it's been a great, you know, been a great ride so far. And I've been able to be in the field, see buildings being constructed, complex problems that take, you know, really, really dedicated and out of the box thinking to bring solutions to our clients, whether it be in the DoD or in our commercial sector, from the firm, or even when I worked as a, you know, civil. Civil servant in the Department of Defense with the Corps of Engineers, it was just really, really a fun place to be. And it was always something new because it doesn't get stagnant because there's always threats coming down, different technology rolling out. And so you're constantly having to be learning and staying engaged. That's. That's one thing that I just love about, you know, the career field and kind of what sparked me to get there.
[00:23:59] Speaker A: Cool. Well, thank you for your service, Daniel.
[00:24:01] Speaker B: I really appreciate it. Thank you. Amara.
[00:24:04] Speaker A: Yeah. Any last advice or parting thoughts there?
[00:24:07] Speaker B: The one big thing that I hope that everyone can take away from this is it's Never too late to start. Right. And then your control systems, you know. Yeah, just. Yeah, we're just controlling, you know, the, the environment from, from the, the temperature in the building. Right. And that may not, that may seem benign to folks, but it has a lot of, you know, secondary and tertiary type flow downs. Right. How much energy are we using to condition that building? How can we be efficient with it? What would happen if, if we cut off the AC and our employees were, were, were not happy and they didn't have good morale and they weren't productive. So all these things feed each other and it just seems like it's just small mundane things as you walk into a facility and you don't really think about it. But behind the scenes there's a lot of people working to keep that building operational or keep that plan plant moving and producing products.
And just imagine supply chain risk associated with a cyber breach that shut down production of something or you know, we weren't able to move fluids out of facility or keep the building cold or keep it warm in the winter.
How that just impacts people in their day to day lives. So I think it's just paramount for corporations, organizations and the facilities world to take this into consideration and don't continue to promulgate bad design practices. Start to think about how to incorporate this into the delivery of your designs. Working in cross functional disciplines to come up with complete, secure and usable facilities that are reliable and resilient. It's paramount in today's technologically advanced world and it's only getting more and more robust. So get ahead of the curve and rethink about how you're, how you're doing and what your posture is. Get someone to come and do a gap assessment against your facilities and you'll be amazed at what you find. Things that you never even considered and that are important to your operations. So that's, that would be my leading thought for folks that are listening.
[00:26:16] Speaker A: Very good. Well, thank you for that big picture view, Daniel.
[00:26:19] Speaker B: You're very welcome. And I appreciate the opportunity to speak today and hopefully this was beneficial for those that were listening.
[00:26:26] Speaker A: Yes. And that was Dewberry's Daniel shepherd on cybersecurity issues for consulting engineers to consider.
And for more information on building automation, controls and cybersecurity, visit Consulting specifying
[email protected].
thanks for listening and catch you next.
