Topics:
Samir Boussarhane, senior cybersecurity engineer at MITRE, joins the Nexus Podcast to discuss new simulator plug-ins added to Caldera for OT. Caldera for OT is an open-source adversary emulation platform that automates security assessments for operational technology (OT) systems.
Samir provides context on a new simulator called the Aloha Water Treatment plant, which emulates a water utility and serves as a training platform for students, engineers, and IT security teams alike. Caldera for OT now also supports protocols such as BACnet, Modbus, and includes an HVAC simulator.
Access the Aloha Water Treatment simulator.
Medium article on the Aloha Water Treatment simulator.
00:01.85
Michael Mimoso
all right, welcome back to the Nexus podcast. Samir Boussarhane of MITRE is my guest. And we're going to talk about Caldera for OT. Caldera for OT is an open source adversary emulation platform designed to test security defenses in OT environments. There have been some recent updates for the platform, and that's what we're going to spend some time talking about. um Before we jump into the episode, though, I hope you've been enjoying the great guests we've had on.
00:29.78
Michael Mimoso
Since the start of the year, um the best way to keep up with the show is really just to subscribe naturally. And we are on every major platform, whether it's Apple, Spotify, Audible, Amazon, you name it.
00:40.98
Michael Mimoso
ah Pretty easy to find out there. So if you have not subscribed yet, please do. It really helps out the show. um So let's get started. Bring in Samir. are you doing, man? Good to see you.
00:51.86
Samir Boussarhane
Doing good and happy to be here.
00:53.66
Michael Mimoso
Thank you. Yeah. Thanks for for taking the time. I appreciate it. um Before we jump in, kind of maybe introduce yourself a little bit to the listeners and maybe talk about your role with the within MITRE and Caldera for OT specifically.
01:07.06
Samir Boussarhane
Sure. I'm Sam Booster-Hahn. I've been with MITRE for about three or four years, supporting the CADERRA team and lot of OT research that we're developing. ah Some part of my role is for the simulators we're talking about today, lead development for that.
01:24.22
Samir Boussarhane
We have the Aloha water treatment simulator. That's a water treatment plant supporting Modbus and BACnet. And then more recently, we had a HVAC simulator for BACnet building automation controls that came out and involved with leading research at the University of Hawaii to develop those.
01:42.16
Michael Mimoso
It's pretty cool stuff. I mean, these there's been a lot of talk. I know I was at the S4 conference and I know that there were a lot of discussion discussions about these attack emulation platforms and how they're really gaining some prominence among some of the OT folks. So pretty yeah pretty timely research and and projects on your end. I'm sure you're hearing the same kind of feedback.
02:05.20
Samir Boussarhane
Yeah, and especially with the recent Sysa directory targeting OT devices feels very timely.
02:12.09
Michael Mimoso
Yeah, for sure. um Before we get into kind of the new simulator and and some of those updates, maybe you could cover just exactly for some listeners that might not be familiar, explain exactly what Caldera for OT is, who uses it, how they use it.
02:27.48
Michael Mimoso
Just take me in from a high level what what it's all about.
02:30.87
Samir Boussarhane
Sure, so Caldera for OT is a plugin and extension for Caldera. Caldera is like an open source adversary relation platform that MITRE helps develop. With that, anyone can download this off the GitHub and you set up your server, you have agents and you can deploy attacks, the payloads, the adversary abilities, the TTPs, map to attack are all there, available open source, and you can customize, you can add your own abilities, open source, and we'd love to see if we're getting anybody to contribute and commit back to MITRE.
03:00.09
Samir Boussarhane
The OT and plugin has for like, when you're looking at Caldera for Caldera in general, it's more IT based OT. We're extending that to have your protocol specific payloads, your adversary and relation capabilities.
03:11.96
Samir Boussarhane
So there's plugins right now for BACnet, Modbus, DMP3, Profinet. And all of those are very, very specific as opposed to IT t where we have a lot more coverage.
03:22.52
Samir Boussarhane
So we're trying to make it as easy as possible. If you want to run adversary capabilities for defenders, red teamers, purple teamers, you can just download these payloads, execute them against your OT infrastructure.
03:33.84
Samir Boussarhane
If you don't have OT infrastructure, we're trying to make these simulators so you can still test your detections and learn adversary like tradecraft and OT space.
03:42.55
Michael Mimoso
And so what are the inputs, so to speak, or ah or the threat intel sources? Are you guys specifically sticking to MITRE ATT&CK for ICS, for example, or um ah what are some of the other input sources?
03:55.64
Samir Boussarhane
Right. So for the OT space, ah the Intel doesn't always necessarily get to a specific like, hey, adversary X ran this TTP or this protocol function.
04:06.90
Samir Boussarhane
So we try to have as much coverage as we can within the protocol and then map to attack for ICS as far as the impact. So for like an OT process, we want be able to read and write to affect the process.
04:17.53
Samir Boussarhane
We can map that to attack and make sure for our abilities that you're able to modify memory on the OT device. And then from there, you can say, all right, if I was an adversary, I'd want change this to turn on this valve.
04:30.11
Samir Boussarhane
That's more of the like, thread informed based on how you can impact, not necessarily like a one for one command line that here's what the adversary ran. Just make sure we have the ability to perform data manipulations or read the vice configurations within the protocol.
04:46.02
Michael Mimoso
And what's a typical, or who is a typical user for for this platform?
04:53.59
Samir Boussarhane
Really anyone. So one great application with Caldera that I've been doing University Hawaii is education and training. I didn't personally learn any OT when I was in undergrad or graduate school, and it's not necessarily covered in all curriculums.
05:08.67
Samir Boussarhane
So being able to put those in payloads, put them in a pretty package, put them in Python scripts that just download. have students able to click a button, see their adversary emulation happen, see the outputs.
05:20.68
Samir Boussarhane
It's great for training and exercise. Like if you're trying to do a large scale exercise and automate this, Kether is great tool for that.
05:30.04
Michael Mimoso
So it's not necessarily for a production environment. This is kind of an educational tool or just a, um just kind of a training mechanism, right?
05:40.99
Samir Boussarhane
The payloads and abilities are modular, so you can take them out in your production environment. It's not necessarily best suited for that, but it's like a server and agent architecture, so you could deploy it wherever you want.
05:54.38
Samir Boussarhane
Very, very useful tool for training and new users of the OT tools.
06:00.00
Michael Mimoso
Yeah. Have you heard of any unique use cases that maybe you guys hadn't considered or?
06:08.99
Samir Boussarhane
I have not yet, but simulator's new, and if anyone has any unique use cases, happy to learn about them.
06:14.01
Michael Mimoso
Yeah. And is this the only platform, ah one of these attack emulation platforms that's specific for OT that you're aware of or are there others?
06:26.53
Samir Boussarhane
As far as free and open source, this is the only one I'm currently aware of. I know there are other out there that aren't necessarily those.
06:35.35
Michael Mimoso
And what what are the kind of the drivers, the need for this, for such a platform? Is it, um you know, like I know there's always a concern about getting your hands on actual PLCs to do testing. You don't want to touch your production environment.
06:51.99
Michael Mimoso
um Does that alleviate that concern? What are some of the drivers for this?
06:56.57
Samir Boussarhane
Right, that was the main driver with the simulators that we developed. We were going out to exercise and show our new OT capabilities where we had plugins like, all right, here's some updates to Modbus plugins so you can run this against your Modbus PLCs, updates to BACnet plugins so you can and run these against your BACnet PLCs.
07:14.65
Samir Boussarhane
As we started doing more workshops, we realized that not necessarily everyone had PLCs and those can be very expensive.
07:15.95
Michael Mimoso
Thank you.
07:20.62
Samir Boussarhane
If you ever go on eBay, you can see them go from hundreds of dollars to thousands of dollars very quickly. So we wanted to make those simulators to where now we can lower the barrier of entry and into OT security and have free open source Python scripts that will behave as a realistic simulator in like PLC.
07:39.34
Samir Boussarhane
It's not going to get into those vendor specific functions that PLCs would have, depending on the vendor, but general reading and writing and memory operations will get to be able to cover that.
07:50.57
Samir Boussarhane
And we're able to see that go across networks so we can build better detections.
07:55.48
Michael Mimoso
So as the users running this, what are what are they seeing? I guess what are the outputs that they would expect to see? Are they seeing and like a simulated environment and values changing or what's the what's the output that they see?
08:08.75
Samir Boussarhane
Yeah, so for Aloha Water Treatment Plant and HVACSIM, there's Python scripts you run and they're just servers. So with the servers, there is like a graphical interface where you can change the values.
08:19.55
Samir Boussarhane
So for Aloha, you can change the inflow rate, the outflow rate, and then in the visualization, you see how the process is affected. ah You can use those as an endpoint for any adversary like tool you have that's OT-specific.
08:31.62
Samir Boussarhane
So I could take my category for OT. I could take a custom-made Modbus client and perform writes against that. And then I will see on the process how that changes.
08:41.96
Michael Mimoso
Mm-hmm.
08:42.94
Samir Boussarhane
So really so like serving as an endpoint to where now I can see just using OT-specific functions, protocol, communication, how my processes changed. And then if you have like Wireshark in the background or whatever you want to use for network traffic analysis, we can see how these different protocol commands are sent across network, how responses are done, how discovery functions are working.
09:08.66
Samir Boussarhane
Instead of just having adversary emulation platform send out these commands and their responses, now we can see like more realistic behaviors.
09:17.14
Michael Mimoso
And can users feed like their own samples of their own traffic to the the platform or is it kind of generic?
09:27.35
Samir Boussarhane
Yeah, so when we say traffic, um we're not actually like recording traffic. It's more of a the server sends its endpoint it'll take any responses. So you could just throw with random responses at it and see how that breaks it.
09:41.88
Samir Boussarhane
But it's just like giving realistic responses back to it based on protocol specification. So any user could take any like client they want and just attack it and test. But we're not in Caldera itself inputting traffic. It's more of a the simulators that respond to anything it gets.
09:58.14
Michael Mimoso
Got it. Okay. right. So let's talk about the Aloha water treatment plant. um Basically, what was kind of the the driver for this one? It mimics a ah water storage tank, correct?
10:09.58
Samir Boussarhane
It's a water treatment plant where very simple. It has your intake for your water, goes through a filter and a center and then outtake. Then you can manipulate those two.
10:20.50
Samir Boussarhane
Then there's an auto mode that tries to keep it around two thirds full, manual mode where you can, wherever you set the inflow and outtake, it overhauls that for the physics. ah That was really just designed as a effort when University of Hawaii but were giving OT lectures and It's very easy to explain like, hey, here's how this works, or the PLC right in front of you.
10:41.46
Samir Boussarhane
But if you want to go home and learn more about OT security and you don't have one of those expensive PLCs, that was a big barrier that we were getting responses saying like, oh, this is really cool, but I don't have the hundreds of dollars spend on the PLC randomly.
10:53.77
Samir Boussarhane
So that's what we developed that for. And then students were able to learn at home and then see the nice like GUI visualization of how their different commands were affecting process.
11:07.92
Michael Mimoso
And so as it as it as it runs, as it works, what can it tell a defender? I mean, what does the platform tell users kind of like about different protocol activity, vulnerabilities, dependencies, et cetera?
11:25.12
Samir Boussarhane
Right. So on the platform itself, the visualization will warn you if you're getting to what the process defines as a like critical or warning system. So if the emergency stop activates, the HMI representation will flash a warning.
11:40.66
Samir Boussarhane
Or if you're about to overflow, it'll warn for that or underflow. As far as like network traffic, um it's not flagging it siit f suspicious traffic.
11:53.17
Samir Boussarhane
but One area that we'd like to look to like recommend users look into is have Wireshark open while you're doing this so you can recognize these like packets. So if we go to like the attack for ICS, we know that there are functions and protocols that we want to be more aware of for our defenders. like We want to know any impact process, reads and writes.
12:14.17
Samir Boussarhane
We want to make sure that we're able to see that across network. So go through the cutter for OT, look at how the abilities are mapped to attack for ICS, and you have your high priority abilities that you want to be able to defend against.
12:27.75
Samir Boussarhane
Look at that traffic, and then from there you can learn how you detect that going across network.
12:33.56
Michael Mimoso
Have you heard from folks in, you know, water utilities, wherever they may be, are they using it or is it strictly from students right now?
12:43.03
Samir Boussarhane
I have not. It's been mostly students and people trying to get into OT space right now.
12:49.37
Michael Mimoso
And what do you think are the the biggest benefits of of this kind of platform for, for a student, for example, or from anyone trying to get into OT? Like what does it help visualize or what does it help make clear? I guess.
13:02.37
Samir Boussarhane
Yeah, like ah at least my experience when i was starting to about OT, you felt like very complicated and hard to wrap your head around, especially you're familiar, like ladder logic or these specific protocols that are all can vary vastly.
13:15.70
Samir Boussarhane
Having all that open source, free and well-documented resources that MITRE is producing, I think is a great learning point. ah Like every ability that we have, we document like the inputs for the payload, the outputs for the payload, how this affects protocol, what the,
13:32.54
Samir Boussarhane
resources and like actual specification in the protocol would be. It's a great resource and well-documented way to get into OT. And then the visualization makes it very easy to just run some Python scripts and actually see, OK, so I'm doing these reads and writes against memory.
13:49.75
Samir Boussarhane
Here's how it would actually affect the realistic process.
13:55.19
Michael Mimoso
So you have different simulators ah with, i'm or I'm sorry, you have the same simulator, but with a Modbus and a BACnet plugin. Is that correct?
14:04.93
Samir Boussarhane
Right, so for Aloha, it's the same backend. We just wrote it in two different libraries, PyModbus open source Modbus library, back zero and open source Bacnet library.
14:15.64
Samir Boussarhane
Same process, but you can run it in these different versions to have it respond against those different protocols so you can do your testing. Then there's Bacnet sim, which is more recent, it's like a fan system that's in BACNAP.
14:29.17
Samir Boussarhane
And same thing where you have different process now, like going from water treatment plants to building automation control systems to where you can just run it on Python scripts, run your adversary behaviors against it and get realistic responses back.
14:43.10
Michael Mimoso
And are there different attack scenarios for each of these ah different plugins? I mean what kind of attack scenarios is are is it mimicking or simulating?
14:52.73
Samir Boussarhane
Right, so for we have medium blogs that we released with both of the simulators that go through different attacks scenarios and has graphics on way you should expect it as far as you like walk through it.
15:04.70
Samir Boussarhane
All these hack scenarios really are how can you impact the process? Can you stop the water flow? Can you stop the fans? And what you need to do to do that. So encourage anyone to go through the Medium articles, and we walk through the attack like vectors and what you need to do and what you expect to get out of it.
15:23.83
Michael Mimoso
Yeah, we can link to those in the in the show notes for sure. um Do you see this as useful, ah perhaps as a red teaming exercise or as part of an overall risk assessment, for example, and and why or why not?
15:39.16
Samir Boussarhane
yeah Absolutely. I think, as you mentioned earlier, people get very wary when it comes to testing any sort of red teaming in their productive environment. I think it serves as a very good starting point as a stand-in to where we know it's going to respond to the protocols for our system.
15:55.32
Samir Boussarhane
We know that we need to be able to detect these before they happen. Let's take that over here, build our little sandbox environment, and do our red teaming. and then ask whether are we'd be able to protect this in our actual infrastructure.
16:09.46
Samir Boussarhane
I think it's a good stand-in to try to avoid all those frictions that comes with trying to red team production.
16:15.22
Michael Mimoso
Yeah. um and And just a question about the inclusion of BACnet. Obviously, it's a building management system protocol, pretty well known and well, pretty popular. What what drove you towards including that as a plugin?
16:29.75
Michael Mimoso
um I know we're hearing about a lot of BMS being connected online. Is that kind of the main driver for that or something else?
16:38.52
Samir Boussarhane
That and a few years ago, the large target hack had happened through the building automation system. So we felt that was like a relevant protocol to make sure and we discussed.
16:48.59
Michael Mimoso
Mm-hmm.
16:48.62
Samir Boussarhane
BACnet also has a lot of interesting discovery capabilities that I think are good for new learners to learn. So for one is the whois command where through BACnet device, you run the whois message.
17:01.21
Samir Boussarhane
Any BACnet device on the network responds. So very easy to in the network, recon, figure out where your BACnet devices are. Then you can use the Caldera for OTability Epic Support, point it at those devices, and they'll give you back all their information, all their objects and memory.
17:17.56
Samir Boussarhane
So it's very easy to go as far as like an adversary emulation plan and storyline in BACnet. And it's one of the more beginner-friendly OT protocols, so want to make sure we include coverage for that.
17:30.12
Michael Mimoso
And how would you characterize the overall security of BACnet as a protocol?
17:37.91
Samir Boussarhane
um Difficult question since it really depends on the implementation. Some people have backnet that uses encryption and authentication. um I would say that's not as common as it should be.
17:54.14
Samir Boussarhane
But as far as protocol, it's very easy to learn more information and do your discovery against endpoints. Modbus, the other protocol that Aloha is written in, you kind of have to know more about the design of the system that you're reading against just because it doesn't give you as much information.
18:11.65
Samir Boussarhane
BACnet's great from an adversary standpoint, but difficult for defenders since you're able to give up so much information.
18:18.05
Michael Mimoso
Yeah. And I mean, Modbus isn't necessarily secure by default either. I mean, there are add-ons for security for for that protocol as well, right?
18:27.68
Samir Boussarhane
Right.
18:30.46
Michael Mimoso
and a you And again, in terms of, did this, the inclusion of the BACnet plugin here, did that set the stage for the HVAC Tell me about a little bit about that simulator.
18:44.32
Samir Boussarhane
Right, so with that, the Modbus and Bacnet plugins were pretty much our very much are most well-documented plugins. So we wanted to make sure to include for that.
18:56.29
Samir Boussarhane
And our workshops, GDUH, that's the two protocols we were really briefing on. And then water treatment's not necessarily a Bacnet-specific process. wanted to make something more realistic as far as like building automations.
19:08.38
Samir Boussarhane
And that simulator was actually developed by a student group at University of Hawaii at Manoa. So for their semester research project, they developed that simulator, tested it, and we were able to release it.
19:20.73
Samir Boussarhane
So the students did a great job with that. And we can show that everyone in that group actually had no background in OT at all. And semester, we were able to develop something new, test it, and have a really good product come out.
19:34.03
Michael Mimoso
And just for Caldera for OT in general, is there a best practice in terms of, you know, how often do I run this daily, quarterly, continuously? I mean, what, is there a recommendation or what do you find most useful?
19:52.15
Samir Boussarhane
yeah So as it stands right now, it's not necessarily a like run this every day to elicit like a detection. It's more of a here's how we can exercise. Here's how we can train. Here's how we can test easily.
20:06.54
Samir Boussarhane
um i wouldn't recommend running like reads and writes the introduction equipment every day or anything like that. But if you're wanting to sandbox detections and ping on how often you want to do that, I think it's a great tool for that.
20:12.61
Michael Mimoso
Right.
20:21.72
Michael Mimoso
And do you hear anything about something like this being used for um i t people that are kind of new to OT or now have to manage and and secure OT?
20:34.94
Michael Mimoso
um Have you heard anything anecdotally like that, that people on the IT t side are kind of ah you know, moving over to check out these kinds of platforms and see what they can do with them.
20:46.34
Samir Boussarhane
Yeah, we have plenty of stories like that where we have different organizations that exercise their IT t red teaming. And now with more recent news and CTI coming out about OT systems being a critical thing that we need to protect, defenders and attackers looking for those OT capabilities and happy to point them towards that for OT.
21:09.04
Samir Boussarhane
But definitely over the years, there's been more of a shift towards critical infrastructure and OTICS, SCADA, and wanting to protect that.
21:15.00
Michael Mimoso
and then
21:18.10
Michael Mimoso
And so I did want to, before we wrap up, talk a little bit about the, the attacks that have been, um connected and linked to Iran against the Rockwell PLCs.
21:30.42
Michael Mimoso
um How much do you see something like that elevating ah or prioritizing the security of these platforms? It's not often you see real world attacks targeting PLCs, for example, um and these vulnerabilities are pretty old. like Just ah curious as to your reaction when you heard that news.
21:52.13
Samir Boussarhane
Yeah, definitely interesting. um I'm not up to on the exact specifics, but I'm tracking that it was a certain model of Allen Bradley PLCs that had a vulnerability, and they were searching for that model.
22:02.97
Samir Boussarhane
Am I correct on that?
22:04.50
Michael Mimoso
Yes, correct. Yeah. I think it's pretty old vulnerability. I think it was 2021. um And it was pretty easy to access um the PLCs and and manipulate from there.
22:20.92
Samir Boussarhane
Yeah, and then ah kind alluded to earlier. So with the OT simulators and Calder for OT plugins, we're using the protocols that are common with the protocol specification, like Modbus to be a compliant Modbus device to support Modbus reads, Modbus writes.
22:38.10
Samir Boussarhane
A lot of those adversarial behaviors that like we're talking about with Iran and the microwave 50s, that's even outside of those functions. So it's very important to when we're trying to protect OT to think of what's already built in a native, but also like what an adversary could do through the vendor space. Because that's not nearly as well documented.
22:57.78
Samir Boussarhane
And we really want to be able to collect traffic that's going towards OT systems at all. And I know in IT t that we say, all right, if you want me to close the firewall, need to know everything because so much traffic goes through.
23:11.45
Samir Boussarhane
OT space, it really should be the opposite. So we have to be better as defenders to flag any c specific traffic in like a OT space to defend against it.
23:22.26
Samir Boussarhane
But as far as like, sorry, go ahead.
23:22.62
Michael Mimoso
And then they're... Go ahead. No, go ahead. Finish your thought, please.
23:26.68
Samir Boussarhane
Yeah, as far as like the Iran vulnerabilities, I think it's great to bring attention to the critical infrastructure may not be as well protected as we think and want to make sure that we're flagging these issues before anything bad could happen.
23:39.35
Michael Mimoso
Yeah. I mean, the whole vulnerability and exposure management question within OT is just... it it it doesn't seem like it's ever going to go away in terms of patching, in terms of, I mean, I don't think this particular vulnerability can be fixed without a major overhaul from the vendor regardless, but um there's there's long windows of exposures with some of these vulnerabilities, unfortunately.
24:00.95
Samir Boussarhane
Right.
24:03.10
Michael Mimoso
um All right, so as so we wrap up, um just what is next for Caldera for OT? And are you expecting more of these plugins or more protocol support? Tell me what's kind of on the drawing board.
24:16.44
Samir Boussarhane
Yeah, so with that ah University Hawaii capstone group that produced HVACSIM last semester, we're also sponsoring another team this year to do DMP3 for electric substation.
24:29.02
Samir Boussarhane
That development's underway and hoping to release that later this year. Let's see. More updates, continued updates, and like improving our documentation and our protocols.
24:41.31
Samir Boussarhane
Be able to announce more of that later in the year, but still trying to improve security posture and lower that barrier of entry.
24:46.95
Michael Mimoso
Yeah. It's very cool stuff. It must be fun to be on kind of the leading edge of this kind of development.
24:53.24
Samir Boussarhane
Yeah, definitely fun research to be part of.
24:55.54
Michael Mimoso
Yeah. All right, Samir. Thank you so much for joining the podcast. I really appreciate it.
25:00.28
Samir Boussarhane
thank you for having me.
25:01.53
Michael Mimoso
All right, man. Take care.
Michael Mimoso is Director of Influencer Marketing at Claroty and Editorial Director of Nexus.
