Ep65 – Dell and Secure Supply Chain
[00:00:40] Tom Garrison: Hi, I’m Tom Garrison. And welcome to the cybersecurity inside podcast with me today is my co-host Camille. Hi, Camille, how are you today?
[00:00:50] Camille Morhardt: I am doing really well today, Tom. I’m excited because we’re going to be listening to an episode near and dear to our heart on the topic of supply chain.
[00:01:00] Tom Garrison: Yes, we have talked about this topic multiple times and up until now, we’ve, we’ve sort of stopped our conversations on supply chain security at the device sort of when it gets provisioned. And we’re going to talk about the digital supply chain and how important it is to understand the platform as it continues throughout its life, not just at that initial build and provisioning.
[00:01:29] Camille Morhardt:: Yeah. And I think it’s really interesting. We have the conversation with a couple of team members from Dell. So we’re kind of looking at right now with Intel and Dell, the two elements within the supply chain for people who are ultimately getting a compute product.
[00:01:45] Tom Garrison: That’s right. And, and the partnership with Dell, you’ll hear a lot of the closeness in our relationship, which I think is important. You know, it’s important for listeners to hear that, you know, it does take a village when you’re trying to do technology and trying to change the ecosystem. The partnership between Dell and Intel is very critical. It’s very strong. And we also talk about an earlier topic that some of our listeners may remember about transparent supply chain. And in this podcast, we highlight the fact that Dell’s secure component verification is compatible with Intel transparent supply chain. And so it’s one more set of progress that we’ve made together in driving platform, security and supply chain security for platforms.
[00:02:37] Camille Morhardt: Yeah, I think it’s really good. It’s the story behind the product. If you will, we don’t spend time really talking much about the products themselves, but more about, you know, why certain things are being done and why we’re looking at this element of security, um, and how we work together across different layers and elements of the supply chain from hardware to software and from, Uh, chip supply to PC or server supply to actually deliver the right kind of secure product.
[00:03:06] Tom Garrison: That’s right. And Dell takes it to a whole new level too. So they build upon the transparent supply chain element. The conversation goes into how they handle their factories, the people that they have in their secure facilities and so forth. And it’s really interesting and, and fun to see. So what do you say? We jump right into it.
[00:03:25] Camille Morhardt: I would love to.
[00:03:32] Tom Garrison: We have two guests today. Our first is Rick Martinez who serves as a Senior Distinguished Engineer for Client Solutions at Dell Technologies where his responsibilities include security strategy for Dell PCs focused on trusted and resilient platforms. He’s an expert resource in secure development, governance and execution for Dell PCs and pan-Dell secure supply chain efforts.
Our second guest is John Boyle who serves as a Product Manager for Client Solutions at Dell Technologies where his responsibilities include supply chain, below the operating system and endpoint security solutions for Dell PC’s focused on trusted and resilient platforms. So welcome to the podcast, both of you.
[00:004:16] Rick and John: Thanks Tom. Thanks Tom.
[00:04:18] Tom Garrison: Uh, obviously I’ve read your introductions there, but what are your day-to-day responsibilities as it relates to security?
[00:04:24] Rick Martinez: Sure. I can start. So I lead a great team of four strategists and technologists, and we essentially create the security strategy to harden and secure our Dell client PCs. So that’s all of our notebooks, desktops, and workstations. We are a CTO group, so we operate about three to five years out. Um, so it’s kind of interesting for us because when you’re setting strategy from a business perspective, you know, there’s a lot of numbers to crunch and, and crystal balls to read.
Our crystal ball is the adversary. So it’s kinda cool that we get to hang out with hackers and hang out with other security, uh, community and practitioners so that we can really have that three to five-year crystal ball of what the adversaries are going to be doing in that timeframe. And then we basically backcast right. We figure out what the adversaries are going to be doing, what our customers are going to be dealing with in five years and, and we build, uh, our mitigations into our systems.
[00:05:22] Tom Garrison: Great. And how about John?
[00:05:26] John Boyle: So I partner closely with Rick and his team on this overall strategy for security and even manageability. We are looking to take that four to five year north star vision that the CTO team has. We definitely interlock with them there. And on the product side, we focus more on the one to three year vision so that we can pull things in and execute on that. And part of that is partnering with great partners like Intel for our below the OS solutions. We work closely with the CTO to execute on those below the OS device hardening. But the other part of my focus is to make sure that from the source of where we source our parts to the very end of the device, when they’re retired, that our supply chain security is the best digitally and physically, as well as that, the device is adding value to any security ecosystem, a customer might have to support their overall security mission.
[00:06:19] Tom Garrison: You talk about operating in the future–the three to five years sort of horizon. Can you share with us some of your, your thoughts and Dell’s thoughts about where’s supply chain security going what’s important. And from an adversary standpoint, like what should people be thinking about?
[00:06:37] Rick Martinez: Sure. So for a large OEM and large manufacturers, such as Dell, uh, you know, everything that we do is supply chain. These are very large systems that, that we’re shipping to customers like large-scale systems. They have a lot of capabilities; they have a lot of memory; they have a lot of compute power, right? So that also means that they have a pretty good threat area threat surface.
So our context, isn’t just getting the system from our shipping facility to the customer’s loading dock. Right. It’s the entire end-to-end–from when we design a system to the end of life of the system in the customer’s environment, and making sure that, you know, the system is secure throughout that entire life cycle.
[00:07:15] Camille Morhardt: Is that something that’s evolved over the last few years, really looking at it? I used to think supply chain was looked to the left. The right is kind of the end-users deal, whatever they’re going to do with it.
[00:07:27] John Boyle: I would say it’s definitely evolved because, you know, one thing that Rick and I always do in these discussions is we start by saying, you know, “close your eyes.” And we say, “supply chain, what do you envision? You’re probably thinking of? Uh, train a truck, an airplane.” And that’s that physical component of moving product, supply chain that we’re all used to. Right?
But we talk about things like digital supply chain, because the moment that we deliver a product to a customer, what do we do is Intel and Dell? We send updates, we send firmware, we send entitlements, we send licenses and we push new product out digitally. In the life cycle of a, of an Intel solution or a Dell device, we are constantly sending things through the digital supply chain, as well as the physical supply chain and customers want to know that those are authentic parts, that they come from the source as intended, and they have nothing else on those parts or in that code that shouldn’t be there.
And, you know, as, as a large OEM, we own the entire lifecycle. So we make sure we keep a close handle on the security–whether it’s physical, digital personnel, facility, whatever.
[00:08:38] Tom Garrison: Where do you see this stacking up in terms of the priority for customers? I mean, in terms of things like, for example, the performance of the device or the form factor, or, you know, other elements costs, uh, where is, uh, security in general and maybe more specifically supply chain, where is that sort of in the parades of customer wants?
[00:09:03] John Boyle: I can take that one, Rick, first. I think, um, This is a very heightened security environment after the attacks on critical infrastructure worldwide, and the continued attacks that are going on, government responses worldwide. People are very aware that security is, is top of the charts. And so we have seen customer feedback where security has been top. And supply chain is either second or it’s tied there with first. And I will, I will also, I’ll also put in there that sustainability is in there as well. They like the fact that we can be secure and sustainable, as well.
[00:09:36] Tom Garrison: Yeah, no, that’s interesting. So you’re saying security now has, has really taken the top position in terms of people wanting to go out and purchase new PCs as an example?
[00:09:47] John Boyle: We’ve seen that as feedback in certain deals. Yes. So not empirically across the board. Everybody, you know, they do have price, they do have performance, but security where it’s usually in the past, you know, in Rick’s and my professional experience, it might’ve been like a check box. Like yes, security is there. It is a very well vetted. A lot of the leading questions around supply chain and security, we get, uh, in a lot of conversations with customers. And it’s great because it’s nice to see customers really taking that proactive approach with security and supply chain security.
[00:10:17] Camille Morhardt: Well, you guys have had to, especially, um, in the last, I guess, 18 months or a year, really look at, I think it’s a pretty big shift in terms of the enterprise level, the shipping and PCs no longer through an IT Department, but many of them going direct to end users, um, who are within a company and need all kinds of protections that come with a company PC.
So how did you pivot that? Was that something that was on your horizon as a potential use case? Or was that something that you had to quickly adapt to?
[00:10:48] Rick Martinez: I think we were prepared for it, but it really also was able to adjust our threat model assumptions. Right? So, you know, I wouldn’t have a job if we were just sitting still from, from a security perspective, right. So we are constantly evolving, just the hardening of the platform, our supply chain security, pretty much everything that we do around manufacturing and developing and shipping systems to users.
From a supply chain and, and shipping systems to end users, we actually have a, you know, some technology that was part of the industry standards, uh, and then something that we capitalized on and, and productized around, being able to securely verify the components when you receive a system based on a digitally signed manifest that’s created in our factory. So I think that’s, that’s something that we already had in flight pre-COVID, but it worked out very well with, again, with these new kind of threat model assumptions that we’re dealing with in this post-COVID world.
[00:11:43] Tom Garrison: Right. And what do you call that?
[00:11:45] Rick Martinez: That product is Secured Component Verification.
[00:11:48] John Boyle: Yeah. And the interesting thing there is that we’ve always had, um, our secure BIOS verification in our supply chain. We will compare the golden hashes against, you know, the BIOS on a box in different steps of the manufacturing process. And so if it goes from one facility to another for customization, for instance, we’ll flash the BIOS, uh, from that facility, then it ships to the other facility and we’ll do the same thing. And so we’re checking the security posture of the devices in the supply chain before we send it to customers with the tools that customers can use on their end to do this exact same thing.
Secure Component Verification takes it to the next level where we are now verifying the physical components on the box to make sure that again, they are Dell components or, you know, in the case and partnering with Intel that they’re Intel components. So that customers get exactly what we shipped them. On the other side of the coin is that when we get things sent back to us, we’ll say as a PC, as a service type of model that we get back, the components that we sent to customers.
[00:12:49] Tom Garrison: Yeah, exactly. Yeah. You ship up a Cadillac and you get back a Yugo.
[00:12:54] John Boyle: That’s right. (laughs) So Secure Component Verification is a fantastic tool and it’s an addition to a lot of the other things we’ve done. We have a lot of things that we put into place to disrupt a kill chain from concluding in an attack. And the thing of it is I always joke. I think I’ve said this to you, Tom is that you know the only person I know who woke up in the morning or got home after school and used the modem to dial into the federal government computer and bring it down was Matthew Broderick in War Games. Other than that usually takes like a lot of steps to get that to happen. And that’s called the kill chain.
And so we’re trying to do is whether it’s the physical type of tamper evidence we have or different things we imbue in the platform or Secure Component Verification in all the steps of the supply chain and in the life. Be as disruptive, as possible as a technology partner and security partner to any threats coming our customer’s way and, and being proactive.
[00:13:51] Tom Garrison: Well, I know that on this podcast, we’ve talked several episodes about, uh, transparent supply chain tools, uh, from Intel. And for anyone wondering about the breadth of what you’re doing with Secure Component Verification. I think it’s important for people to know that the standards that they’re based on are the same. So they’re compatible in that sense from the verification element and the platform certificates.
And obviously Dell has built a tremendous amount of other capabilities on top of that. But it’s all, I think that’s the importance of working together is that we have the solution now. That absolutely works together. The tools that we’ve built are very analogous to the tools that you have, uh, with Secure Component Verification.
[00:14:38] Rick Martinez: Yeah, I agree, Tom. It was, it was a great collaboration, not only within TCG, the Trusted Competing Group, but also between Dell and Intel to kind of make sure that all of those things, you know, work together for the benefit of security and our customers.
[00:14:51] Tom Garrison: You know, I, for one, I think back, even all the way back to 2019, which doesn’t sound like that long ago, but actually feels like a lifetime ago. And supply chain just was not on the horizon at all. It just, people didn’t think about it. And if they did, they were thinking about how can I optimize the, my, my supply chain to get just in time delivery and rip out any costs that I could have. That was the context of supply chain. Now, the dialogue has very much turned into supply chain security. And I think that we’re starting to turn the corner for companies to really realize that. But I think there’s so much more that we, as an industry have to accomplish in terms of the breadth of how important secure supply chain actually is.
[00:15:40] John Boyle: You’re right. It has come to the forefront, unfortunately, because of some major security compromises in the supply chain– specifically digitally–as well as just the attacks on critical infrastructure. And one of the things that we always like to call attention to is that yes, there’s revenue and there’s data and all that at stake. But when you have these attackers bringing down environments for healthcare, financial services, energy it is impacting real lives.
And one of the things I sent Rick last week was the article about how a hospital got impacted by ransomware and an impacted their NICU. And I don’t need to go into any details, but I think that we all need to pause and think about that the work we do does impact lives. It’s not just sending computers to people, to run spreadsheets and to surf the web and that sort of thing. It’s that we are impacting businesses that impact real lives; and when those are compromised, it’s not good for people around in the global community.
And so, I think that, that the supply chain is important because it is the device. It is the patches. But think about if we have a cloud deployment about something. Before you push code from a development environment to a production environment, that’s digital supply chain, too. So customers need to really ask their providers or their partners, “Hey, great you want to push a product on me because you say it’s secure. Have you had any security compromises in the past? Who are your partners?” You know, I think, uh, there’s a few names that pop up where you’re like, “well, wait a minute, you know, that’s kind of something to talk about.” And then there’s also the element of “how do you manage your, your development supply chain of your code or your parts or whatever. Do you do pen testing on your own environment?” Cause one thing Rick and I are responsible for is pen testing our platforms. And we do that proactively again so that we can keep on top of anything and everything that makes it the most secure environment for customers to, to provide all of these people around the world with an environment that’s secure.
[00:17:37] Rick Martinez: Right. And, and you kind of talked about, you know, supply chain, not necessarily being on the radar. Again, if we look at a, at a, you know, the way that we do from an adversarial perspective and how do we protect our customers, supply chain is always been on the radar. That’s, that’s part of what we do. So, um, and, and, and certainly with some of the recent events around software updates and, and malware getting into software updates in that path, it’s pretty obvious that that’s an important and critical piece, as well.
[00:18:05] Tom Garrison: Yeah. Relatively speaking, now that there are tools to help prevent these attacks. And I think that’s at least part of what Camille and I, when we started this podcast, our goal was to educate people so that they understand what’s possible and what’s out there. And supply chain was one of the first topics we started hitting on because it’s exactly that. The scale impact to these attacks is enormous and they are preventable. If you work with partners like Dell, like Intel, with the transparent supply chain tools, there are solutions out there that can mitigate these attacks. Rick, you said before that your team operates sort of the three to five years in the future. I mean, when it comes to security, how do you do that? What are some maybe best practices for, for other companies to think about, how do I know where security needs to go in the future?
[00:19:00] Rick Martinez: This is one way to do it. And this is not the only way to do it, but I think being directly involved in the security community, being involved with security researchers, being involved at the ground level with hackers and other really smart people that know how to break systems. And understanding what they do, not only from a very discreet, “Okay. They found a vulnerability, they filed a CVE.” Uh, you know, that’s only part of it, but really understanding what they’re doing from an adversarial perspective and building that into our threat modeling assumptions. Right?
So a few years ago, uh, there was a lot of research around a UEFI and a firmware tampering, right? So, you know, we, we put a lot of investment into our systems to make sure that we can mitigate that at various levels, either physical or digital or, you know, uh, you know, using anti-tamper type technologies. That didn’t materialize on its own. That was from years and years of, you know, following research and making sure that we are ahead of the game.
Now, one of the things about research, you could say, “well, if it’s, if it’s already been published or it’s at, at a security conference, isn’t it already out there?” Well, it is, but it may be the first time that something like that has been presented. So it usually takes a little bit of lag time before, especially with the very, hardware specific or physical type attacks. It usually takes a little while before the actual adversaries to go in and implement those types of attacks. So we do have a little bit of runway there. But being there again at the, at the very beginning, when that stuff is presented, um, when those presenters are chatting with the community and engaging with them and interacting, uh, I think is the, is the best foot forward for anybody.
You know, and, and security conferences, aren’t just for system designers or product security folks, you know, there’s a ton of customers there. We get a great feedback. Customers, you know, learn about these same types of adversaries and attacks. Um, so it all kind of, you know, it works together.
[00:20:53] Tom Garrison: That’s great feedback. I always tell people too, if you want the simplest highest return action you can possibly do is keep updating your system. If you take no other action, just update your system. You’re going to take care of a lot of these attacks and, um, I think we’re getting better at that, but again, uh, for all the listeners, if you’re working at a company that isn’t quickly adopting security updates from your vendors and deploying them out, you are at risk
[00:21:26] Rick Martinez: That’s right. And, and you make incremental improvements in security every day and year over year, of course, with products. But, um we send out a BIOS updates, for example, with new features or fixing vulnerabilities and things like that. So it’s always best to have the latest and greatest on your system.
It’s almost, though, this kind of cyclical paradox, right? So you have customers that, especially with the supply chain threats and in the digital realm that maybe don’t want to update their BIOS, or maybe they don’t want to update and patch their system because maybe that’s, uh, an avenue for, for a supply chain threat, right? So we need to make sure that we get quite a bit of advantage by putting out really solid patches that either incrementally improve the security of the system or, very consistently and accurately patch and update systems, so that, again, it goes back to building that trust that we have with the customer. They know that when they get a, an update from Dell or they get an update from Intel, they know it’s going to work. They know it’s not going to break their system. They know it’s safe to apply and then they can get the benefit of that added incremental security. But if they don’t patch, then you know, we can’t send that out to them.
[00:22:36] Camille Morhardt: I did question for John. Like, do you think that people are becoming more and more–I think people are definitely more and more aware of like applications and the risk of, you know, downloading applications or clicking on certain links. Do you think that people are becoming more aware of the underlying hardware and requirements for security as part of the hardware? Or is that more focused on the enterprise still?
[00:23:00] John Boyle: I think that, um, people are, have, have a greater awareness, uh, whether it’s consumer, commercial. And also on the design side. I mean, we look at ways to make it more of an elegant interaction with your average user, because you know, the, you know, the BIOS, for instance, it’s not something that a mom is really going to be chatting about at the dinner table. But, you know, understanding how to keep her system up to date cause, uh, you know. Or my dad, for instance, he’s a healthcare. So, you know, he has his personal system, he has his it system and there, they can be both in the same network. And, and me being able to understand how to keep those things updated in a very simple manner. Because, you know, not everybody is intimate with all of us on, on the nuances of the system.
And so I think that people are aware that there is a personal responsibility in the security posture. You know, software, firmware, hardware can only do so much. And so part of it is that awareness. It’s the training. It’s a, you know, how do I report something that looks suspicious? Cause it’s better to get something back from IT or the SOC saying “Yeah. That’s okay. That’s actually a valid email or whatever” versus assuming that it’s okay and clicking on it.
From the technology side and the product side, we’re always looking at ways to make the interaction with these capabilities, whether the below the operating system, whether they’re above the operating system, a more elegant user-friendly experience that can be embraced. It’s an evolving part of what we’re looking at, you know, how the average consumer is engaging with security.
[00:24:38] Tom Garrison: Before we let you go, uh, we have a segment on the podcast we do every time called “fun facts.” And so I know we gave you a heads up about. Um, and I’m always intrigued with what are people going to bring in, but, uh, Rick, why don’t you start first with your “fun fact.”
[00:24:55] Rick Martinez: Sure, so recently I’ve kind of a renewed interest and joy in skateboarding. So I’m, my fun fact is related to skateboarding. So, um, one of the greatest improvements in skateboarding over the last 50 years has been what we call concave. So, I f you look at the skateboard, it’s not completely flat, right? It’s got, um, the nose and the tail they bend up in there. It also bends in the middle kind of to keep your feet in there. Right? So that concave is actually created by a very large press, that presses down on the board while it’s drying, while it’s gluing. But one of the interesting things about concave is that if you look at skateboard, do you kind of assume that from a supply chain standpoint, that there, they may all be the same if they’re the same model. But what manufacturers of skateboards do is they actually, to get scale, they put for five to 10 to 15 skateboards in that mold, in that press at one time.
So what ends up happening with the concave is that you do not get a consistent concave across all those boards. So all those boards are the same model. Those are the same shape they’re going to be marketed the same, but every one of those boards in the stack, depending on where it is on the stack, will have a different curvature to the board. And I’ve been listening to a lot of skateboard podcasts too, and it turns out pro skateboarders–even some that were in the Olympics–will actually be very specific about which board in the stack they want to skate. So maybe some of them will only skate that top board. Someone will only escape the bottom board. Some of them will be somewhere in between, but they get very used to the very specific and discreet kind of levels of, of curvature in that board based on where that board sits in the press in the stack.
[00:26:36] Tom Garrison: Yeah, that’s really cool. I had absolutely no idea, but uh, very, very interesting. John, how about you? What’s your fun fact?
[00:26:43] John Boyle: Uh, so my fun fact, I thought about it and Tom, uh, being a Northwest born and raised guy, I’m going to throw this one out when I was, I didn’t know how old I was. 10 years old, we were driving, uh, north of Kelso on May 18th, 1980 and a mountain blew up
[00:26:57] Tom Garrison: Yeah, important date.
[00:26:58] John Boyle: And this is the memory green Volvo, AM radio, three comic books–they’re all Sergeant Rock comic books. My mom had one cassette tape, which was “Wings Greatest Hits” which I can’t stand to this day ‘cause we had to listen to it for 18 hours. Um, but the thing about that is that I always tell people the tidbit is if you ever go buy something and then people say, “oh, here’s some Mount St. Helens ash” like a souvenir or something like that. I will tell you that if it’s a coarse kind of gritty ash, that is not the first explosion ash. First explosion ash is baby powder, talcum powder because of the force of that explosion. So they pass it off as like original explosion ash, don’t buy it because the secondary and tertiary explosions were less powerful so they became a lot coarser ash.
Um, and so there’s your pro tip on buying Mount St. Helen’s Ash from somebody who was stuck in for 18 hours and walked around on it like I was on the moon for my next couple of years. And I will say this also is that nobody thought the mountain blew up. We initially thought that Boeing got hit by a nuclear attack because that was more likely in 1980 than a mountain blowing up. So we were happy that the mountain blew up what a childhood, right?
[00:28:09] Tom Garrison: Good. All right. That market, that market. For our first eruption ash is hot!
[00:28:16] John Boyle: Buy your security software get, get first eruption ash as a bonus!
[00:28:22] Tom Garrison: But Camille, so what is your fun fact for today.
[00:28:24] Camille Morhardt: Okay, well, I was recently at the New Mexico Balloon Fiesta, so I was trying to figure out what are interesting facts about hot air balloons. And the one that I kind of found the most interesting was that true or not, I read that the National Transportation Safety Administration ranks it as the safest manned travel. Because of course I heard all kinds of stories about people crashing and there’s no way to steer.
[00:28:50] Tom Garrison: Yeah, I’ve heard the exact opposite!
[00:28:52] Camille Morhardt: Yeah. There’s no way to steer, obviously you’re in the wind. Um, subject to the only way to steer is to find the altitude that you want to be at, depending on wind speed. So we saw the balloons drifting, you know, all the way across Albuquerque. With a vast majority of them following what appeared to be a desirable trajectory, but definite outliers going, you know, into neighborhoods, straying, close to power lines, all kinds of stuff. So when I read that, I was like, oh, very interesting.
[00:29:20] Tom Garrison: Yeah. That sounds like it’s put on by the Albuquerque Tourism Board.
[00:29:25] Camille Morhardt: I did not double check my source on it, but I thought it was interesting.
[00:29:30] Tom Garrison: Yeah. All right. So my fun fact, um, is, uh, returning to the animal world, which for the listeners know that I tend to do a lot of animal fun facts. That in Switzerland, you are not allowed to have just one guinea pig or parrot. And the reason is that there was a law that was passed in 2008. And the reason is because those animals are highly social and they need interaction and it’s considered cruel not to have a partner. So there you go. If you’re in Switzerland you better have two of them.
[00:30:09] John Boyle: Cause then you’ll have four and six and eight. (laughs)
[00:30:13] Tom Garrison: Just make sure that the same gender, that’s all. Well, hey, uh, Rick and John, thanks so much for joining us today on the podcast. It was a great topic. I think it’s really important that, uh, you know, people understand the importance of supply chain security. I think it’s also important to understand that the solutions that come from both Intel and from Dell are compatible. They, you know, in no way do they compete against each other. And obviously Dell has built a tremendous set of capabilities on top of, uh, the Intel transparent supply chain tools that we’ve already talked about. The importance of supply chain security is more important now than ever and growing over time. So thanks for being here.
[00:30:52] Rick Martinez: Thanks
[00:30:53] John Boyle: Thanks ya’ll. Appreciate it.
