Threat modeling is a foundational part of any security conversation. Security experts John Andersen and John Whiteman, better known as “John Squared” at Intel, live and breathe threat models.
Here they share with the Open at Intel podcast the basics of threat modeling and their advice on getting started creating and maintaining good threat models as part of the software development lifecycle.
Host Katherine Druckman:
Threat modeling should be the beginning of any security conversation, so just to lay the groundwork, what are the basics that any developer should know about it?
John Whiteman:
I use the analogy that it’s like a house. You own this house. You live with your family in this house, and of course you want to protect them, and there are assets inside the house as well. Then, you might live in a neighborhood that’s dangerous, or you might live in a safe neighborhood or something in between, but either way you still look at things like if I have a door, I should put a lock on it, right? If I have windows and I leave the house, maybe I should close them. So, you start putting all these things in place. And it might be a neighborhood that is a bit sketchy, and you might say, well, maybe I should put up some security cameras because the return on investment is there because my car got busted into a couple of weeks ago.
Now, take that analogy and then put it into a project. I’m not saying your project is in a bad neighborhood, but you’re going look at what you're trying to protect. You’re going to look at some of the mitigations to protect these things, and some might not have a good return on investment...What your assets are, who your attackers or adversaries are, and what you are going to do to try to protect them. You usually try to set these objectives up front by building your threat model.
Katherine Druckman:
So the basic questions are:
- What are you trying to protect?
- Who are you trying to protect it from?
- What's the damage if you fail?
John Whiteman:
Yes. In the end, it could also be your environment. You may have that same house, but it depends where that house is located. All of those things matter if you're going out and building something and creating something for a customer, the things you have to worry about are protecting assets that might be important to the customer and probably the most common things we hear about today are data and privacy, so we try to identify what the assets are in a project -- it could be health information, for example -- and we look for ways to protect that. There could be different types of adversaries out there. They could be nation states. They could be script kiddies or whatever, but you put these things in place to protect those assets, to protect you, your reputation and your customers. Fundamentally, that's what threat modeling is.
Katherine Druckman:
How does threat modeling differ when you're considering open source software versus proprietary software?
John Andersen:
It depends on what angle you're looking at it from. If it’s a vendor providing proprietary software, you might have a different threat model than an open source project, a community, or a set of open source projects.
While we don't endorse security by obscurity, you might have a few trade secrets that you want to keep a secret. From that perspective, you’d be looking at a different level of confidentiality for different actors. Your customer is somebody you're trying to protect yourself against if you're a video game, for example. So, from that aspect you end up with different threat actors, and that's all highly dependent on your situation.
Educational software doesn’t need to protect against the customer, but video games must protect against the customer. With open source software, everyone and everything is a threat because you have no idea how your software will be used, whereas with third-party software you're selling something for purpose. You have a pretty good idea of why somebody is paying for that, whereas with open source software you're putting it out there and you really don't know how people are going to use it, which can be a challenge when you’re releasing reference implementations as a company.
Katherine Druckman:
Open source is everywhere. A lot of people who aren't involved in open source don't realize that. Is there much software that doesn't use open source? How important is that distinction when it comes to threat assessment?
John Whiteman:
John and I gave a presentation this year about looking at threat modeling and even treating your threat model as source code, particularly an open source model where everyone in that project becomes part of the threat model development piece. Treat your threat model like you treat source code: You look and identify them, and you try to be open about it. That's one huge difference, as opposed to proprietary software where there may be certain considerations, maybe the proprietary software itself is restricted and not given to a whole bunch of people... I think the approach from open source software should be open source and open as well the threat model.
John Andersen:
There is the providing side, and then there's also the consuming side and of course, with the recent supply chain security focus, we’ve seen a shift to consider how the things included in a project affect its security posture. From that perspective, when you're sourcing your components, you're almost making a life cycle decision there like that XKCD of the maintainer in Nebraska. That’s a choice that factors into the health of your project as its maintenance status can affect the security status of it long term.
Katherine Druckman:
What makes a good threat model? Is it ever complete?
John Whiteman:
A good threat model starts with the basics: identifying what the assets are and what you're trying to protect. Then you want to get into things like trust boundaries. If your threat model is on a network or in the cloud, for example, there are considerations and there are different levels of trust that you have to think about, as well as the flow between the trust boundaries.
If it's as simple as a web application and you have a browser, you know there's of course this big span. It's called the untrusted network that you need to cover and think about who your adversaries are. In some cases that could be advanced attackers like state actors for example.
Or it could be more simple things or even accidental things or inadvertent types of attacks that may occur. But you want to put all of these components in, and you want to make sure that you identify the potential tax that they have, particularly on a capabilities basis, and then consider things like mitigations against them.
This is where risk management comes into play, where mitigations could be something. Another part might be transference of that. You may not have that ownership, or it could even be acceptance. Maybe the risk is low, or the complexity is very high, or the damage is minimized. You get into all those aspects as well, but John, do you think that the threat model is ever done?
John Andersen:
No, because your scope might change in different areas, and you might be scoping things in or out depending on your deployment and that is fundamentally tied to how you envision your project or how your project will actually be used.
It’s highly dependent on scope. If your project is alive, your threat model should be too.
Katherine Druckman:
When you approach the threat modeling process, how much is about a methodology versus mindset?
John Whiteman:
It's a combination of both. Some teams will come in and they've never done threat modeling, so they're not sure what happens, but I see this process that takes place as you go through the threat modeling. They learn about these things, and there are great touch points, particularly as a security person interacting with them.
I've seen a transformation when there is no mindset, meaning there hasn't been any thought about threat modeling to the end of the process where you have a nice complete threat model at that time where the team finally gets it. They say, “Oh, I see what you're talking about,” and it continues after that and you can apply more trust into teams to do it for future releases. As we asked that question before, the threat model is never really complete.
If you're adding new features or there are new kinds of attacks out there that we're not aware of, or just simply things that we forgot about or missed, that is an evolutionary process. Just as the mindset is, too.
Katherine Druckman:
Do you incorporate storytelling or creativity into the process when you're thinking about this?
John Andersen:
As you look at the different attackers and their various capabilities and motivation, they almost tell the story themselves. With Nation state attackers, for example, your story might be, “I'm Lockheed Martin, defend against World War three.” There's your story at a very high level.
Some are more complex, especially if you start looking at physical access when physical access exists and you're defending against an abusive partner situation or something. A lot of physical devices and social networking must deal with that. That’s a sad story, but a much more complex story. If you don't dive into that, you're not going to see the various angles you need to defend against.
Katherine Druckman:
Getting back to the technical aspects here of threat modeling, threat modeling should be undertaken early in the software design process. But can you elaborate on how to incorporate it into the development life cycle? And at which stages do you revisit? All of them?
John Whiteman:
Yeah, pretty much all, but the sooner the better. Especially when teams are doing the design and the architects are putting the pieces together. That's a good time to get somebody from security involved because as with anything, if you can find problems early on, it's going to save a lot of problems later, particularly cost and time.
Through the whole process of it, the upfront work is always the most. You have the people come in, you get all these pieces together, but once you get that initial architect in there and the initial threat model that represents that architecture, then it's a lot less maintenance and you should keep at it, particularly if there are new features, particularly if you're having new releases that are coming into place.
You want to make sure that your threat model is up to date. We had a running joke last summer: “A threat model never expires like milk, it never expires like milk. It's more like top ramen, where you can maybe eat that top ramen two years later. It's probably still “best-by-use date.” Things like that are important to realize. Unless something has totally changed and your project goes out of scope, that threat model should still be living. It shouldn't be dead. It should be kept alive.
Katherine Druckman:
So how often do you need to re-evaluate? How do you find the right balance between too often and not enough?
John Whiteman:
A threat model is just like the product. If you look at the threat model like you look at your project, the evolution of your project. If you're getting new features and a release that's coming out, you probably want to take another look at the threat model to see if anything changed, including things that might have been deprecated, maybe things that you've removed. You want to make sure those are updated.
Just think of the worst case: If something happens to your project and it's breached. First, you probably want to look at the threat model to find out where this happened, why it happened and make those changes accordingly.
You also might have the exact same threat model, and maybe it was in some research department, but now that research will go public. It's going to become a production product. Even though all the technologies are the same, that threat model is going to change because now you have a different audience, and you have a different attack surface from there as well.
Finally, there are always new techniques, new things that people are discovering, particularly on the adversary side, which you may not have considered before as possible in your threat model, so those are things that you want to be aware of too.
It’s saying, “Oh, I didn't realize that somebody could do that. Let's revisit the threat model, update it. and make sure we have the right mitigations in place.”
For more of this conversation and others, subscribe to the Open at Intel podcast:
- Openatintel.podbean.com
- Google Podcasts
- Apple Podcasts
- Spotify
- Amazon Music
- Or your favorite podcast player (RSS).
About the Author
Katherine Druckman, an Intel Open Source Evangelist, is a co-host of podcasts Open at Intel, Reality 2.0 and FLOSS Weekly. A security and privacy advocate, software engineer, and former digital director of Linux Journal, she's a long-time champion of open source and open standards.