Should your laptop ever get stolen and fall into the wrong hands, you would probably be comfortable in the knowledge that the data on it is protected by full disk encryption. But what if a malicious adversary could get around that encryption and access the data anyway? F-Secure’s Olle Segerdahl and Pasi Saarinen have discovered a flaw that allows attackers to do just that, and it affects almost all modern corporate laptops – probably yours too. Last month we published their research, and today Olle and Pasi are joining us for Episode 14 of Cyber Security Sauna to talk about bypassing the mitigations in place for cold boot attacks, and what companies can do to mitigate the risk of exploitation.
Janne: Welcome, guys.
Olle: Thank you, Janne.
Pasi: Thanks.
So what is a cold boot attack?
Pasi: So the cold boot attack is something that was discovered in 2008, or at least made popular in 2008, which basically is that when a computer is turned off, there is some data left in your RAM memory. In normal cases, you think that all data is lost when the computer is turned off. But the data remains for a couple of seconds and the cold boot attack is basically that you cool the memory to make sure that it remains for a longer time, and then you reboot the system into a different operating system that will dump the memory and read out whatever secrets was in RAM.
So you basically reboot the computer before all the information is gone?
Pasi: Yes.
All right. Tell us about this particular hack. What did you guys discover?
Olle: Well, the stuff that we were looking at was how do we actually perform this ten-year-old attack on modern machines? Because during these ten years since the Princeton researchers popularized the cold boot attacks, and actually provided people with the tools to try it out for themselves, a lot of mitigation work has been done to try to minimize the risk of such an attack working in practice. A lot of those mitigations are around firmware, so your BIOS, as it used to be called. Nowadays it’s called EFI firmware, but it’s still the same kind of thing. It’s what actually boots first in your PC before you load your OS. And there are lots of settings you can do there. Everything from locking it down with a password so you can’t boot from a USB stick without knowing the password, to modern features like secure boot, which actually requires the OS to be to be signed in order for it to be loaded. And what we tried to do is figure out how to still perform this attack on modern machines even though all these mitigations might be in place. And that’s when we discovered the memory overwrite mitigation.
Pasi: The memory overwrite request is something that an operating system tells the BIOS that “I have secrets in memory” and if the BIOS starts and this more bit is set then the BIOS will clear the memory. So a normal procedure is that the operating system starts, loads some secrets into memory, sets this bit, and then when the computer is shut down, it clears its bit so that the BIOS doesn’t have to clear the memory, as the operating system has already done that. But if the computer is crashed like you would do in the cold boot attack, then the operating system never clears this bit. And therefore, the BIOS will clear the memory itself.
Olle: You could wonder why is this thing, sounds very complex, needed? Why can’t you just wipe all memory on every boot? Basically, this way of signaling from the operating system to the firmware that the memory wipe is needed, it’s something that’s done for performance reasons. So the platform vendors, the PC makers, they don’t want every boot to include this unnecessary memory wipe that might take a couple of seconds. They want the machine to start as quickly as possible. So this is kind of a compromise where the operating system can tell the firmware if a memory wipe is needed or not.
So you’re preventing the computer from clearing the memory so that some residual information remains in RAM for those couple of seconds for you guys.
Olle: Yeah, so the EFI firmware will check for this signaling bit from the OS on every boot and that’s when it will clear the memory if the bit is set. So what we do basically is when we do our reset attack, when we shut off the computer and we’re going to boot it into our own OS, we simultaneously clear this bit physically in the actual memory where those settings are stored. On every PC motherboard, there’s a small flash chip which contains both the BIOS code and the BIOS settings so the EFI firmware and, and all the settings for the EFI firmware and of course including this memory overwrite request variable that we’re attacking. So what we actually do is connect a small hardware device directly onto the flash chip and change that value to zero, which then of course fools the BIOS or firmware into thinking that a clean shutdown was performed and everything is fine and they don’t actually need to wipe memory on the next boot.
I see. But information remains in RAM that enables you to carry out the rest of the attack. So what happens next? What does the attack enable?
Pasi: Basically it allows us to extract information from the memory that was supposed to be secret. In normal attacks, you don’t think about this physical vector. So if you’re thinking about having some normal encryption key or password in your memory, you usually don’t think about this hardware attacks, but therefore we have focused on hard drive encryption as that is specifically protecting us against these.
Olle: You could say that most impactful thing to attack would be the hard drive encryption keys because once you have access to those, then you also have access to all of the data stored on the hard drive that you could then steal or potentially you can even tamper with and leave a backdoor, for example, in the computer. But of course there are other secrets as well. You might have something that you were writing, a document or something that would remain in memory, or a password that you entered somewhere that could also remain in memory. But all of that is very case by case. We don’t know exactly what we expect to find there. But these hard drive keys are a great impact. And we also know that these keys will be present since almost everybody today is using hard drive encryption.
Sure. So you can read whatever is on the hard drive. Can you alter the data as well? Could you, for example, add a key logger or something like that?
Pasi: Yeah. Once we have extracted the encryption keys for the hard disk encryption, then we can modify it as much as we want.
Right. So when the user resumes using that machine they’re in your control?
Olle: Yeah, they would perhaps notice that it’s been shut off, but that’s pretty much it. If we were able to disassemble the machine to get to this chip on the motherboard, that is of course easier on some models of laptops than others. So very modern machines that don’t have a lot of space to spare, they usually require a bit more disassembly to get to the motherboard. So, I mean, as long as we can do that without leaving any traces, then definitely it could be a risk for the user just to continue using that machine.
Right. Okay. So first of all, you need physical access to the device and then some disassembly is required. Take me through this. I’m standing next to you guys watching you do this. What am I seeing?
Olle: Well, if it’s a standard business laptop, normally these are made to be quite serviceable so you would flip it upside down, remove a couple of screws, probably remove a panel or the whole underside of the laptop, and that would give you access to the motherboard. Most cases, again for serviceability and production reasons, the flash chip would then be accessible so we could access that. And instead of having to solder wires directly onto the chip, we use a test clip, like a clothesline clip or something like that, that we put on top of the chip, make contact with all the pins and then we use the little hardware tool to just speak to the chip and change the contents. And then of course we can reassemble the machine after we’re done with the attack.
Pasi: Before we start all of this, we look that the computer is booted and we have to crash the machine too.
Olle: Yeah. The the important parts that we haven’t mentioned yet is of course that there has to be some secrets in memory for us to have something to steal. So this is where the, the most effective mitigation against these attacks comes in, which is if you have a machine that’s powered off completely and that requires some kind of password to unlock the hard drive when you boot it up again, then of course we don’t have anything to work with because then we would have to know the password for us to be able to boot the machine up and load the encryption keys into memory. But if we find a machine that’s either powered on or sleeping that already has the encryption keys in memory, or if it’s a machine where we can start the machine and it will boot up Windows, for example, without requiring any user input, without requiring a password to unlock the hard drive, then of course those encryption keys will be loaded into memory. That’s also the default configuration of Bitlocker on Windows, to boot the machine without requiring any input from the user.
Okay. So sleep mode is not enough. What about hibernate?
Pasi: In hibernate you actually store the RAM contents to the hard drive before shutting down. So hibernate is actually a safe mode.
So how long does it take you guys to perform this attack?
Olle: With unscrewing the machine and finding the chip and everything. I’d say a couple of minutes, but the actual attack definitely seconds because we’re relying on this memory to keep its contents and it will only hold it for so long. Even with cooling spray.
So what kind of machines are vulnerable to this attack? You talked about most modern computers. Is that the case?
Olle: We can’t speak for all machines because of course we haven’t tested all models from all vendors but we’ve done extensive testing on different vendors’ machines and it looks very much like all of the PCs that we’ve tested have this problem where the firmware settings can be manipulated by physical access. And of course that means that our attack, will work on these machines. Because we haven’t said this before, but those other mitigations such as setting a password on the BIOS to prevent you from booting from external media, secure boot, all these things, are also settings that are stored in this flash chip and that means that they can be tampered with. So for example, we could perhaps reset that password or just change the boot device, the boot order so that it will boot from USB stick happily on next boot at the same time that we’re doing this tampering with the memory overrwrite request variable.
That’s pretty cool.
Olle: So basically all models that we’ve looked at, and that includes the Apple Macbook line, Macbook Pro line. Except for the very latest model, which actually does hard drive encryption in a very different way – the very latest Macbook Pro 2018 models I think. All of the PC hardware running Windows and Linux that we’ve looked at is as possible to perform this attack on. And of course we haven’t talked about desktop PCs that much because those aren’t probably going to be stolen as often or lost as often in a taxi, for example, as a tablet or a laptop device.
Absolutely.
Pasi: Sorry, to go back quite much actually, when Olle mentions these mitigations, the mitigations need to try to protect against us being able to boot our own code that then dumped this memory. So that’s why we mentioned BIOS password or boot order for example, because we are changing which system is starting.
No, I get it. Yeah, sure, sure.
Olle: Yeah. So the common mitigations that were recommended to try to minimize the risk of an attack like this were all based around locking down the firmware settings so that it was harder to perform this kind of attack. But since all of these mitigations revolve around settings stored in the same flash chip we can basically bypass all of them were the same techniques.
Okay. So the original attack was introduced in 2008, and now ten years later you’ve updated it to sort of bypass all the mitigation since to work on modern machines. How likely is it that this attack would be carried out in the real world? Have you guys seen any indications that it has already happened, been exploited in the wild?
Olle: It’s very hard to see indications of it in the wild because a correctly performed attack won’t leave many traces. Especially not if it’s a stolen device because you might never see that device again, right? So you don’t know what’s happened to it. So it’s very hard to get any kind of indication of how prevalent this might be. What we can do is speculate that people who are tasked with actually extracting information from PCs, from hard drives may be engaged in forensics for law enforcement or some other government agency that might want to inspect people’s computers. It would surprise us if these techniques weren’t already known to such agencies. And what’s to say that a determined and well-funded criminal adversary wouldn’t develop these techniques as well.
What would be a situation where this attack and not something else would be the best way forward for an attacker?
Olle: Well, definitely what we’re looking at as the major scenario is a device lost scenario where a laptop is lost in the back of a taxi or stolen in a restaurant or something or something like that. Of course, for somebody to go to all this trouble forming this attack and developing the tools needed to perform the attack, they probably are going after something that they know is going to be on that machine, and not just doing it speculatively. If somebody steals your laptop in a restaurant, 99 percent chances are that they’re just going to resell the laptop on eBay or something like that for the hardware value. And they’re not actually looking to steal your secrets. But if you do have an adversary, like I mentioned, it could be for example, that your adversary is either a foreign government or in some cases where working with human rights defenders for example, it might even be your own government. If you’re working in a line of business where you actually handle secrets that are worth a lot of money. So one example could be the pharmaceutical industry, R&D there is worth a lot of money. That creates the motivation that might motivate somebody to invest money into performing an attack like this. Whereas if you’re just a consumer or a small business owner, then the chances are this kind of slightly advanced attack will not be what’s going to happen to your laptop if it gets stolen. Probably it’s just going to be resold.
Right. So is there anything companies can do to protect their laptops from this attack? Is there anything the users themselves can do?
Olle: We mentioned before that the effective mitigation for this is to make sure that the machine is powered off and that it requires a password to be switched on again. What we didn’t mention is that in Linux and Mac OS is this is kind of easy. It’s the default that you actually have to enter a password before you unlock the hard drive, or to actually unlock the hard drive. The difference in Windows with Bitlocker is that the default configuration stores these encryption keys in what’s called the TPM, the trusted platform module, which is a small secure chip on the motherboard that’s created for exactly this purpose. And it enables the machine to actually power on without having to supply a password, which is much more convenient for users. This has been the recommended and default way to use Bitlocker on Windows. Problem is that that means that the machine can always be booted up to a state where it has encryption keys in memory where they can be stolen using a cold boot attack. So this mitigation actually requires that you have an IT department that’s aware of this issue and a business that can actually take the decision to inconvenience users with having to remember yet another password to be able to boot up the machine, right? And for, for these reasons, that it inconveniences users and that you actually have to have to have a it department that manages devices actively, this is probably only going to happen in a corporate environment with high security requirements, let’s put it that way.
Pasi: None of these mitigations help if you have removable RAM. Then you can do the original cold boot attack from 2008.
Olle: Yeah, for a lot of modern machines, small factors, slim line notebook type devices and tablet devices, these RAM chips are going to be soldered to the motherboard directly, but it used to be the case, and for a number of different models it still is the case, that the memory chips are socketed, so they’re quite easy to remove. And as Pasi mentioned, then of course you can just remove the RAM chips and put them in a different machine if you want to extract the contents. That was also mentioned more than ten years ago as an option to get at these encryption keys, but it’s a bit more cumbersome. You have to have more hardware tooling as well. You have to have some kind of device to plug these chips into, to read out the contents for example.
How easy is this for vendors to fix?
Olle: Not that easy. As mentioned, the operating system vendors, they can only do so much. They’ve already allowed you to protect yourself by having a boot up password to unlock the hard drive, so you should definitely investigate if that’s the case that you’re using it. It’s hard for them to do much more because what we’re actually doing is tampering with the firmware settings and that’s in the domain of the PC hardware manufacturer and the firmware vendor that they might rely on to help them implement the firmware. And what we’re hoping to see is some updates to this mitigation that is harder to bypass when you have physical access to the machine.
Let’s talk about that. You guys notified Intel, Microsoft, and Apple about your discovery. What actions are you looking for? What actions have they taken in response?
Olle: Well, as we mentioned, the operating system vendors like Microsoft and Apple, in the operating system vendor capacity, they’ve done pretty much all they can except for maybe investigate future additions to hardware and firmware to make these kinds of attack harder. Then of course Intel and Apple are all also platform vendors, firmware vendors themselves, so they are definitely looking into how they can make these types of attacks harder, even for older models of machines. At least that’s the response we got from Apple is that they’re recommending that you use the firmware password to make it harder to boot from a USB drive, for example, and that they’re investigating possible additions to firmware on the older machines that would make it harder to perform these attacks. And as we mentioned earlier, Apple have also in, in the very latest models actually changed the way they do hard drive encryption, so that that’s handled by a separate secure chip that doesn’t share a memory with the main OS CPU.
How do you guys get into researching this topic? Like what’s the story there?
Pasi: We had a great customer that actually wanted us to investigate this stolen laptop scenario. They had made great mitigations against all attacks that can happen when you have a stolen laptop, but they did not have preboot authentication. So even if we got a shut down laptop, we could get it into a booted mode. So this made researching this attack quite handy and useful.
Olle: Yeah, it seemed like this would be the most effective way to gain access to one of their laptops, and using that access to the laptop to plant the back door or to steal credentials that would allow us to connect to their protected networks and use access to the machine to gain access to other systems and attack the business that way. And that’s the scenario that they asked us to investigate.
Hibernation + pre-boot authentication is the best protection against cold boot attacks. No keys in memory to steal!
— olle@nxs.se (@nxsolle) September 4, 2018
So was it just you guys researching this or did you get help from other people?
Olle: Our colleague Timo Hirvonen was also involved in collaborating with us on this research because he’d done some previous work on the software side. So we’re actually reusing his software to boot from USB and extract secrets from memory in this attack. And he has also done the Apple research of actually finding out how these encryption keys are stored in memory in the Macbook Pro for example.
What was the research process like? Was it smooth, rocky, exciting?
Olle: We planned out the whole thing really from day one, using attack path mapping. We figured out what are all the possible ways that we could get to our goal. And then we just went through all of those possible ways and tried to weigh the pros and cons and find the one that seemed most feasible to us. And since we’d already seen what Timo Hirvonen, our colleague, had done with trying to make cold boot attacks work on modern machines, we figured, “We got some help from him, we should be able to do this pretty easily.”
Pasi: After we used his tools, we realized that we managed to dump the memory of the machine, but the complete memory was cleared. So the whole memory dump of an eight gigabyte machine was compressible to 20 megabytes. And then we realized that something is wrong here. We need to check this out.
Olle: Yeah. So we asked Timo and he said he’d also seen this behavior and that’s where our contribution started, to actually research why is this happening? How can we potentially bypass this mitigation that the firmware zeroes out the memory on boot? So that was kind of a snag in our plan, but still it was an interesting research problem to try to find out how this was happening and then figure out a way around it.
But other than that snag, if you want to call it that, you made a plan, you stuck to it, and everything worked out according to that plan.
Olle: Pretty much.
Is that common? Does that happen often?
Olle: Often, but not every time. You have to be able to adapt to circumstances. And that’s essentially what we did here. We had to spend a bit more time than we’d planned for in developing this mitigation bypass. But you know, you roll with the punches, like the old Mike Tyson saying, “Everybody has a plan until they get punched in the face.” It’s part of the plan to be flexible as well. And we try to always have some buffer in there to make sure that we can handle unforeseen consequences.
“In Finland cold boot attacks are just called boot attacks” /cc @Csucsorr
— vierito5 (@vierito5) September 21, 2018
So is this a typical process for you guys when it comes to hacking stuff?
Olle: If it’s work for a client, then you need to be very structured in order to first get to some kind of estimate of how much time you’re going to need and how much it’s going to cost the client. But then also if you want to achieve results, you have to stay within that budget and make sure that you’re working on the right thing. And if the first line of attack doesn’t work out, you need to be able to switch to a different attack before the time runs out. So you need to be more structured when you’re working professionally with it. If it was a hobby project, we could have all the time or all the weekends we want. But on a client engagement, you need to be a bit more structured.
You guys speak of a hobby engagements and when you were younger, what was that like? When did you know that you were going to be a hacker?
Pasi: That’s a very hard question. I guess if you always play around with machines and try to make them do stuff that they were not supposed to do, then I guess it’s a very good indication of what you’re going to do in the end.
Olle: I mean, looking back at it, you could see indications from an early age. For example, I know that when I was only four or five, I was taking apart or old rotary phones to figure out what was inside and what made the noises, things like that. So I was always very interested in taking things apart to see how they worked on the inside. And that’s definitely the kind of motivation and mindset that helps you do this kind of work.
Yeah, sure. So how would you summarize this? What is the one takeaway you want to leave our listeners with about this attack? Don’t use sleep mode.
Pasi: We have used this phrasing, I think Olle invented it, “Sleep mode is vulnerable mode.” I think that’s a really nice saying.
That’s going on a T-Shirt.
Olle: And don’t forget that sleep mode might be bad, but you might be vulnerable if you’ve completely powered off the machine too, if you’re not using the preboot authentication system. So if you’re running Mac OS, if you’re running Linux then you probably are. But if you’re a Windows user, you need to look for that blue screen that says Bitlocker PIN, then asks you to enter something when you start the computer. That in combination with turning the computer off or in hibernate mode, which also turns it off, it what you want to be looking for to minimize the risk of this attack.
Pasi: And I think one more thing is that you should not expect things to work the way that you assume them to.
Olle: That’s a good general rule, that unless you’ve actually verified that something works in the way you think it does, don’t put too much trust in it. I think people have become a bit too complacent about lost laptops, for example, because they think, “Oh well, it’s encrypted. It doesn’t matter if we lose one.” But do you actually know what the exposure might be? That’s important to find out.
If I lose my laptop permanently, there’s not much I can do about that. But if I lose my laptop for a minute and then find it again later, should I be super worried about starting to use it again?
Olle: Maybe not if you lose it for a minute, but if you’re leaving your laptop unattended somewhere, then you should be aware that there are physical access attacks that can be a risk. So for example, we have published other guidance last year about the evil made class of attacks, which is basically a tampering attack with physical access to hardware as well. Those are also relevant to that use case. I think the most important thing is that if you lose your machine, you should have a backup plan for what you do when your machine has been lost. So that could be, for an organization, to keep track of what is stored on there, what kind of passwords might be used with that machine, what kind of VPN credentials, and make sure that those are invalidated once the machine is lost. So make sure that users actually report that they’ve lost the machine so that you can put in some kind of protection in place that make those credentials useless to an attacker, for example.
All right, but you don’t have to go as far as to always store your laptop in a tamper evident bag or anything like that.
Olle: That depends on your on your paranoia level, and it also depends on what you have on that machine that somebody might want to steal, right? Back to the whole, “If you don’t have anything to steal, then nobody’s going to spend the effort and time to actually steal it directly from you.”
Pasi: If you have winning lottery numbers, you should protect it quite well.
Olle: Yeah, but the average consumer probably doesn’t have an attacker that’s out to steal their specific data, so this is probably not in their threat model. But for business users it might make sense to actually find out what the exposure is and take according steps.
Yeah. I know my paranoia level is super high and it seems to be getting higher the longer I’m in this business.
Olle: You can’t take it too far, man.
Alright. Thanks guys for coming and taking us through this attack.
Pasi: Thank you very much.
Olle: Thank you, Janne.
Categories