Unveiling a Gatekeeper bypass vulnerability: Exploring the Impact of Launch Services on macOS Security
Looking for vulnerabilities is not my usual daily routine. I am a software developer for Endpoint Security software. I implement new features, improve existing functionality, fixing bugs. So, the discovery of this vulnerability was a surprise. And it made me scared that a macOS update broke our product. In the end, it turned out to be quite a severe vulnerability on macOS.
The discovery:
I was testing how our product behaves on a new macOS beta pre-release version meant for developer use. One of its main features is catching malware, which I expected it to do when I downloaded a test malware sample using Google Chrome.
To my surprise, there was no Gatekeeper prompt warning me that I was dealing with content from the Internet. It was caught by our product, but the missing Gatekeeper prompt meant that the system did not try to validate the safety of that file.
To verify my observation, I downloaded other software to confirm the same behavior. And indeed, I was allowed to run software that was received from the Internet without a confirmation dialog from the system, which is very uncommon.
I noticed that the files Google Chrome was downloading were missing quarantine attribute, which is the main prerequisite for macOS to trigger Gatekeeper for validation of downloaded files and apps.
I was allowed to run an app I downloaded from the Internet without any security validation from macOS. And that’s scary – it would mean that if I was tricked into downloading a malicious app, macOS would not protect me.
Quarantine attribute issue with Google Chrome
Google Chrome sets LSFileQuarantineEnabled flag in its Info.plist manifest, which means that Google explicitly declared to macOS that all the files it creates should be automatically marked with quarantine attribute. But this did not happen for some reason.
It turned out that using Google Chrome was the most crucial part of the discovery process. Using Apple’s Safari did not produce the same results. Files downloaded with Safari were properly marked with quarantine attribute, ensuring that the system invokes Gatekeeper for security assessment.
No Quarantine flag on a file downloaded by Google Chrome means:
- Google Chrome users are not protected from well-known threats.
- macOS will not consult Gatekeeper about validity of code signature and origin of the file.
- Apps that were not properly prepared for distribution will be allowed to run.
- Known malicious signer identities will not be detected.
- Known malicious files will not be detected.
- 3rd party Anti-Virus software depending on the presence of Quarantine flags will not get triggered.
- 3rd party AV software usually tries to limit its effect on system performance and user experience by scanning only files that pose the most threat (downloaded from the Internet)
When I had a clearer understanding of the fact that we are dealing with a vulnerability on macOS side, we reported it to Apple.
Disclosure timeline:
17/01/2023 Vulnerability discovered during F-Secure Endpoint Security product beta release testing on macOS Ventura
17/01/2023 Vulnerability reported to Apple Product Security
14/02/2023 F-Secure releases Mac Endpoint Security product update 19.0 with improved detection capabilities to mitigate macOS vulnerability
24/03/2023 Apple released macOS Ventura 13.3 which addresses the issue
24/03/2023 Apple published Security Advisory for macOS Ventura 13.3
https://support.apple.com/en-us/HT213670
Here’s a demo about the vulnerability:
Root cause analysis:
According to Apple security release notes for macOS Ventura 13.3 release, the affected area of OS is LaunchServices. (reference https://support.apple.com/en-us/HT213670)
So now we have our first suspect. One problem that prevents us from investigating LaunchServices right away is dyld cache.
The dyld shared cache is slightly misnamed because it’s not actually a cache. Rather, it’s part of macOS’s mastering process. macOS takes all the commonly used dynamic libraries and pre-links them together into a single shared file. This allows for many optimizations that improve app startup time. This includes LaunchServices system library. So, in the default configuration, there is no separate binary that we can look at to investigate this further.
Lucky for us, there are tools like https://github.com/keith/dyld-shared-cache-extractor that can take the shared dyld cache and extract individual binaries.
Let’s run this tool and identify which binaries refer to “LSFileQuarantineEnabled” Info.plist flag.
We can see that the main binary of LaunchServices framework refers to that name which means that we are on the right track.
Let’s load the binary into Hopper Disassembler and look at LaunchServices binary more closely. We can see that several functions refer to “LSFileQuarantineEnabled” string.
“LSRegisterSelf” function collects various app metadata and sends it over XPC to lsd service.
Later on, it invokes “LSSetProcessQuarantineProperties” function and forwards the collected metadata to libquarantine library by calling “qtn_proc_apply_to_self” function.
Loading “libquarantine.dylib” into Hopper and searching for “_qtn_proc_apply_to_self” reveals that it invokes a syscall and passes collected metadata to macOS kernel.
So, let’s summarize. During the application launch sequence, LaunchServices framework performs registration of the app. It collects app-specific metadata, like which types of files it can open, where it is located, and whether it requires special handling for files it creates on disk. The information is sent to lsd service for population of app registry database and to the kernel to complete the self-registration process.
Differences between vulnerable and patched macOS
Now that we know what’s happening during app launch, let’s see if we can find any notable differences between the vulnerable and patched version of macOS. The reported vulnerability can be reproduced on macOS Ventura up to version 13.3 beta 2. So, let’s compare how function “LSSetProcessQuarantineProperties” looks like on macOS 13.3 beta 2 (fixed version) and 13.2.1 (last publicly available vulnerable version).
As we can see, the version of “LSSetProcessQuarantineProperties” function on macOS 13.3 beta 2 has two additional condition branches that invoke function “qtn_set_proc_flags”.
The “qtn” prefix tells us that this function is from libquarantine dylib. Let’s look at this function closer by loading the corresponding binary into Hopper Disassembler.
As we can see, “qtn_set_proc_flags” function writes an integer value that represents flags associated with the app at question into a _qtn_proc_t struct at 0x198 offset.
We can also see that the syscall invocation caused by “_qtn_proc_apply_to_self” function during app registration passes the data from _qtn_proc_t struct at the same 0x198 offset to the kernel.
Now let’s switch gears from Hopper Disassembler to LLDB debugger. It allows us to inspect the state of app registration with LaunchServices at runtime.
We will need to configure the system to lower overall security, as by default macOS security policies do not allow attaching to running processes with a debugger. We load into macOS Recovery and run system configuration tool to enable live process debugging:
csrutil enable –without debug –without dtrace
After system reboots, we should be able to proceed with our investigation and troubleshoot the app registration.
Let’s launch lldb and ask it to wait for Google Chrome app to start.
Ildb –wait-for –attach-name “Google Chrome”
As soon as it starts, LLDB will pause its execution and will give us control of the app lifecycle.
Let’s set a breakpoint on “qtn_syscall_quarantine_setprocinfo” function by executing the following LLDB command:
break set -r qtn_syscall_quarantine_setprocinfo
and allow the app to continue running. Our breakpoint is hit. Let’s print out the current backtrace by calling “bt” command to make sure we have stopped at the right place.
As we can see from the backtrace, LaunchServices on behalf of Google Chrome app ended up calling function “qtn_syscall_quarantine_setprocinfo” during the app registration process as we expected.
The registration process starts by invoking functionality from LaunchServices system library which involves calling “LSRegisterSelf” and “LSSetProcessQuarantineProperties” functions. Then, it proceeds to pass the collected information to libquarantine dylib. We have now stopped at calling “qtn_syscall_quarantine_setprocinfo” function which allows us to inspect the values of the function parameters. We know that argument #6 contains the flags associated with apps. Let’s print out the value of these flags on vulnerable and fixed versions of macOS.
We can also see that argument #6 has a value of 0 on the vulnerable version of macOS and value of 1 on the fixed version. Let’s confirm the significance of this discovery by changing the value and observing changes in the behavior of app registration.
Let’s modify the value of argument #6 from 0 to 1 on the vulnerable version of macOS version 13.2.1. We download an unsigned app not meant for public production distribution. Now we see that the system behaves as expected. It shows a Gatekeeper prompt showing an expected prompt, which informs us that the app was not checked by Apple for malware, and that the user is not allowed to run it. That is indeed the case. The app was not properly prepared for distribution and must not be allowed to run.
Now let’s modify the value of argument #6 on the fixed version of macOS 13.3 beta 2 from 1 to 0 and let the app continue execution. Let’s download the same unsigned app. Now we see that the app was allowed to run. There was no Gatekeeper prompt whatsoever which means that we managed to bypass macOS Gatekeeper protection for this file. Now we have confirmation that this was the fix that Apple applied to address the reported vulnerability.
To summarize: as the result of our investigation, we found a way to apply a runtime fix to the vulnerable version of macOS by changing the value of parameter #6 of “qtn_syscall_quarantine_setprocinfo” function invocation and fixing the problem. We also managed to break the fixed version of macOS by applying the reverse change to the value of parameter #6 of “qtn_syscall_quarantine_setprocinfo” function. Thus, confirming our theory of what action Apple took to rectify the vulnerability report.
The root cause of the vulnerability
LaunchServices system library was not passing quarantine flags to libquarantine dylib, causing the operating system to skip Gatekeeper validation for all downloads made by Google Chrome and any other browser vendor that relies on the automatic assignment of quarantine attribute to files activated by setting LSFileQuarantineEnabled flag in the app Info.plist manifest. Apple addressed the issue by making sure the field of the _qtn_proc_t struct is properly populated with necessary quarantine flags in each case during the app registration.
Future research on the topic
- The kernel side of the story remains somewhat a mystery.
- Attempts of live system debugging on latest macOS versions were so far unsuccessful.
- Apple provided tooling (Kernel Debug Kit) is causing kernel panics and memory corruption after the resume of a breakpoint which makes it difficult to inspect the live state of the kernel during the decision-making process.
- The vulnerability was reproduced when quarantine attribute was set on the app with LSFileQuarantineEnabled key in Info.plist manifest.
- Why was the vulnerability only seen in this case? We observe different system behaviour for apps which are downloaded from the Internet vs. apps built locally.
- Is it the reason why Apple did not catch this in time? What testing practices do Apple engineers follow? Do they test their OS updates with 3rd party software? Do they try simulating the full user flow – which involves downloading software from the Internet?
- macOS Monterey was not affected by the reported issue.
- What logic did the kernel have before Ventura? According to our investigation, LaunchServices framework did not pass the quarantine flags to the kernel prior to the release of macOS Ventura, which hints that the kernel had a different decision-making logic for apps in the same setup as Google Chrome. Was it filling in the flag metadata passed on macOS Ventura with __qtn_proc_set_flags on its own?
- Was this discovered behavioural change in macOS Ventura meant to bring more flexibility into why quarantine attribute can be assigned to a file, but broke the most crucial flow in the process?
Final thoughts:
Relying just on one signal for activation of protection mechanisms (quarantine attribute) is not a bullet-proof solution. One of the main reasons why this vulnerability became possible is because macOS relies on the presence of quarantine attribute to activate Gatekeeper protection capabilities. And as we’ve seen, a small logic bug in LaunchServices implementation prevented quarantine attribute from being set on files.
Apple should rethink the security approach and either introduce additional, more reliable metrics, or a completely different security procedure for validation of content originated from the Internet. Otherwise, we will keep seeing similar security incidents.
It’s also a good idea to review and expand the testing techniques to make sure that the changes int the operating system’s internal logic do not break any existing expectations from app developers. Especially, when it comes to wide-spread popular software on the platform – like one of the most used browsers.
Glossary
macOS has multiple layers of protection when it comes to validation of safety of files, apps, documents, etc.
Gatekeeper:
macOS includes a security technology called Gatekeeper, which ensures that only trusted software runs on a user’s Mac. Gatekeeper verifies that downloaded software is from an identified developer, is notarised by Apple to be free of known malicious content and hasn’t been altered. Gatekeeper also requests user approval when opening new software to make sure the user hasn’t been tricked into running executable code.
Extended attributes:
Extended attributes are metadata components that can be unique to specific files and file types on Mac OS. They can be anything from identifying data about the file, to quarantine information, origin data, label information, etc.
The metadata is often a null-terminated UTF-8 string but can also be arbitrary binary data. File metadata is stored in extended file attributes (EA) in macOS, it can be displayed at the command line with ls -l@ filename
Quarantine attribute:
Files which are downloaded from the Internet can have a quarantine flag attached to them by the app which performs the downloading.
The quarantine flag is an opt-in system, not one imposed by macOS itself. Any developer can download files from the Internet without setting the flag on them, and any app on your Mac can change or strip the quarantine flag on any item to which it has write permission. The use of these flags in security is very much a gentleman’s agreement, which is easily broken when software doesn’t behave like a gentleman.
Presence of quarantine attribute is one of the prerequisites for macOS to activate Gatekeeper. If quarantine attribute is missing on a file/app, Gatekeeper will not be activated, and the operating system will allow access without additional enforced security validation.
Both Gatekeeper and 3rd party Endpoint Security software might be interested in files downloaded from the Internet and might rely on similar triggers for activation of protection capabilities. This might be done to prevent degradation of performance for the user. The fewer files scanned; the less impact Endpoint Security software has on user’s day to day interaction with a Mac.
Launch Services:
Launch Services is a macOS system component used to manage launching/opening of applications, documents, etc. It acts as a central hub that facilitates the interaction between the user, the operating system, and the applications installed on the system.
One of the primary functions of Launch Services is to determine the appropriate application to handle a specific file or URL based on its file type or scheme.
LSFileQuarantineEnabled Info.plist flag:
Info.plist is a special app manifest which contains important information that helps the system decide how to launch. It describes the main executable that needs to run whrn the app is launched, which file types that app can handle, what’s the version of the app, etc.
Files created or downloaded by an application that has the LSFileQuarantineEnabled flag set to true in the Info.plist will have the Quarantine attribute (com.apple.quarantine). By default, the LSFileQuarantineEnabled is set to false.
When quarantining files, the system automatically associates the timestamp, app name, and the bundle identifier with the quarantined file whenever possible.
App developer can decide to use this flag to avoid accidental misses of not properly setting the quarantine attributes on newly created files.
Categories