Processes running inside an OS are isolated from each other and from the OS itself. But, once a flaw in this isolation is exploited by a malicious program, other apps become unprotected, as a privileged process has access to virtually every aspect of the computation.

The goal of Intel Software Guard Extensions (SGX) is to protect the sensitive parts of programs. SGX tries to achieve this by creating an isolated environment consisting of code and data called an enclave. The isolation protects access even by privileged processes (OS, firmware) and provides confidentiality and integrity. [11]

SGX forms a Trusted Computing Base (TCB) - a minimal subset of the computing system with the responsibility to enforce a security policy [2]. The SGX TCB consists of the chip package and does not rely on the BIOS or OS. This means that even in cases where BIOS, ring 0 or hypervisor are compromised, the security can be maintained.

The main use cases for this technology are:

• Secure remote computation
• DRM - copyright protection for Bluray and media streaming
• Protection of proprietary algorithms - the code is protected against reverse engineering
• Attestation of a programs integrity in P2P apps (Tor)

SGX provides confidentiality and integrity of memory with secrets without changing the programming model. The enclave is distributed without any secrets in plaintext. After initialization, the enclave forgoes authentication with a remote server. After trust and a secure channel have been established, confidential data can be safely passed between the enclave and the remote server. Data can also be sealed on the CPU, minimizing the amount of communication needed.

At the core of the communication model, there are two keys. Both are unique to a single CPU, unchangeable and generated by the manufacturer. The Root Provisioning Key (RPK) is known by Intel and is used to verify the CPU’s genuineness. The other key, Root Sealing Key (RSK), is known only by the CPU itself and is used to keep data private to the CPU. Note that the keys are used as key material when deriving other keys and are not used directly. [1]

Multiple enclaves can exist in a processor simultaneously. Two types of enclaves coexist on a platform: architectural enclaves and application enclaves. The architectural enclaves are authored by Intel and run in a special privileged mode, which grants them access to some special keys. Their goal is to provide services to application enclaves, such as provisioning, which will be described later. Architectural enclaves implement parts of SGX to reduce the complexity of the hardware. [1]

In order to communicate securely, two problems must be solved: verification between two enclaves on the same platform (Local Attestation) and verification between an enclave and an outside Service Provider (Remote Attestation).

SGX has been designed to introduce minimal runtime overhead.

Every application enclave must be properly initialized before it can run. Special instructions ECREATE, EADD, EEXTEND and EINIT serve this purpose.

ECREATE creates the structures describing an enclave. It cannot be executed yet. EADD can now be called to copy pages with code and initial data to protected part of the memory. EEXTEND updates the measurement of the enclave. Measurement is a hash of the initial state of the enclave, used later for attestation. Basically, the enclave's measurement is used as its identity and proof of contents.

With SGX 1.0 all memory must be preallocated. SGX 2.0 lifts this limitation and can perform dynamic allocations.

Finally, calling EINIT makes the enclave executable from ring3 (user mode). The enclave is marked as initialized and the following happens:

• Loading of additional memory pages is disabled
• A hash of the state is computed (called Measurement hash)
• The enclave can start executing

EINIT requires an EINIT Token, which can be acquired only from the Launch Enclave. The Launch Enclave will create this token based on the Launch Key and the measurement of the enclave to be launched.

EINIT also checks a certificate created by the enclave author and distributed with the enclave. The certificate has the form of SIGSTRUCT structure and contains fields like enclave’s measurement, attributes and a RSA signature. This is how architectural enclaves are launched -- they are signed by a special Intel certificate, which allows them to gain more privileges.

Once the enclave is running, a user application can call into it. Instructions EENTER and EEXIT are used to enter and exit the enclave. They are, however, not used by application programmers directly, but indirectly through the SDK.

The transition to an enclave is similar to a context switch between processes. The entry point of an enclave is specified by the author of the code, so that the code can depend on the fact that no security checks have been skipped. Also, all modified registers are saved to be restored on enclave exit. On enclave entry, some of the debugging functionality like hardware breakpoints are disabled to protect from a class of attacks that exploit these features. [5]

If an exception occurs while executing in enclave mode, the enclave is temporarily exited and can later be resumed using ERESUME. The execution state is stored to a Safe State Area

Enclave may need cryptographic keys for some tasks. An enclave can request specific keys using the EGETKEY instruction. The keys are derived using enclave specific values and a processor key. Only architectural enclaves are authorized to request some keys (like Provisioning Key).

Secrets can migrate between enclaves with certificates issued by the same author.

Provisioning is an act in which the CPU proves to Intel the authenticity and system version. In exchange, the CPU receives an attestation key. Provisioning takes place after setup and after every firmware or microcode update.

For provisioning, Intel provides online infrastructure similar to a certification authority. Provisioning Enclave is responsible for communication with the service.

The Root Provisioning Key is uniquely generated at manufacture time by Intel. Intel keeps this key, it is therefore a shared secret. From this key, the signature of the enclave and the CPU version (SVN), a second key called Provisioning Key (not to be confused with Root Provisioning Key) is derived. This allows Intel to reject CPUs with unpatched firmware.

Provisioning Enclave uses Provisioning Key to authenticate itself to Intel servers and the service sends back Attestation Key. The key is then sealed using the Provisioning Seal key, which is derived in part using the enclaves’ certificate. This means that only the Provisioning Enclave on that specific processor can access the attestation key. Not another enclave and not Intel.

The Sealed Attestation key can now be stored in an untrusted part of the system (hard drive). It is effectively cached, reducing the number of times provisioning needs to happen.

The CPU is able to prove itself to anybody. Any provisioned secret can be sealed in a similar fashion.

Let’s take a closer look at the use for the attestation key -- attestation.

Enclaves running on the same CPU need to prove to one another that they are to be trusted, so they can communicate. In most cases, an application enclave needs a service from an architectural enclave.

The result of local attestation is verification that two enclaves run on the same TCB platform, and a secure channel providing confidentiality, integrity and replay protection. The channel can be used for exchange of information.

First, the enclave generates a REPORT structure, which cryptographically binds a message with the enclave identity (measurement). The REPORT is verified by the target enclave. The core idea of verification that both enclaves live on the same CPU is the fact they both have access to the same unique Report Key.

The passed messages can include data for Diffie-Hellman Key Exchange. After successful verification, a secure channel can be created.

• Software authors (users, SGX Service Providers) register with Intel and get access to the services

Remote parties require three things before communicating with a remote enclave:

1. the application’s identity,
2. the application’s intactness (that the code or data has not been tampered with),
3. assurance that the application is running securely within an enclave on an Intel SGX enabled platform.

All of these can be proven with the attestation key and service provider.

When a remote client (Service Provider) wants to attest an enclave, it sends a challenge to the enclave. The Quoting Enclave converts local attestation to a remote QUOTE using Attestation Key. QUOTE is sent back. The remote Service Provider needs to relay the QUOTE to the Intel Attestation Service (IAS). Provider gets back Attestation Verification Report, which, if positive, confirms that a genuine SGX processor runs the specified program.

IAS uses Intel Enhanced Privacy ID (EPID) scheme. Its characteristics are:

• group signature scheme
• Cpus are divided into groups (Core i3, …)
• impossible to uniquely identify the signer (only the group)
• Revocation Lists (keys, groups or based on signatures)

The secrets obtained from the remote party can be safely stored on disk using sealing. As a result, remote attestation does not need to happen on every application launch.

During boot time, the BIOS binds a portion of the DRAM for the exclusive use by the enclaves. No memory access, not even by kernel or DMA is allowed.

A part of the protected memory is occupied by the Enclave Page Cache (EPC). Here, the individual 4kB pages of all enclaves are stored. A structure called Enclave Page Cache Map holds additional metadata for each page, which tracks type and ownership of pages.

Each enclave is described by its SGX Enclave Control Structure (SECS). The structure is allocated in a protected part of the memory. The SGX instructions refer to a particular enclave using this structure’s virtual address. [10]

The SECS holds data crucial for attestation.

SGX adds security checks to Page Miss Handler if the logical processor (thread) executes in enclave mode. Basically, an enclave can access non-trusted memory and parts of the EPC that are allocated specifically for it.

Page eviction of the Processor Reserved Memory (PRM) is disallowed as the pages hold sensitive data that should not end up on disk. At the same time, it is desirable to allow page eviction for better resource utilization.

SGX has a solution for eviction. It uses symmetric cryptography and nonces to secure the pages and then evicts them to another part of the RAM, which is not protected from paging and therefore can be evicted by the usual mechanisms.

When switching tasks from executing a container to untrusted code, the translation lookaside buffer (TLB) of the core is flushed.

The software stack consists of the Intel SGX driver, the Intel SGX SDK, and the Intel(R) SGX Platform Software (PSW). The SDK is a collection of tools, libraries, documentation and samples that help in designing SGX enabled applications in C/C++.

The installation guide is thorough, but in my personal experience with installing the SDK for Linux, the process is not straightforward. I ended up using Docker containers, which were also not without their problems.

The programming model is very similar to that of a library. The enclave code is written in a module with a defined interface. In fact, enclave is distributed as a shared object (.so).

The interface between the trusted and untrusted parts of the code is specified in a file in Enclave Definition Language (EDL) format. Edger8r tool generates code files (.c and .h) with bridge functions based on this description. After that, the application can be compiled with a traditional C compiler.

The generated proxies provide additional runtime security checks. Passing arrays is done using copies. [12]

1enclave {
2
3from "sgx_tstdc.edl" import *;
4
5
6
7trusted {
8
9/* define ECALLs here. */
10
11public uint32_t get_sealed_data_size();
12
13public sgx_status_t seal_data([out, size=data_size] uint8_t* sealed_blob, uint32_t data_size);
14
15};
16
17
18
19untrusted {
20
21/* define OCALLs here. */
22
23};
24
25};

The code above is an example of a .edl file describing an enclave’s interface. This enclave provides two functions and does not need to call into the untrusted environment. Outgoing calls are typically needed to interact with the OS. The sealed_blob pointer has additional attributes (out, size), which are needed for memory copying.

Design must think of the application as a trusted part and untrusted part. The trusted part will run on the enclaves. An application can use multiple enclaves. The trusted component should be as small as possible, with minimal API, to reduce the attack surface.

An enclave can be compiled in debug mode that allows a special SGX debugger to inspect it. Production enclaves naturally cannot be debugged by software or hardware debuggers. The code can be easily compiled in simulation mode, which allows the application to be developed in environments without the SGX hardware.

The sample applications showcase different building blocks like enclave initialization, remote attestation, or sealing. They serve as a great reference or starting point to develop a custom solution, especially the build step.

1/* Call sgx_create_enclave to initialize an enclave instance */
2
3ret = sgx_create_enclave(enclave_path, SGX_DEBUG_FLAG, NULL, NULL, eid, NULL);
4
5/* Call into the enclave */
6
7ret = seal_data(eid_seal, &retval, temp_sealed_buf, sealed_data_size);

The code above is an example of creating an enclave and calling a function. seal_data is a glue function generated by the SDK. It calls into the enclave, but looks like an ordinary function. The enclave must be destroyed and each call must be checked for returned error value (omitted).

Libraries are mostly transparent to the programmer. They include Untrusted RunTime System (URTS), KeyExchange or Trusted Support Libs.

The isolated containers of SGX have the goal of providing confidentiality, integrity, and freshness. Intel SGX greatly reduces the attack surface of an application. Still, this solution does not reduce it to zero.

The trusted computing base consists of the CPU package, the code and data running in the enclave, and the Intel Attestation Service (IAS). The rest of the system may be compromised. [9]

The foundation of trust is based on the manufacturer's trustworthiness. More specifically the trust in the key the manufacturer uses to sign each processor's unique attestation key. The threat model assumes that the keys used are not compromised.

Both hardware and microcode of a processor are sometimes updated, often to fix a security issue. The attestation is able to check the revision of the TCB, preventing the use on unpatched hardware. [3]

Analysis of SGX is made harder by the lack of disclosure of details of implementation.

The chip package is the security perimeter, meaning SGX’s threat model excludes physical attacks targeting the chip itself or power analysis [8]. It still, however, makes efforts to make the attacks more expensive.

Intel states that the hardware e-fuses holding the keys are encrypted with AES key, which should make the cost of key extraction higher. It may be the case that Physical Unclonable Function (PUF) is used in physical protection as well [7].

DRAM and the bus are however outside the perimeter so they are not trusted. All data destined to DRAM is encrypted, so it does not make sense to do physical attacks on it or on the memory bus. The key is randomly changed on every boot, so cold boot attacks should be mitigated [4].

Hardware attacks like fault injections are also possible.

SGX does not protect the application from side channel attacks [11]. For example, cache timing attacks are possible, meaning that code running in the container mustn’t use data dependent memory accesses. Software must be designed to not be susceptible to these attacks. [1]

A malicious program executing on the same core in a hyperthreading scenario shares much of the hardware, potentially allowing for side channels to be exploited. One such side channel could be the performance counters. A possible solution here is to disable hyperthreading. CPU with hyperthreading enabled can be rejected during remote attestation. The tradeoff is lost performance. [1]

To demonstrate the efficacy of Intel SGX, consider the following scenario. An application keeps a private key in its working memory. This key can easily be read by anyone with sufficient permissions. However, by implementing the same application within an enclave secured by Intel SGX, the private key is protected.

Let’s compare the two scenarios. With root privileges, one might attempt to access the private key through conventional means, such as querying the /proc filesystem.

By utilizing secure enclaves, it is possible to ensure that even with root privileges, malicious actors are unable to read secrets.

This has been a surface level overview of Intel SGX. For more thorough reading, I refer you to [1, 4, 11].

SGX offers a satisfactory level of protection, especially in cases where the administrators of a system can be hostile, such as in remote computation or client applications. In my view, the biggest security concerns come from the lack of protection from side channel attacks.

Intel SGX is being deprecated from consumer CPUs [6], which limits viable use cases. SGX will continue to be supported on server platforms. The move towards discontinuation, together with the possible attacks may show lack of trust from Intel regarding security of the SGX technology.

Intel Trust Domain Extensions (TDX) is Intel’s newest trusted execution environment technology that may be worth investigating further.

From the developers’ perspective, the programming model is mostly the same, which is good. Existing applications seeking to migrate to SGX may need to adjust their architecture to separate the sensitive part more.