Performance and resource guidelines

General statement on performance and resource tuning

The performance of a system depends on the entire system. For a real system, the application, its workload, and the Anjuna Nitro Runtime will all influence the performance.

This page focuses on Anjuna’s benchmarks of the Anjuna Nitro Runtime’s performance only, so you should also consider the application, workload, cloud environment including network configuration, and other related configuration when profiling your own system.

Performance expectations

Generally, performance overhead in AWS Nitro Enclaves is caused by the communication with the enclave over the vsock through the Anjuna Network and File Proxies, as well as the encryption and decryption of memory within the enclave.

If an application fits in the enclave’s trusted RAM, it can be faster than running outside of the enclave. The more data that moves across the vsock (network and file), the higher the performance overhead. This means that generally, CPU-bound applications perform better than IO-bound applications.

Communication over vsock

Testing recommendations

Anjuna recommends creating a test environment for a meaningful, apples-to-apples comparison between the enclave and the unprotected application.

If an unprotected application has access to 4 CPUs and 8 GB of memory, while an enclave only has access to 2 CPUs and 4 GB of memory, then the unprotected application is likely to outperform the enclave by having more resources. This comparison would not give you meaningful information about the performance overhead of running in an enclave.

Anjuna recommends this high-level list of considerations in your testing:

  1. Same environment:

    1. Container running on VM: run the same unprotected container with Docker (or another container engine like Podman). This ensures similar containerization overhead.

    2. EKS: run the same unprotected container (Pod), ideally on the same Node in the same cluster using Node affinity.

    3. Also ensure the same networking environment: region, availability zones, load balancer(s), DNS resolution, etc.

  2. “Same” resources:

    1. See Resource recommendations below for enclave and parent/launcher resource recommendations.

    2. Container running on VM: use Docker options like --memory, --cpus to limit the resources available to the unprotected container to match the enclave.

      1. Although you can limit the userspace memory and CPUs with these options, the container’s kernel is shared with the host instance, so it may be able to use more kernel memory than the corresponding enclave. So ideally, the instance size is also as close to these limits as possible.

    3. EKS: when using the Anjuna Webhook, use the same specification file with the nitro.k8s.anjuna.io/managed: yes label. If not using the Webhook, ensure that the same limits and requests are set on the enclave Pod and the unprotected Pod.

  3. Use automated testing and reporting, which can be easily run again for any application or environment changes.

  4. Warm up caches outside the enclave, by running the test a few times before recording results.

  5. If the application or configuration changes and you want to run a new test, run the test on both the enclave and the unprotected application again - cloud environments are dynamic and something may have changed (e.g. the VM was migrated to a different physical server by the CSP).

If you conduct performance testing and find that the performance overhead is unacceptable for your use case, contact support@anjuna.io for guidance.

Resource recommendations

Enclave size

CPU: same as unprotected application

Memory:

  • Minimum: same as the unprotected application

  • If the application has significant disk writes without persistent mounts (e.g. temp files), it will occupy enclave memory up to a maximum of 50% of enclave memory. So, add the total max size of those disk writes.

Parent instance (VM) or launcher Pod (EKS)

When you start an AWS Nitro Enclave, the resources for the enclave are subtracted from the parent instance.

For example:

  • Start with a parent instance with no enclaves, 8 CPUs, and 16 GB memory

  • Create an AWS Nitro Enclave with 2 CPUs and 4 GB memory

  • The parent instance is now left with 6 CPUs and 12 GB memory

Recommendations:

  • CPU for the parent: 50% of the enclave’s CPUs

  • Memory for the parent: 1 GB per enclave

Following is an example for running one enclave with 4 CPUs and 8 GB memory. Anjuna recommends that the parent instance, after subtracting the enclave resources, should be left with at least 2 CPUs and 1 GB memory. Therefore, the total original instance size should be at least 6 CPUs and 9 GB memory.

Resource Enclave Parent (once enclave has started) Total

CPU

4

2 (50% of 4)

6

Memory

8

1

9

Increasing resources to improve performance

With equal resources, you may see some performance degradation, especially for IO-heavy applications (see Performance expectations above).

To reach equivalent performance, you can increase the resources available to the enclave (CPU and memory). By adding more resources, you can achieve the security of the enclave at the same performance levels as a (less-resourced) unprotected application.

Generally, increasing resources available to the parent instance or launcher Pod will not significantly impact performance. Test in your own environment to confirm.