Preparing EKS Nodes

In this section you will configure your K8s Nodes and install AWS Nitro dependencies.

Add a label to each of your AWS Nitro-based Nodes in order to let the Anjuna Device Manager access its devices and map them to the Pods running under the Node:

$ kubectl label nodes [AWS Nitro-based Node name] \
      "anjuna-nitro-device-manager=enabled"

Configure your AWS Nitro-based K8s Nodes and install all AWS Nitro dependencies by executing the following Bash script on your K8s Node.

This script reserves two vCPUs and 4GB of RAM for AWS Nitro Enclaves, which is suitable for a single enclave per Node.

To change these settings (for example, to reserve eight vCPUs to run multiple AWS Nitro Enclaves), change lines 3 and 4 of the script:

  • Amazon Linux 2

  • RHEL 8

#!/bin/bash

export NITRO_RESERVED_CPU=2
export NITRO_RESERVED_MEM_MB=4196

# To build EIFs on-the-fly on EKS v1.24 and later, you must ensure that the
# Docker service is installed and running:
INSTALL_DOCKER=true

# Create a group for accessing the AWS Nitro Enclaves hardware and set a static GID to it
sudo groupadd --gid 75 --system ne

# Install dependencies
sudo amazon-linux-extras install -y aws-nitro-enclaves-cli
sudo yum install -y aws-nitro-enclaves-cli-devel jq openssl11-libs

# Add the current user to the Nitro Enclaves and Docker groups
sudo usermod -aG ne "${USER}"
sudo usermod -aG docker "${USER}"

# Automatically load the device drivers needed for communicating with the AWS Nitro Enclaves hardware
echo 'KERNEL=="vsock", MODE="660", GROUP="ne"' | sudo tee /usr/lib/udev/rules.d/99-vsock.rules
echo 'KERNEL=="nitro_enclaves", MODE="660", GROUP="ne"' | sudo tee /usr/lib/udev/rules.d/99-nitro_enclaves.rules
sudo udevadm control --reload-rules
sudo udevadm trigger

# Configure the Nitro Allocator Service
cat <<EOF | sudo install -D --mode 0644 /dev/stdin /etc/nitro_enclaves/allocator.yaml
---
cpu_count: ${NITRO_RESERVED_CPU}
# keep memory_mib under 1GB to force usage of 2MB hugepage
memory_mib: 512
EOF

# Start the Nitro Allocator Service
sudo systemctl daemon-reload
sudo systemctl enable nitro-enclaves-allocator.service
sudo systemctl start nitro-enclaves-allocator.service

# Find out number of pages (2MB in size) required to allocate
RES_PAGES=$(( ${NITRO_RESERVED_MEM_MB} / 2 ))

# If the requested memory is odd number, add one more page
REMAINDER=$(( ${NITRO_RESERVED_MEM_MB} % 2 ))
if [[ ${REMAINDER} == "1" ]]; then
    RES_PAGES=$(( ${RES_PAGES} + 1 ))
fi

# Set the number of hugepages to reflect the reserved memory for AWS Nitro Enclaves
sudo sysctl -w vm.nr_hugepages=${RES_PAGES}
# Automatically set the number of hugepages
echo vm.nr_hugepages = ${RES_PAGES} | sudo tee /etc/sysctl.d/99-anjuna.conf

if [[ "$INSTALL_DOCKER" == "true" ]]; then
    sudo amazon-linux-extras install -y docker
    sudo systemctl daemon-reload
    sudo systemctl enable docker.service
    sudo systemctl start docker.service
fi
#!/bin/bash

export NITRO_RESERVED_CPU=2
export NITRO_RESERVED_MEM_MB=4196

# To build EIFs on-the-fly on EKS v1.24 and later, you must ensure that the
# Docker service is installed and running:
INSTALL_DOCKER=true

# Install common prerequisites
sudo yum update -y
sudo yum upgrade -y
sudo yum install -y git jq make

# Create a group for accessing the AWS Nitro Enclaves hardware and set a static GID to it
sudo groupadd --gid 75 --system ne

# Create the 'ne' group, used for AWS Nitro Enclaves
sudo groupadd --system ne
sudo usermod -aG ne "${USER}"

# Set up the required directory for AWS Nitro Enclaves logs
sudo mkdir /var/log/nitro_enclaves
sudo chgrp ne /var/log/nitro_enclaves
sudo chmod u+rwx,g+rwx,o-rwx /var/log/nitro_enclaves

# Automatically load the device drivers needed for communicating with the AWS Nitro Enclaves hardware
echo 'KERNEL=="vsock", MODE="660", GROUP="ne"' | sudo tee /usr/lib/udev/rules.d/99-vsock.rules
echo 'KERNEL=="nitro_enclaves", MODE="660", GROUP="ne"' | sudo tee /usr/lib/udev/rules.d/99-nitro_enclaves.rules
sudo udevadm control --reload-rules
sudo udevadm trigger

# Create the /run/nitro_enclaves directory on boot
echo 'd  /run/nitro_enclaves  0775 root ne' | sudo tee /usr/lib/tmpfiles.d/nitro_enclaves.conf
# Make directory available without rebooting
sudo systemd-tmpfiles --create /usr/lib/tmpfiles.d/nitro_enclaves.conf

# Install the AWS CLI (only needed for developers)
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
sudo yum install -y unzip
unzip "awscliv2.zip"
sudo ./aws/install
rm -rf "awscliv2.zip" aws

# Install the AWS Nitro Enclaves allocator service from source
git clone https://github.com/aws/aws-nitro-enclaves-cli.git
cd aws-nitro-enclaves-cli
sudo install -D -m 0755 bootstrap/nitro-enclaves-allocator /usr/local/bin/nitro-enclaves-allocator
sudo install -D -m 0664 bootstrap/allocator.yaml /etc/nitro_enclaves/allocator.yaml
sudo install -D -m 0644 bootstrap/nitro-enclaves-allocator.service /usr/lib/systemd/system/nitro-enclaves-allocator.service
cd ..
rm -rf aws-nitro-enclaves-cli

# Configure the Nitro Allocator Service
cat <<EOF | sudo install -D --mode 0644 /dev/stdin /etc/nitro_enclaves/allocator.yaml
---
cpu_count: ${NITRO_RESERVED_CPU}
# keep memory_mib under 1GB to force usage of 2MB hugepage
memory_mib: 512
EOF

# Start the Nitro Allocator Service
sudo systemctl daemon-reload
sudo systemctl enable nitro-enclaves-allocator.service
sudo systemctl start nitro-enclaves-allocator.service

# Find out number of pages (2MB in size) required to allocate
RES_PAGES=$(( ${NITRO_RESERVED_MEM_MB} / 2 ))

# If the requested memory is odd number, add one more page
REMAINDER=$(( ${NITRO_RESERVED_MEM_MB} % 2 ))
if [[ ${REMAINDER} == "1" ]]; then
    RES_PAGES=$(( ${RES_PAGES} + 1 ))
fi

# Set the number of hugepages to reflect the reserved memory for AWS Nitro Enclaves
sudo sysctl -w vm.nr_hugepages=${RES_PAGES}
# Automatically set the number of hugepages
echo vm.nr_hugepages = ${RES_PAGES} | sudo tee /etc/sysctl.d/99-anjuna.conf

if [[ "$INSTALL_DOCKER" == "true" ]]; then
    sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
    sudo yum install -y docker-ce docker-ce-cli containerd.io
    sudo usermod -aG docker "${USER}"
    sudo systemctl enable docker
    sudo systemctl start docker
fi

After executing the script above, reboot your K8s Nodes for the settings to take effect.