Running Nginx in an Anjuna Confidential Container

In this section, you will run a simple confidential container in a secure enclave with the Anjuna CLI. Starting from the Docker container for Nginx, you will configure and run an Anjuna Confidential Container, verify it is using AMD SEV, and inspect its measured boot values.

The following basic steps are needed to start an Anjuna Confidential Container:

  1. Identify a Docker image that contains an application

  2. Build the Docker image into a CSP compatible disk image for an Anjuna Confidential Container

  3. Upload the disk image to the CSP’s storage

  4. Start an Anjuna Confidential Container using the previously created disk image

Using the Anjuna CLI, you can automatically start existing containers in an Azure Confidential VM or a GCP Confidential VM. This documentation refers to these as Anjuna Confidential Containers.

Depending on the application, these additional steps may be needed:

  • Configure the network and firewall rules for the Anjuna Confidential Container

  • Check the SEV attestation report and Measured Boot output to ensure that the Confidential VM has not been tampered with

Next, you will learn about the basic usage of the Anjuna CLI to build and run an Anjuna Confidential Container.

Identify a simple Docker image

You will use the official Nginx Docker image to start a simple web server as an Anjuna Confidential Container.

To simplify the setup in this example, there is no TLS configuration. Instead, you will run Nginx as an HTTP server and learn how to configure the Anjuna Confidential Container to allow HTTP clients to connect to Nginx.

Build the Anjuna Confidential Container disk image

The Anjuna CLI works with unmodified Docker images. In this example, the Anjuna CLI will pull the Nginx Docker image from the public Docker Hub. If you are using a private Docker registry, use the docker login command to authenticate to your registry before running the following command.

  • Google Cloud

  • Microsoft Azure

$ anjuna-gcp-cli disk create --docker-uri=nginx:latest --size 20G

This command will create a disk image file named disk.raw in the current working directory.

$ anjuna-azure-cli disk create --docker-uri=nginx:latest --disk-size 20G

This command will create a disk image file named disk.vhd in the current working directory.

The disk image contains a bootloader that starts a Linux kernel that boots directly into the Nginx application, as defined by the ENTRYPOINT and CMD parameters of the Docker container.

All the dependencies needed to run Nginx are included in the disk image, so when the Confidential VM starts, it will not need to download the Docker image.

Upload the disk image

In order to create an Anjuna Confidential Container, the disk image must be uploaded to the cloud provider first. This may require pre-existing cloud resources.

Disk upload prerequisites

  • Google Cloud

  • Microsoft Azure

This section requires being authenticated to GCP with the Google Cloud CLI (gcloud). See Install and authenticate to your cloud provider’s CLI for reference.

The disk upload command will create a Google Cloud Storage bucket if necessary, so no pre-existing cloud resources are necessary.

This section requires being authenticated to your Microsoft Azure subscription with the Azure CLI (az). See Install and authenticate to your cloud provider’s CLI for reference.

The following command displays your account and subscription, then sets an environment variable to the subscription for later use. If you have multiple subscriptions, you will need to select the correct one (use az account list to see your accounts).

$ az account show
$ export MY_AZURE_SUBSCRIPTION=$(az account show -o json | jq -r .id)

You will need to have the following resources ready before you issue the anjuna-azure-cli disk upload command. The anjuna-azure-cli command does not create the resources.

  • Resource Group: This command will create a resource group called myResourceGroup

    $ az group create --name myResourceGroup --location eastus
  • Storage Account: The command below creates a storage account named anjunaquickstart<SUFFIX> in the resource group myResourceGroup, with public access disabled to the blobs or containers in the account.

    $ export RANDOM_SUFFIX=$(cat /dev/urandom | LC_ALL=C tr -dc '[:lower:][:digit:]' | head -c 5)
    $ export STORAGE_ACCOUNT_NAME="anjunaquickstart${RANDOM_SUFFIX}"
    $ az storage account create \
                --resource-group myResourceGroup \
                --allow-blob-public-access false \
                --name ${STORAGE_ACCOUNT_NAME}
  • Storage Container: A storage container is similar to a directory and is used to organize data blobs. It is created inside a storage account. Hierarchically,

    resource group storage account storage container blob (disk)

    The command shown below creates a storage container called mystoragecontainer in the storage account from above:

    $ az storage container create \
                --name mystoragecontainer \
                --account-name ${STORAGE_ACCOUNT_NAME} \
                --resource-group myResourceGroup
  • Azure Compute (Shared Image) Gallery: Image galleries are used to organize and share OS images and applications.

    The command below will create a gallery called myGallery in the resource group called myResourceGroup.

    $ az sig create --resource-group myResourceGroup --gallery-name myGallery

    Refer to Creating a Compute Gallery for more details.

  • Image Definition: Image definitions are a logical grouping for versions of an image. Hierarchically,

    resource group image gallery image definition image (for the confidential container)

    The command below will create an image definition called myFirstDefinition in myGallery. This Microsoft document explains the options used in the command.

    $ az sig image-definition create \
           --resource-group myResourceGroup \
           --gallery-name myGallery \
           --gallery-image-definition myFirstDefinition \
           --publisher Anjuna \
           --offer CVMGA \
           --os-type Linux \
           --sku AnjGALinux \
           --os-state specialized \
           --features SecurityType=ConfidentialVMSupported \
           --hyper-v-generation V2 \
           --architecture x64

    Anjuna requires the following settings. The other parameters for the definition can be configured as needed.

    Architecture: "x64"
    Features: {
        SecurityType: "ConfidentialVmSupported"
    }
    HyperVGeneration: "V2"
    OsState: "Specialized"
    OsType:  "Linux"

After you have created the necessary cloud resources, you can upload the disk image.

  • Google Cloud

  • Microsoft Azure

The anjuna-gcp-cli disk upload command will upload your disk image to a Google Cloud Storage bucket, creating the bucket if necessary. The compressed disk image is uploaded to that storage bucket with a name like compressed-image.tar.gz, and a Custom Image for GCE is created using that bucket object.

For example, the following command will create a Custom Image named anjuna-gcp-nginx-image using the Cloud Storage bucket anjuna-gcp-nginx-bucket-<random_suffix>. anjuna-gcp-nginx-bucket will be created if it does not already exist.

$ export RANDOM_SUFFIX=$(cat /dev/urandom | LC_ALL=C tr -dc "[:lower:]" | head -c 10)
$ export BUCKET_NAME="anjuna-gcp-nginx-bucket-${RANDOM_SUFFIX}"
$ export IMAGE_NAME="anjuna-gcp-nginx-image"
$ anjuna-gcp-cli disk upload --bucket=${BUCKET_NAME} --image=${IMAGE_NAME}
Google Cloud Storage bucket names are globally unique, so you may see errors if the bucket name is already used. See Bucket naming guidelines to specify a --bucket name that meets the GCP requirements.

This command uploads the local disk image to an Azure Storage Container and creates a shared image in a Compute Gallery. The shared image is saved as an Image Version of a pre-existing Image Definition.

$ anjuna-azure-cli disk upload \
  --disk disk.vhd \
  --image-name nginx-disk.vhd \
  --storage-account ${STORAGE_ACCOUNT_NAME} \
  --storage-container mystoragecontainer \
  --resource-group myResourceGroup \
  --image-gallery myGallery \
  --image-definition myFirstDefinition \
  --image-version 0.1.0 \
  --location eastus \
  --subscription-id ${MY_AZURE_SUBSCRIPTION}

Create a network with the proper firewall rules

You will need to update the Anjuna Confidential Container’s cloud network configuration to make it reachable through the internet from your management host.

  • Google Cloud

  • Microsoft Azure

Run the following commands to create a network (anjuna-gcp-network), which allows access to port 80 on the VM public interface.

$ export NETWORK_NAME="anjuna-gcp-network"
$ gcloud compute networks create ${NETWORK_NAME}
$ gcloud compute firewall-rules create anjuna-gcp-rule-http  \
    --network ${NETWORK_NAME}  \
    --allow tcp:80

The example below shows one way to create network resources in order to be able to communicate using TCP over port 80. It also attaches a public IP address to the Network Interface called myNic.

$ # Replace these with your own values as needed
$ export RESOURCE_GROUP_NAME="myResourceGroup"
$ export LOCATION="eastus"
$ export VNET_NAME="myVnet"
$ export SUBNET_NAME="mySubnet"
$ export NSG_NAME="myNSG"
$ export PUBLIC_IP_NAME="myPublicIP"
$ export NIC_NAME="myNic"

$ # Create a Virtual Network
$ az network vnet create --resource-group ${RESOURCE_GROUP_NAME} --location ${LOCATION} --name ${VNET_NAME} --address-prefix 10.0.0.0/16

$ # Create a Subnet
$ az network vnet subnet create --resource-group ${RESOURCE_GROUP_NAME} --vnet-name ${VNET_NAME} --name ${SUBNET_NAME} --address-prefixes 10.0.0.0/24

$ # Create a Network Security Group
$ az network nsg create --resource-group ${RESOURCE_GROUP_NAME} --name ${NSG_NAME}

$ # Create a Security Rule allowing TCP traffic over port 80
$ az network nsg rule create --resource-group ${RESOURCE_GROUP_NAME} --nsg-name ${NSG_NAME} --name Allow-80 --protocol Tcp --direction Inbound --priority 1000 --source-address-prefix '*' --source-port-range '*' --destination-address-prefix '*' --destination-port-range 80 --access Allow

$ # Associate the Network Security Group with the Subnet
$ az network vnet subnet update --resource-group ${RESOURCE_GROUP_NAME} --vnet-name ${VNET_NAME} --name ${SUBNET_NAME} --network-security-group ${NSG_NAME}

$ # Create a Public IP address
$ az network public-ip create --resource-group ${RESOURCE_GROUP_NAME} --name ${PUBLIC_IP_NAME} --sku Standard --allocation-method Static

$ # Create a Network Interface with the Public IP address
$ az network nic create --resource-group ${RESOURCE_GROUP_NAME} --name ${NIC_NAME} --vnet-name ${VNET_NAME} --subnet ${SUBNET_NAME} --network-security-group ${NSG_NAME} --public-ip-address ${PUBLIC_IP_NAME}

Controlling log access

The Anjuna CLI for SEV supports cloud logging, which may need additional configuration.

  • Google Cloud

  • Microsoft Azure

Controlling log access with service accounts

The recommended way to manage logs is with the Google Cloud Logging service, which is automatically supported by the Anjuna Confidential Container. Cloud Logging requires the VM to have the Logs Writer IAM role (roles/logging.logWriter) within the project scope. It also requires anyone viewing the logs to have the Logs Viewer IAM role (roles/logging.viewer). See GCP’s Logging roles for more information.

In order to use Cloud Logging, create a service account in your project and grant it the Logs Writer role. Later, you will use this service account when creating the Anjuna Confidential Container instance. The Anjuna Confidential Container will automatically detect the service account and forward the application logs to Cloud Logging.

$ export SERVICE_ACCOUNT_NAME=anjuna-nginx-service-account
$ export GCP_PROJECT=$(gcloud config get project)
$ export SERVICE_ACCOUNT_EMAIL="${SERVICE_ACCOUNT_NAME}@${GCP_PROJECT}.iam.gserviceaccount.com"
$ gcloud iam service-accounts create ${SERVICE_ACCOUNT_NAME} \
    --description="Service Account for Anjuna Nginx Quickstart" \
    --display-name="Anjuna Nginx Quickstart"
$ gcloud projects add-iam-policy-binding ${GCP_PROJECT} \
    --member="serviceAccount:${SERVICE_ACCOUNT_EMAIL}" \
    --role="roles/logging.logWriter"
The anjuna-gcp-cli uses IAM resources such as service accounts, scopes, projects, and roles, enabling you to use Google’s IAM tools to reduce risk. For more information, see GCP’s documentation regarding Access control with IAM and Service accounts.

If a service account with write access to Cloud Logging is not available, the Anjuna Confidential Container falls back to printing all logs to the serial console.

The serial console can be viewed by anyone with access to the project, which may expose confidential information in the application logs. The serial console is also extremely slow and may cause a significant performance impact. Using Cloud Logging by attaching a service account with appropriate permissions is recommended instead.

Currently, the Anjuna CLI for SEV on Azure supports logging to the Azure Serial Console.

The Anjuna CLI already handles the configuration required for the Anjuna Confidential Container to write logs to the Azure Serial Console. Additionally, when the --storage-account parameter is provided to the instance create command, the Azure boot diagnostics are written to that storage account.

Start the Anjuna Confidential Container

Run the following command to start the Anjuna Confidential Container on a new instance.

  • Google Cloud

  • Microsoft Azure

$ export INSTANCE_NAME=anjuna-gcp-nginx-instance
$ anjuna-gcp-cli instance create \
    --instance=${INSTANCE_NAME} \
    --image=${IMAGE_NAME} \
    --network=${NETWORK_NAME}  \
    --service-account=${SERVICE_ACCOUNT_EMAIL}

You should see output similar to the following:

INFO   [0001] Using configured GCP project:  my-project
INFO   [0010] Instance created: anjuna-gcp-nginx-instance

Later, you will use the IP address (internal if the same GCP network, external if different) to connect to the server.

The following command uses the cloud resources created in Disk upload prerequisites. You may need to change the parameter values if you chose different names.
$ export INSTANCE_NAME=anjuna-azure-nginx-instance
$ anjuna-azure-cli instance create \
  --name ${INSTANCE_NAME} \
  --location eastus \
  --image-gallery myGallery \
  --image-definition myFirstDefinition \
  --image-version 0.1.0 \
  --resource-group myResourceGroup \
  --storage-account ${STORAGE_ACCOUNT_NAME} \
  --nics myNic

Verify that the Nginx Confidential VM is running

Once the instance create command completes, your Anjuna Confidential Container will be running. You can run the following command to see its status:

  • Google Cloud

  • Microsoft Azure

$ anjuna-gcp-cli instance describe \
    --instance=${INSTANCE_NAME} \
    --attestation-report

The command above will print the GCP Audit Log events that show:

  • The SEV launch attestation report, which includes:

    • Integrity Check, which is the result of an integrity check performed by the Virtual Machine Monitor on the measurement computed by AMD SEV.

    • sevPolicy, which is the AMD SEV policy bits set for this VM; policy bits are set at Confidential VM launch time to enforce constraints such as whether debug mode is enabled.

  • The measurements created by Measured Boot, which use platform configuration registers (PCRs) to store information about the components and component load order of both the integrity policy baseline (a known-good boot sequence), and the most recent boot sequence.

    The command should produce output similar to this:

    INFO   [0000] Using GCP project:  my-project
    INFO   [0000]
    INFO   [0000] Instance (anjuna-gcp-nginx-instance)	ID: 1360527994822224099
    INFO   [0000] Created: 2022-01-14T12:27:41.611-07:00
    INFO   [0000] M/C type: n2d-standard-2	Zone: us-central1-a	Confidential: true
    +
    Measurements:
       PCR_0  0xC032C3B51DBB6F96B047421512FD4B4DFDE496F3
       PCR_1  0xA397259104C4DFE42A77F269BD3FBC5281B33E2D
       PCR_2  0xB2A83B0EBF2F8374299A5B2BDFC31EA955AD7236
       PCR_3  0xB2A83B0EBF2F8374299A5B2BDFC31EA955AD7236
       PCR_4  0x3BE35BA596CEA84FD2330181999C7781E190D31A
       PCR_5  0x2A6AB2900EABD0BE97B664CB4C4FF03CD4EC93DF
       PCR_6  0xB2A83B0EBF2F8374299A5B2BDFC31EA955AD7236
       PCR_7  0x8F0938646BEA0FF83B71B080EFAD8400B89D345C
       PCR_8  0x360BC4823BBDEA3861F7B6331F4395AD23F316C4
       PCR_9  0xCA087A7BD7CAEC2B8C4C0CC0E51D1A70D27DEA1F
    INFO   [0002]
    SevPolicy:
    {
        "debugEnabled": false,
        "domainOnly": false,
        "esRequired": false,
        "keySharingAllowed": false,
        "minApiMajor": 0,
        "minApiMinor": 0,
        "sendAllowed": true,
        "sevOnly": true
    }
    INFO   [0002]
    Integrity Check: true

    By inspecting the Audit Log events, you can be sure that the newly created GCP Confidential VM is running in an AMD SEV-enabled virtual machine.

$ anjuna-azure-cli instance describe \
  --name ${INSTANCE_NAME} \
  --resource-group myResourceGroup

The command should produce output similar to this:

Name                         ResourceGroup       PowerState      PublicIps         Fqdns    Location    Zones
---------------------------  ------------------  --------------  ---------------  -------  ----------  -------
anjuna-azure-nginx-instance  myResourceGroup     VM Running      172.171.202.232           eastus

Verify that Nginx is running

Using the IP address of the Anjuna Confidential Container, make a request to Nginx:

  • Google Cloud

  • Microsoft Azure

$ export IP_ADDRESS=$(anjuna-gcp-cli instance describe \
    --instance=${INSTANCE_NAME} \
    --show-ip | egrep -m 1 -i "AccessConfig: External NAT IpAddr:\s*[0-9]+" \
    | sed -E 's/.*IpAddr\:\s*([0-9.]+).*/\1/')
$ curl ${IP_ADDRESS}:80
$ export IP_ADDRESS=$(anjuna-azure-cli instance describe \
    --resource-group myResourceGroup --name ${INSTANCE_NAME} \
    --json --query publicIps | grep -oE "[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+" | awk 'NR==1')
$ curl ${IP_ADDRESS}:80

You should see HTML output similar to the following:

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Congratulations! Nginx is running and is accessible from outside the Anjuna Confidential Container.

View application logs

Once the application is running, you can view its logs using the Anjuna CLI.

  • Google Cloud

  • Microsoft Azure

To view the logs, you will need to use an account that has the Logs Viewer role. Once you are logged in to this account, the following command will pull the logs from Cloud Logging and display them on your terminal:

$ anjuna-gcp-cli instance describe \
    --instance=${INSTANCE_NAME} \
    --logs --tail

ANJ-ENCLAVE: Container setup finished
ANJ-ENCLAVE: Launching command /usr/sbin/nginx
2023/02/17 22:07:34 [notice] 1#1: using the "epoll" event method
2023/02/17 22:07:34 [notice] 1#1: nginx/1.23.3
2023/02/17 22:07:34 [notice] 1#1: built by gcc 10.2.1 20210110 (Debian 10.2.1-6)

You can also log on to the web console to view the logs: Cloud Logging Console.

If you did not specify a service account earlier, the logs will go to the serial console instead. To view the serial console logs, use anjuna-gcp-cli instance describe --instance=${INSTANCE_NAME} --serial --tail

You can tail the application logs with the following command:

$ anjuna-azure-cli instance log --tail \
  --name ${INSTANCE_NAME} \
  --resource-group myResourceGroup

Terminate the Anjuna Confidential Container

  • Google Cloud

  • Microsoft Azure

To terminate the GCP Confidential VM, run the following command:

$ anjuna-gcp-cli instance delete --instance=${INSTANCE_NAME}

To terminate the Azure Confidential VM, run the following command:

$ anjuna-azure-cli instance delete --name ${INSTANCE_NAME} --resource-group myResourceGroup