Troubleshooting

This section addresses possible warnings or errors you might encounter while using the Anjuna tools.

anjuna-azure-cli instance create fails with error code DeploymentFailed

Microsoft Azure deployments can fail with an error similar to the following:

{
  "status": "Failed",
  "error": {
    "code": "DeploymentFailed",
    "target": "/subscriptions/137014eb-bb41-4f84-9939-35393af2f98a/resourceGroups/resgroup/providers/Microsoft.Resources/deployments/vm_deploy_eWZv8jdj3ht4W0rrVZJd8kN1PEV8T5AN",
    "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.",
    "details": [
      {
        "code": "ResourceDeploymentFailure",
        "target": "/subscriptions/137014eb-bb41-4f84-9939-35393af2f98a/resourceGroups/resgroup/providers/Microsoft.Compute/virtualMachines/podvm-test-pod",
        "message": "The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.",
        "details": [
          {
            "code": "InternalDiskManagementError",
            "message": "An unexpected error occurred while processing disk. Please retry later."
          }
        ]
      }
    ]
  }
}

When this occurs, follow this Azure documentation to view details about the specific operations that failed and how to resolve them.

Sometimes, this error can occur after the VM was already created. This VM will need to be deleted manually. You can run the az deployment group delete command, using the target reported in the error message to identify the deployment to delete.

anjuna-gcp-cli disk create fails when running in a Docker container

When running anjuna-gcp-cli disk create in a Docker container, you may see the following error:

# A Docker socket error about /var/run/docker.sock
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

This error occurs because the environment within a Docker container is different from the host machine’s environment. anjuna-gcp-cli requires access to the Docker socket (/var/run/docker.sock) to pull and process the container images.

To resolve this error, mount /var/run/docker.sock into your Docker container that will run anjuna-gcp-cli. For example, use the flag -v /var/run/docker.sock:/var/run/docker.sock in your docker run command.

The Confidential Container was started with a non-empty disk

You may see the following error from your instance logs:

ANJ-ENCLAVE: The Confidential Container was started with a non-empty disk.
Please follow the directions in the Anjuna documentation to delete the disk and create a new instance of the Confidential Container.
Stopping the Confidential Container.

This error indicates the instance re-used an existing OS disk, which results in a different initial state and could compromise the security of the deployment.

This occurs when you restart the instance using the cloud service provider CLI or console. Occasionally, the cloud service provider will also restart your instances for maintenance, causing the same error.

To resolve the error, delete the old instance using anjuna-azure-cli instance delete or anjuna-gcp-cli instance delete, and create a new instance of your Confidential Container.

Errors when deleting and re-creating a cloud resource

You may see errors that indicate you are creating a resource (like a VM or storage bucket) with a duplicate name, even after you have just deleted the old one. For example: The storage account name '<name>' is already taken.

As global distributed services, Microsoft Azure and Google Cloud have eventually-consistent updates, meaning changes to cloud resources may not be immediately available.

It may take several seconds or minutes before your previous update takes effect. Wait, and then try again.

In some cases, resource deletions may take multiple days, due to retention policy. For example, this StackOverflow answer describes a scenario where an Azure storage account takes multiple days to be deleted.