Skip to content

Z‐AndriiHandoverNotes

Jump to bottom
akolensky edited this page Sep 19, 2024 · 6 revisions

Boot Diagnostics in Terraform for Azure Virtual Machines

Boot Diagnostics is a feature in Azure that helps capture logs and screenshots during the boot process of a Virtual Machine (VM). This is particularly useful when diagnosing issues where the VM may not be reachable via remote access, such as SSH or RDP.

In Terraform, the boot_diagnostics block allows you to configure boot diagnostics for Azure VMs, providing either a managed storage account or specifying an existing storage account URI.

Configuration

You can enable boot diagnostics in Terraform for an Azure VM with a managed storage account using the following configuration:

boot_diagnostics {
  storage_account_uri = null
}

These changes can be found in any VM related terraform script such as templates/workspace_services/guacamole/user_resources/guacamole-azure-linuxvm and templates/workspace_services/guacamole/user_resources/guacamole-azure-windowsvm.

By setting storage_account_uri = null, Terraform instructs Azure to use a managed storage account, meaning the platform automatically handles the storage for diagnostic data. You don’t need to create or manage a separate storage account.

Utilising Boot Diagnostics

Once enabled, Boot Diagnostics gathers essential boot-time information, such as:

  • Console Logs: Captures diagnostic messages from the VM's operating system during the boot process.
  • Screenshots: Provides graphical screenshots of the VM, which are particularly useful for diagnosing issues with GUI-based systems.

This information is accessible via the Azure Portal, where it can be viewed to troubleshoot problems with VM startup or connectivity.

Benefits of Boot Diagnostics

  1. Enhanced Troubleshooting: When a VM becomes unresponsive or fails to start, Boot Diagnostics offers insight into where the issue lies, whether at the operating system level or a misconfiguration during startup.

  2. Minimal Management Overhead: By using a managed storage account (via storage_account_uri = null), you avoid the need to provision and maintain a separate storage account for diagnostics data.

  3. Supports Both Windows and Linux: Boot Diagnostics captures relevant information regardless of the operating system, making it a versatile feature for various workloads.

  4. Non-Intrusive: As the diagnostics data is collected during the boot process, it does not interfere with the running VM or its performance post-boot.

Boot Diagnostics can be an essential tool for ensuring smooth operations in production environments, offering critical insights when things go wrong during the boot phase.

Encryption at Host for Azure Virtual Machines

Encryption at Host is an advanced security feature that ensures your data is encrypted before it is stored on the underlying hardware that hosts your Azure Virtual Machines (VMs). This feature provides an extra layer of protection, ensuring that both data-at-rest and data-in-use are encrypted without compromising performance.

In Terraform, enabling Encryption at Host for an Azure VM is straightforward using the encryption_at_host_enabled argument in the VM configuration.

Configuration Example

To enable Encryption at Host for a VM in Terraform, you would use the following configuration:

resource "azurerm_virtual_machine" "example" {
  # Other VM configurations...

  encryption_at_host_enabled = true
}

Setting encryption_at_host_enabled = true ensures that all data is encrypted at the host level, with no need for additional storage account configurations or management.

Utilising Encryption at Host

When Encryption at Host is enabled, all VM disks—both operating system and data disks—are encrypted by Azure. This ensures that data is protected throughout the entire lifecycle of the VM, providing security at the hardware level. Encryption is handled automatically by the platform, and the process is completely transparent to users.

You can check if the VM supports Encryption at Host by looking at the VM size and region, as not all VM types and locations support this feature.

Benefits of Encryption at Host

  1. Enhanced Data Security: Data is encrypted before it is written to the host's physical storage, ensuring that it remains protected from physical threats, such as unauthorised access to the hardware itself.

  2. Compliance: Many industries have strict compliance requirements around data encryption. Encryption at Host helps meet those standards by ensuring data is encrypted throughout its lifecycle, from processing to storage.

  3. No Performance Impact: As encryption is handled by Azure's infrastructure, there is no noticeable performance degradation on the VM workloads. The encryption is performed at the host level without impacting VM operations.

  4. No Extra Management Overhead: Enabling Encryption at Host requires no additional management, and the encryption keys are managed automatically by Azure, removing the need for you to maintain or rotate keys.

When to Use Encryption at Host

This feature is particularly useful in scenarios where data protection is paramount, such as:

  • Workloads handling sensitive information.
  • Organisations with strict regulatory or compliance needs.
  • Environments where enhanced security is necessary, without the overhead of managing custom encryption keys.

By enabling Encryption at Host in Terraform with a simple configuration, you can easily enhance the security posture of your Azure VMs while maintaining operational efficiency.

Encryption at Host for Azure Virtual Machines with Custom Managed Keys

In case we wish to keep the encryption keys history and better control over them, we can utilise Key Vault Keys along with Disk Encryption Set.

In theory this is the set of steps.

  1. Create the Key Vault and Key
# Azure Key Vault

resource "azurerm_key_vault" "key_vault" {
  name                        = "workspace-kv"
  location                    = var.region
  resource_group_name         = azurerm_resource_group.rg.name
  sku_name                    = "standard"
  tenant_id                   = data.azurerm_client_config.current.tenant_id
  soft_delete_enabled         = true
  purge_protection_enabled    = true

  access_policy {
    tenant_id = data.azurerm_client_config.current.tenant_id
    object_id = data.azurerm_client_config.current.object_id

    key_permissions = [
      "Get",
      "List",
      "Create",
      "Update",
      "Delete",
      "Recover",
      "Backup",
      "Restore",
    ]

    secret_permissions = [
      "Get",
      "List",
      "Set",
      "Delete",
      "Recover",
      "Backup",
      "Restore",
    ]
  }
}

data "azurerm_client_config" "current" {}

# KV Encryption Key Key

resource "azurerm_key_vault_key" "encryption_key" {
  name         = "disk-encryption-key"
  key_vault_id = azurerm_key_vault.key_vault.id
  key_type     = "RSA"
  key_size     = 2048

  lifecycle {
    prevent_destroy = true
  }
}
  1. Create Disk Encryption Set and assign RBAC Role for Disk Encryption The Disk Encryption Set will use the key stored in Key Vault for encrypting the disks. Azure RBAC role must be assigned to allow the Disk Encryption Set access to the Key Vault.
# Disk Encryption Set

resource "azurerm_disk_encryption_set" "des" {
  name                = "workspace-des"
  location            = var.region
  resource_group_name = azurerm_resource_group.rg.name
  key_vault_id        = azurerm_key_vault.key_vault.id
  key_vault_key_id    = azurerm_key_vault_key.encryption_key.id

  identity {
    type = "SystemAssigned"
  }
}

# Role Assignment for Disk Encryption Set

resource "azurerm_role_assignment" "key_vault_role" {
  scope                = azurerm_key_vault.key_vault.id
  role_definition_name = "Contributor"
  principal_id         = azurerm_disk_encryption_set.des.identity[0].principal_id

  depends_on = [
    azurerm_disk_encryption_set.des
  ]
}
  1. Create VM with CMK

The VM is created with disk encryption at the host, using the Disk Encryption Set for both the OS and data disks.

# Virtual Network
resource "azurerm_virtual_network" "vnet" {
  name                = "workspace-vnet"
  location            = var.region
  resource_group_name = azurerm_resource_group.rg.name
  address_space       = ["10.2.3.0/24"]

  subnet {
    name           = "${lower(var.virtual_machine_name)}-subnet"
    address_prefix = "10.2.3.0/24"
  }
}

# Public IP Address
resource "azurerm_public_ip" "public_ip" {
  name                = "workspace-ip"
  location            = var.region
  resource_group_name = azurerm_resource_group.rg.name
  allocation_method   = "Dynamic"
  sku                 = "Basic"
}

# Network Interface
resource "azurerm_network_interface" "nic" {
  name                = "workspace-nic"
  location            = var.region
  resource_group_name = azurerm_resource_group.rg.name

  ip_configuration {
    name                          = "internal"
    subnet_id                     = azurerm_virtual_network.vnet.subnet[0].id
    private_ip_address_allocation = "Dynamic"
    public_ip_address_id          = azurerm_public_ip.public_ip.id
  }
}

# Virtual Machine
resource "azurerm_virtual_machine" "vm" {
  name                  = var.virtual_machine_name
  location              = var.region
  resource_group_name   = azurerm_resource_group.rg.name
  network_interface_ids = [azurerm_network_interface.nic.id]
  vm_size               = var.vm_size

  os_profile {
    computer_name  = var.virtual_machine_name
    admin_username = "adminUserIsTest"
    admin_password = "penpineapplepen"
  }

  os_profile_windows_config {}

  security_profile {
    encryption_at_host_enabled = true
  }

  storage_os_disk {
    name              = "${var.virtual_machine_name}-osdisk"
    caching           = "ReadWrite"
    create_option     = "FromImage"
    managed_disk_type = "Premium_LRS"

    disk_encryption_set_id = azurerm_disk_encryption_set.des.id
  }

  storage_data_disk {
    name              = "workspace-vm01-datadisk1"
    lun               = 0
    caching           = "ReadWrite"
    create_option     = "Empty"
    disk_size_gb      = 128
    managed_disk_type = "Premium_LRS"

    disk_encryption_set_id = azurerm_disk_encryption_set.des.id
  }

  storage_image_reference {
    publisher = "MicrosoftWindowsServer"
    offer     = "WindowsServer"
    sku       = "2019-Datacenter"
    version   = "latest"
  }

  depends_on = [
    azurerm_role_assignment.key_vault_role
  ]
}

Airlock Notifier

The main goal of the airlock notifier is to send out emails based on the airlock review process. The current spike document tries to address two things:

  1. Runtime errors
  2. Email Senders (SMTP and others)

With current setup, the logic app deployes with runtime errors, and although it sucessfully deployes on TRE portal it does not appear to function and going through Azure Portal it indicates errors in the runtime.

Runtime errors

At present Azure Logic apps support Azure Functions V4 while V3 versions are at end of support.

By default the terrafform module azurerm_logic_app_standard is deployed with default version of V3, although it does suggest end of support too.

Given these conditions, I updated function app versions and subsequently the runtime in airlock_notifier.tf, as seen in this commit linked.

**Note: The version update also requires a more recent NODE version so I updated that to the ~14 but it can be updated to most up to date version "~18" too.

When the terraform file is updated, make sure to iterate version in the porter.yaml as we are modifying the overall bundle.

Deployment

Make sure the current airlock notifier is disabled and deleted.

Once it is deleted, run the

make shared_service_bundle BUNDLE=airlock_notifier

This should deploy the new Porter bundle version, and Airlock Notifier can be redeployed with Template version updated in the Details of the Shared Service deployed.

On Azure portal, the logic app and the workflow should both have a Runtime displayed in the right corners of the Overview sections.

Email Sender (SMTP vs Outlook)

Given I did not have the necessary information for SMTP provider, I decided to trial out any other connector which would allow me to send emails to the users as long as I knew my email, therefore I chose outlook.

Once the new logic app is deployed, its workflow can be adjusted via the Developer - Designer mode to have the "Send an email (V2)" instead of SMTP, which requires an outlook connection, and creates it for you along with manual authorisation. Once the API connection is set it can be used by any other logic app as long as it is within the same resource group (I think).

The only limitation at the moment with current outlook API connection is that it requires a manual authorisation before deploying it as part of terraform.

I believe this could cause an issue related to SMTP connection, where the API resource is not deployed because it doesnt have a runtime URI referenced in connections.json in the airlock airlock_notifier folder. "connectionRuntimeUrl": "@appsetting('smtp_connection_runtime_url')" The @appsetting smtp_connection_runtime_url indicates it could be part of Logic App's Environment Variables (found under Logic Apps Settings blade on Azure Portal), because all other appSettings mentioned in connection.json can be found there and I couldn't find in codebase what step deployes this specific API connection.

As alternative, i did the steps above, where I deployed outlook API conection manually first (as part of Logic App Workflow Send Email V2), and then added it as part of connections base with functional connectionRuntimeUrl to connections.json.

There is a layered method to deploy API connections with terraform and I thought to try it with Outlook 365, assuming the NHS email domains are hosted on O365.

For now the one manual step with rest of runtime issues resolved Airlock Notifier, and I was able to get emails as necessary.

ADF Unit Test

  1. In the working directory, create a file named emp.txt to upload. You can use standard Bash commands, like cat, to create a file:

    cat > emp.txt
This is text.

    Use Ctrl+D to save your new file.

  2. To upload the new file to your Azure storage container, use the az storage blob upload command:

    az storage blob upload --account-name adfquickstartstorage --name input/emp.txt \
    --container-name adftutorial --file emp.txt --auth-mode key

    This command uploads to a new folder named input.

Create a linked service and datasets

Next, create a linked service and two datasets.

  1. Get the connection string for your storage account by using the az storage account show-connection-string command:

    az storage account show-connection-string --resource-group ADFQuickStartRG \
    --name adfquickstartstorage --key primary

  2. In your working directory, create a JSON file with this content, which includes your own connection string from the previous step. Name the file AzureStorageLinkedService.json:

    {
    "type": "AzureBlobStorage",
    "typeProperties": {
        "connectionString": "DefaultEndpointsProtocol=https;AccountName=<accountName>;AccountKey=<accountKey>;EndpointSuffix=core.windows.net"
    }
}

  3. Create a linked service, named AzureStorageLinkedService, by using the az datafactory linked-service create command:

    az datafactory linked-service create --resource-group ADFQuickStartRG \
    --factory-name ADFTutorialFactory --linked-service-name AzureStorageLinkedService \
    --properties AzureStorageLinkedService.json

  4. In your working directory, create a JSON file with this content, named InputDataset.json:

    {
    "linkedServiceName": {
        "referenceName": "AzureStorageLinkedService",
        "type": "LinkedServiceReference"
    },
    "annotations": [],
    "type": "Binary",
    "typeProperties": {
        "location": {
            "type": "AzureBlobStorageLocation",
            "fileName": "emp.txt",
            "folderPath": "input",
            "container": "adftutorial"
        }
    }
}

  5. Create an input dataset named InputDataset by using the az datafactory dataset create command:

    az datafactory dataset create --resource-group ADFQuickStartRG \
    --dataset-name InputDataset --factory-name ADFTutorialFactory \
    --properties InputDataset.json

  6. In your working directory, create a JSON file with this content, named OutputDataset.json:

    {
    "linkedServiceName": {
        "referenceName": "AzureStorageLinkedService",
        "type": "LinkedServiceReference"
    },
    "annotations": [],
    "type": "Binary",
    "typeProperties": {
        "location": {
            "type": "AzureBlobStorageLocation",
            "folderPath": "output",
            "container": "adftutorial"
        }
    }
}

  7. Create an output dataset named OutputDataset by using the az datafactory dataset create command:

    az datafactory dataset create --resource-group ADFQuickStartRG \
    --dataset-name OutputDataset --factory-name ADFTutorialFactory \
    --properties OutputDataset.json

Create and run the pipeline

Finally, create and run the pipeline.

  1. In your working directory, create a JSON file with this content named Adfv2QuickStartPipeline.json:

    {
    "name": "Adfv2QuickStartPipeline",
    "properties": {
        "activities": [
            {
                "name": "CopyFromBlobToBlob",
                "type": "Copy",
                "dependsOn": [],
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "source": {
                        "type": "BinarySource",
                        "storeSettings": {
                            "type": "AzureBlobStorageReadSettings",
                            "recursive": true
                        }
                    },
                    "sink": {
                        "type": "BinarySink",
                        "storeSettings": {
                            "type": "AzureBlobStorageWriteSettings"
                        }
                    },
                    "enableStaging": false
                },
                "inputs": [
                    {
                        "referenceName": "InputDataset",
                        "type": "DatasetReference"
                    }
                ],
                "outputs": [
                    {
                        "referenceName": "OutputDataset",
                        "type": "DatasetReference"
                    }
                ]
            }
        ],
        "annotations": []
    }
}

  2. Create a pipeline named Adfv2QuickStartPipeline by using the az datafactory pipeline create command:

    az datafactory pipeline create --resource-group ADFQuickStartRG \
    --factory-name ADFTutorialFactory --name Adfv2QuickStartPipeline \
    --pipeline Adfv2QuickStartPipeline.json

  3. Run the pipeline by using the az datafactory pipeline create-run command:

    az datafactory pipeline create-run --resource-group ADFQuickStartRG \
    --name Adfv2QuickStartPipeline --factory-name ADFTutorialFactory

    This command returns a run ID. Copy it for use in the next command.

  4. Verify that the pipeline run succeeded by using the az datafactory pipeline-run show command:

    az datafactory pipeline-run show --resource-group ADFQuickStartRG \
    --factory-name ADFTutorialFactory --run-id 00000000-0000-0000-0000-000000000000
Clone this wiki locally