How to create HPC Cluster based on Azure CycleCloud

From PUBLIC-WIKI
Revision as of 16:14, 11 July 2019 by Eyales (talk | contribs)
Jump to navigation Jump to search

Installing Azure CLI

  • Login to the machine using privileged account.
  • Download the latest build of Azure CLI:
  • Windows download instruction and location:
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-windows?view=azure-cli-latest
  • Linux download instruction and location:
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-linux?view=azure-cli-latest

Login to the Azure subscription

  • Run the command below to login to the Azure subscription:
az login
  • List available subscriptions:
az account list --output table
  • Change the context to a specific Azure subscription:
az account set --subscription "My Subscription"
Note: Replace "My Subscription” with the relevant subscription name
  • Run the command below to verify the currently selected Azure subscription:
az account show

Configure CycleCloud pre-requirements

  • From a command prompt, run the command below to configure a service principal:
az ad sp create-for-rbac --name CycleCloudApp-MySubscription --years 1
Note: Replace CycleCloudApp-MySubscription with a unique name, relevant to the target Azure subscription
  • Document the command output (appId, displayName, name, password, tenant)
  • From Linux command prompt, run the command below to generate SSH key pair:
ssh-keygen -f ~/.ssh/id_rsa -m pem -t rsa -N "" -b 4096

Deploy the Azure CycleCloud

  • Login to the Azure Portal:
https://portal.azure.com/
  • From the left pane click on Subscriptions -> from the main pane, click on “global subscriptions filter” -> make sure the target Azure subscription is the only subscription selected
  • Inside the top search pane, write “Marketplace”
  • Inside the “Search the Marketplace” field, write “Azure CycleCloud”
  • Click on Create:
  • Subscription: Select the target subscription name
  • Resource group: Select “Create new”
  • Virtual machine name: Specify here the Azure CycleCloud VM name
  • Region: Specify a region close to your location (such as West Europe)
Note: For full list of regions, see:
https://azure.microsoft.com/en-us/global-infrastructure/locations/
  • Availability options: Leave the default settings
  • Image: Leave the default “Azure CycleCloud”
  • Size: For small HPC cluster, you may select Standard D2s v3 (2 vCPU, 8GB), otherwise, leave the default Standard D4s v3 (4 vCPU, 16GB)
  • Authentication type: SSH public key
  • Username: Specify here a username for the cluster admin account (for SSH login to the machine)
Note: Remember to document this detail for future use
  • SSH public key: Paste here the content of the SSH public key (id_rsa.pub) previously created
  • Click Next: Disks
  • OS disk type – For test purpose or small scale cluster, select “Standard SSD”
For production or large scale cluster, select “Premium SSD”
  • Click Next: Networking
  • Virtual network: Leave default settings
  • Subnet: Leave default settings
  • Configure network security group: Click on “Create new” -> leave the default settings -> click OK
  • Click Next: Management
  • Boot diagnostics: Select Off
  • Click Next: Advanced -> click Next: Tags
  • Name: Specify here Project
  • Value: Specify here Azure subscription name
  • Click Next: Review + create
  • Click on Create
  • Wait for the deployment process to complete
  • When the deployment process completes, click on Go to resource
  • From the newly created VM “Overview” page, locate the Public IP address and document it.