/ azure

Packer로 Azure VM 이미지 만들기 #2 - Powershell로 실행하고 최신 이미지 Tag

Packer로 Azure 이미지 만들기 #1 - Packer 템플릿에서는 Packer의 기본을 알아봤는데 이제 실제 사례를 살펴보자.

상황설명

수십 또는 수백 대의 VM을 관리하는 상황에서 VMSS(Virtual Machin Scale Set)을 사용하거나 VM의 베이스 이미지를 만들어서 관리하는 상황을 생각해보자. VM의 설정이나 애플리케이션이 변경되면 Packer 템플릿을 수정하고 새로 이미지를 만들어 업데이트 하려고 한다. 이미 VM 이미지가 있고 그 이미지 바탕으로 업데이트 해야한다. Packer가 실행되어 업데이트가 적용된 새로운 이미지를 만든다. 새로운 이미지에는 Tag를 version:latest로 달아서 Azure CLI, PowerShell 등의 코드에서 최신 이미지를 찾을 수 있도록 해준다.

소스코드

전체 소스코드를 Github에 올려놨다. 주요 파일에 대한 설명은 아래와 같다.

  • packer.exe 윈도우 버전 packer 실행 파일. packer.io에서 다운로드
  • build-baseimage.ps1 파워쉘로 작성한 실행 스크립트
  • baseimage.json Packer 템플릿
  • powershell/update-latest-imagename.ps1 Packer의 post-processor가 최신이미지 이름을 파라미터와 함께 호출해서 최신이미지의 Tag 적용
  • variable.json Azure의 Service Priciple과 베이스 이미지가 만들어질 리소스 그룹 이름

여기서는 Windows VM을 대상으로 했다. 따라서 몇 가지 스크립트들이 PowerShell로 작성되었다.

소스코드

VSTS의 Git Repository에서 소스를 다운 받는다. 이 소소코드는 Github에 있다.
1-get-source

Powershell로 로컬에서 실행하기

VSTS를 사용해서 자동화 하기 전에 build-baseimage.ps1 스크립트를 이용해서 실행해보자. AzureRM Powershell 모듈을 가져오고 Azure에 로그인 한다. 이때 로그인 창이 뜬다. Tag version:latest를 이용해서 최신 이미지를 찾는다. Packer를 실행시켜 새로운 이미지를 만든다.

# Azure RM import
Import-Module -Name AzureRM
# Get variable for service principle
$variables = Get-Content '.\variable.json' | Out-String | ConvertFrom-Json
$SecurePassword = $variables.client_secret | ConvertTo-SecureString -AsPlainText -Force
$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $variables.client_id, $SecurePassword
# login to Azure
Connect-AzureRmAccount -Credential $cred -Tenant $variables.tenant_id -ServicePrincipal

# Get Latest base image 
$latestimagename = (Get-AzureRmResource -ODataQuery "`$filter=tagname eq 'version' and tagvalue eq 'latest'").Name

$cmdPath = "$PSScriptRoot\packer.exe"
$cmdArgList = @(
	"build",
	# "validate",
	# "-debug",
	"-var-file=.\variable.json",
	"-var","custom_managed_image_name=$latestimagename"
	".\bdm-baseimage.json"
)

Write-Output "image name: $latestimagename"
Write-Output "cmd: $cmdPath"
Write-Output "arg: $cmdArgList"

& $cmdPath $cmdArgList

Packer 템플릿은 아래와 같다. Provisioner에 추가되는 설정을 넣어주거나 chocolatey로 툴을 설치할 수 있다. 여기서는 chocolatey를 설치하는 스크립트를 실행하고 Sysprep 하는 스크립트가 실행된다.

{
    "variables": {
        "client_id": "{{env `ARM_CLIENT_ID`}}",
        "client_secret": "{{env `ARM_CLIENT_SECRET`}}",
        "tenant_id": "{{env `ARM_TENANT_ID}}",
        "subscription_id": "{{env `ARM_SUBSCRIPTION_ID`}}",
        "object_id": "{{env `ARM_OJBECT_ID`}}",
        "custom_managed_image_resource_group_name": "{{env `ARM_CUSTOM_IMAGE_RG_NAME`}}",
        "custom_managed_image_name": "{{env `ARM_CUSTOM_IMAGE_NAME`}}",
        "image_name": "base-image-{{isotime \"2006-01-02\"}}_{{isotime \"03-04-05\"}}"
    },
    "builders": [
        {
            "type": "azure-arm",

            "client_id": "{{user `client_id`}}",
            "client_secret": "{{user `client_secret`}}",
            "tenant_id": "{{user `tenant_id`}}",
            "subscription_id": "{{user `subscription_id`}}",
            "object_id": "{{user `object_id`}}",

            "managed_image_resource_group_name": "{{user `custom_managed_image_resource_group_name`}}",
            "managed_image_name": "{{user `image_name`}}", 

            "location": "Korea Central",
            "vm_size": "Standard_D2s_v3",

            "os_type": "Windows",
            "custom_managed_image_resource_group_name": "{{user `custom_managed_image_resource_group_name`}}",
            "custom_managed_image_name": "{{user `custom_managed_image_name`}}",

            "communicator": "winrm",
            "winrm_use_ssl": "true",
            "winrm_insecure": "true",
            "winrm_timeout": "3m",
            "winrm_username": "ansibleuser"
        }
    ],
    "provisioners": [
        {
            "type": "powershell", 
            "script": "./powershell/install-chocolatey.ps1"
        },
        {
            "type": "powershell",
            "inline": [
                "if( Test-Path $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml ){ rm $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml -Force}",
                "& $env:SystemRoot\\System32\\Sysprep\\Sysprep.exe /oobe /generalize /quiet /quit",
                "while($true) { $imageState = Get-ItemProperty HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup\\State | Select ImageState; if($imageState.ImageState -ne 'IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE') { Write-Output $imageState.ImageState; Start-Sleep -s 10  } else { break } }"
            ]
        }
    ],
    "post-processors": [
        {
            "type": "shell-local",
            "execute_command": ["powershell.exe", "{{.Script}} {{user `image_name`}}"],
            "script": ".\\powershell\\update-latest-imagename.ps1"
        }
    ]
}

post-processor에는 update-latest-imagename.ps1 라는 스크립트를 실행하게 되어 있는데 최신이미지에 Tag version:latest를 적용해서 다음번 빌드를 준비한다.

Param(
    [string]$newimagename
)

# Azure RM import
Import-Module -Name AzureRM
# Get variable for service principle
$variables = Get-Content '..\variable.json' | Out-String | ConvertFrom-Json
$SecurePassword = $variables.client_secret | ConvertTo-SecureString -AsPlainText -Force
$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $variables.client_id, $SecurePassword
# login to Azure
Connect-AzureRmAccount -Credential $cred -Tenant $variables.tenant_id -ServicePrincipal

#update latest-imagename.txt
Write-Output "New Image name: $newimagename"

#remove all tag in the resoruce group 
$current = (Get-AzureRmResource -ODataQuery "`$filter=tagname eq 'version' and tagvalue eq 'latest'")
Set-AzureRmResource -Tag @{ } -ResourceId $current.ResourceId -Force

#set tag to the new image  
$r = Get-AzureRmResource | Where-Object ResourceName -eq $newimagename
Set-AzureRmResource -Tag @{ version="latest"} -ResourceId $r.ResourceId -Force

실행결과

> .\build-baseimage.ps1

Account          : 220b9c3c-fac5-4df4-bf47-3f296123a447
SubscriptionName : Microsoft Azure Internal Consumption
SubscriptionId   : e47f0bbb-cd59-41dc-86b7-2e2432d53604
TenantId         : 72f988bf-86f1-41af-91ab-2dcd01531b47
Environment      : AzureCloud

image name: base-image-2018-06-07_05-03-53
cmd: C:\Users\iloh\source\packer-azure-vsts\packer.exe
arg: build -var-file=.\variable.json -var custom_managed_image_name=base-image-2018-06-07_05-03-53 .\baseimage.json

azure-arm output will be in this color.

==> azure-arm: Running builder ...
    azure-arm: Creating Azure Resource Manager (ARM) client ...
    azure-arm: You have provided Object_ID which is no longer needed, azure packer builder determines this dynamically from the authentication token
==> azure-arm: Creating resource group ...
==> azure-arm:  -> ResourceGroupName : 'packer-Resource-Group-rbggjbzqf5'
==> azure-arm:  -> Location          : 'Korea Central'
==> azure-arm:  -> Tags              :
==> azure-arm: Validating deployment template ...
==> azure-arm:  -> ResourceGroupName : 'packer-Resource-Group-rbggjbzqf5'
==> azure-arm:  -> DeploymentName    : 'pkrdprbggjbzqf5'
==> azure-arm: Deploying deployment template ...
==> azure-arm:  -> ResourceGroupName : 'packer-Resource-Group-rbggjbzqf5'
==> azure-arm:  -> DeploymentName    : 'kvpkrdprbggjbzqf5'
==> azure-arm: Getting the certificate's URL ...
==> azure-arm:  -> Key Vault Name        : 'pkrkvrbggjbzqf5'
==> azure-arm:  -> Key Vault Secret Name : 'packerKeyVaultSecret'
==> azure-arm:  -> Certificate URL       : 'https://pkrkvrbggjbzqf5.vault.azure.net/secrets/packerKeyVaultSecret/5038000621384cc1ad6a15603abb32df'
==> azure-arm: Setting the certificate's URL ...
==> azure-arm: Validating deployment template ...
==> azure-arm:  -> ResourceGroupName : 'packer-Resource-Group-rbggjbzqf5'
==> azure-arm:  -> DeploymentName    : 'pkrdprbggjbzqf5'
==> azure-arm: Deploying deployment template ...
==> azure-arm:  -> ResourceGroupName : 'packer-Resource-Group-rbggjbzqf5'
==> azure-arm:  -> DeploymentName    : 'pkrdprbggjbzqf5'
==> azure-arm: Getting the VM's IP address ...
==> azure-arm:  -> ResourceGroupName   : 'packer-Resource-Group-rbggjbzqf5'
==> azure-arm:  -> PublicIPAddressName : 'pkriprbggjbzqf5'
==> azure-arm:  -> NicName             : 'pkrnirbggjbzqf5'
==> azure-arm:  -> Network Connection  : 'PublicEndpoint'
==> azure-arm:  -> IP Address          : '52.231.68.202'
==> azure-arm: Waiting for WinRM to become available...
    azure-arm: #< CLIXML
    azure-arm: WinRM connected.
    azure-arm: <Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04"><Obj S="progress" RefId="0"><TN RefId="0"><T>System.Management.Automation.PSCustomObject</T><T>System.Object</T></TN><MS><I64 N="SourceId">1</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj><Obj S="progress" RefId="1"><TNRef RefId="0" /><MS><I64 N="SourceId">1</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj></Objs>
==> azure-arm: Connected to WinRM!
==> azure-arm: Provisioning with Powershell...
==> azure-arm: Provisioning with powershell script: ./powershell/install-chocolatey.ps1
    azure-arm: Getting latest version of the Chocolatey package for download.
    azure-arm: Getting Chocolatey from https://chocolatey.org/api/v2/package/chocolatey/0.10.11.
    azure-arm: Downloading 7-Zip commandline tool prior to extraction.
    azure-arm: Extracting C:\Users\packer\AppData\Local\Temp\chocolatey\chocInstall\chocolatey.zip to C:\Users\packer\AppData\Local\Temp\chocolatey\chocInstall...
    azure-arm: Installing chocolatey on this machine
    azure-arm: Creating ChocolateyInstall as an environment variable (targeting 'Machine')
    azure-arm:   Setting ChocolateyInstall to 'C:\ProgramData\chocolatey'
    azure-arm: WARNING: It's very likely you will need to close and reopen your shell
    azure-arm:   before you can use choco.
    azure-arm: Restricting write permissions to Administrators
    azure-arm: We are setting up the Chocolatey package repository.
    azure-arm: The packages themselves go to 'C:\ProgramData\chocolatey\lib'
    azure-arm:   (i.e. C:\ProgramData\chocolatey\lib\yourPackageName).
    azure-arm: A shim file for the command line goes to 'C:\ProgramData\chocolatey\bin'
    azure-arm:   and points to an executable in 'C:\ProgramData\chocolatey\lib\yourPackageName'.
    azure-arm:
    azure-arm: Creating Chocolatey folders if they do not already exist.
    azure-arm:
    azure-arm: WARNING: You can safely ignore errors related to missing log files when
    azure-arm:   upgrading from a version of Chocolatey less than 0.9.9.
    azure-arm:   'Batch file could not be found' is also safe to ignore.
    azure-arm:   'The system cannot find the file specified' - also safe.
    azure-arm: WARNING: Not setting tab completion: Profile file does not exist at
    azure-arm: 'C:\Users\packer\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1'.
    azure-arm: Chocolatey (choco.exe) is now ready.
    azure-arm: You can call choco from anywhere, command line or powershell by typing choco.
    azure-arm: Run choco /? for a list of functions.
    azure-arm: You may need to shut down and restart powershell and/or consoles
    azure-arm:  first prior to using choco.
    azure-arm: Ensuring chocolatey commands are on the path
    azure-arm: Ensuring chocolatey.nupkg is in the lib folder
==> azure-arm: Provisioning with Powershell...
==> azure-arm: Provisioning with powershell script: C:\Users\iloh\AppData\Local\Temp\packer-powershell-provisioner392428847
    azure-arm: IMAGE_STATE_COMPLETE
    azure-arm: IMAGE_STATE_UNDEPLOYABLE
    azure-arm: IMAGE_STATE_UNDEPLOYABLE
    azure-arm: IMAGE_STATE_UNDEPLOYABLE
    azure-arm: IMAGE_STATE_UNDEPLOYABLE
    azure-arm: IMAGE_STATE_UNDEPLOYABLE
    azure-arm: IMAGE_STATE_UNDEPLOYABLE
    azure-arm: IMAGE_STATE_UNDEPLOYABLE
    azure-arm: IMAGE_STATE_UNDEPLOYABLE
==> azure-arm: Querying the machine's properties ...
==> azure-arm:  -> ResourceGroupName : 'packer-Resource-Group-rbggjbzqf5'
==> azure-arm:  -> ComputeName       : 'pkrvmrbggjbzqf5'
==> azure-arm:  -> Managed OS Disk   : '/subscriptions/e47f0bbb-cd59-41dc-86b7-2e239d536c04/resourceGroups/packer-Resource-Group-rbggjbzqf5/providers/Microsoft.Compute/disks/pkrosrbggjbzqf5'
==> azure-arm: Querying the machine's additional disks properties ...
==> azure-arm:  -> ResourceGroupName : 'packer-Resource-Group-rbggjbzqf5'
==> azure-arm:  -> ComputeName       : 'pkrvmrbggjbzqf5'
==> azure-arm: Powering off machine ...
==> azure-arm:  -> ResourceGroupName : 'packer-Resource-Group-rbggjbzqf5'
==> azure-arm:  -> ComputeName       : 'pkrvmrbggjbzqf5'
==> azure-arm: Capturing image ...
==> azure-arm:  -> Compute ResourceGroupName : 'packer-Resource-Group-rbggjbzqf5'
==> azure-arm:  -> Compute Name              : 'pkrvmrbggjbzqf5'
==> azure-arm:  -> Compute Location          : 'Korea Central'
==> azure-arm:  -> Image ResourceGroupName   : 'BaseImageGroup'
==> azure-arm:  -> Image Name                : 'base-image-2018-06-20_05-01-33'
==> azure-arm:  -> Image Location            : 'koreacentral'
==> azure-arm: Deleting resource group ...
==> azure-arm:  -> ResourceGroupName : 'packer-Resource-Group-rbggjbzqf5'
==> azure-arm:
==> azure-arm: The resource group was created by Packer, deleting ...
==> azure-arm: Deleting the temporary OS disk ...
==> azure-arm:  -> OS Disk : skipping, managed disk was used...
==> azure-arm: Deleting the temporary Additional disk ...
==> azure-arm:  -> Additional Disk : skipping, managed disk was used...
==> azure-arm: Running post-processor: shell-local
==> azure-arm (shell-local): Running local shell script: .\powershell\update-latest-imagename.ps1
    azure-arm (shell-local):
    azure-arm (shell-local):
    azure-arm (shell-local): Account          : 220b9c3c-fac5-4df4-bf47-3f212ecea447
    azure-arm (shell-local): SubscriptionName : Microsoft Azure Internal Consumption
    azure-arm (shell-local): SubscriptionId   : e47f0bbb-cd59-41dc-86b7-2e223436c04
    azure-arm (shell-local): TenantId         : 72f988bf-86f1-41af-91ab-2d42121db47
    azure-arm (shell-local): Environment      : AzureCloud
    azure-arm (shell-local):
    azure-arm (shell-local): New Image name: base-image-2018-06-20_05-01-33
    azure-arm (shell-local):
    azure-arm (shell-local): Name              : base-image-2018-06-07_05-03-53
    azure-arm (shell-local): ResourceId        : /subscriptions/e4123bbb-cd59-41dc-86b7-2e239d536c04/resourceGroups/BaseImageGroup/providers/Microsoft.Compute/images/base-image-2018-06-07_05-03-53
    azure-arm (shell-local): ResourceName      : base-image-2018-06-07_05-03-53
    azure-arm (shell-local): ResourceType      : Microsoft.Compute/images
    azure-arm (shell-local): ResourceGroupName : BaseImageGroup
    azure-arm (shell-local): Location          : koreacentral
    azure-arm (shell-local): SubscriptionId    : e47f0bbb-cd59-41dc-86b7-2e239d536c04
    azure-arm (shell-local): Tags              : {}
    azure-arm (shell-local): Properties        : @{sourceVirtualMachine=; storageProfile=; provisioningState=Succeeded}
    azure-arm (shell-local):
    azure-arm (shell-local):
    azure-arm (shell-local): Name              : base-image-2018-06-20_05-01-33
    azure-arm (shell-local): ResourceId        : /subscriptions/e4123bbb-cd59-41dc-8627-2e239d536c04/resourceGroups/BaseImageGroup/providers/Microsoft.Compute/images/base-image-2018-06-20_05-01-33
    azure-arm (shell-local): ResourceName      : base-image-2018-06-20_05-01-33
    azure-arm (shell-local): ResourceType      : Microsoft.Compute/images
    azure-arm (shell-local): ResourceGroupName : BaseImageGroup
    azure-arm (shell-local): Location          : koreacentral
    azure-arm (shell-local): SubscriptionId    : e47f0bbb-cd59-41dc-86b7-2e239d536c04
    azure-arm (shell-local): Tags              : {version}
    azure-arm (shell-local): Properties        : @{sourceVirtualMachine=; storageProfile=; provisioningState=Succeeded}
    azure-arm (shell-local):
    azure-arm (shell-local):
    azure-arm (shell-local):
Build 'azure-arm' finished.

==> Builds finished. The artifacts of successful builds are:
--> azure-arm: Azure.ResourceManagement.VMImage:

ManagedImageResourceGroupName: BaseImageGroup
ManagedImageName: base-image-2018-06-20_05-01-33
ManagedImageLocation: koreacentral

--> azure-arm:

Packer가 실행되어 새로운 이미지가 만들어진다. 새로 생성된 이미지에 Tag도 정확히 적용된 걸 확인했다. Packer가 실행되는 동안 Azure Portal을 열어보면 Packer가 만든 리소스가 만들어지고 삭제되는 모습을 볼 수 있다.
packer-portal

로컬 PowerShell에서 잘 작동하니 VSTS(Visual Studio Team Services)에 올려서 자동화 해보자. 다음편 Packer로 Azure VM 이미지 만들기 #2 - VSTS로 자동화 하기