First run of az-cli is slow

[lanni-energinet]

Description

For quite some time we have been impacted by poor performance due to the nature of az-cli's command and module caching on it's first run. It would seem that az-cli was not written with ephemeral runners in mind, and it doesn't look like they could change this easily. Hence why we are opening this issue here.

We can see that the issue is fixed on windows images as a simple warmup has been implemented

runner-images/images/windows/scripts/build/Install-AzureCli.ps1

Line 22 in c5c9a7c

# Warm-up Azure CLI

Can you please implement a simple warm up like on the windows runners?

From a sample of our logs we have measured the following impact
Minimum 2 s
Maximum 31 s
Average 5 s

We estimate wasting roughly 6500 minutes per month which is more than 10% of our of total usage

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 20.04
• Ubuntu 22.04
• Ubuntu 24.04
• macOS 11
• macOS 12
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• Windows Server 2019
• Windows Server 2022

Image version and build link

https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240609.1

Is it regression?

no

Expected behavior

A build-in warmup in the ubuntu image creation similar to the windows image

Actual behavior

No warmup

Repro steps

None needed

[kishorekumar-anchala]

HI @lanni-energinet ,

Thank you for bringing this issue to us, we are investigating on this issue and we will update you on this issue after our findings.

[lanni-energinet]

@kishorekumar-anchala

i've attached a proposed implementation on the PR can you take a look
#10126 (comment)

[lanni-energinet]

@kishorekumar-anchala

Can we in any way assist in the resolution of this issue. We are wasting a lot of time and money on this

[j15e]

Not sure if the proposed implementation would fix our issue, but when we run the login action on Ubuntu, it sometimes takes up to 2 minutes to simply run the first clear command like this, which is insanely long for such a basic task:

[kishorekumar-anchala]

@lanni-energinet , We are considering your workflow and doing changes . we're facing the some issue , we'll get back to you Asap. THANK YOU !

[adelinn]

It would be awesome if this could get fixed so az cli would run nice and quick.

[lanni-energinet]

@kishorekumar-anchala Any progress to report?

[illrill]

Feeling this pain too. Basically every one of our jobs begin with azure/login@v2, which annoyingly always takes between 10 and 30 seconds.

[lanni-energinet]

@kishorekumar-anchala Any progress to report?

[kishorekumar-anchala]

HI @lanni-energinet ,

By implementing this we're getting permission error . it would be helpful if you provide repro steps . thank you for your support !

==> azure-arm.build_image:     with open(self.config_path, 'w', encoding=CONFIG_FILE_ENCODING) as configfile:
==> azure-arm.build_image:          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
==> azure-arm.build_image: PermissionError: [Errno 13] Permission denied: '/home/root/.azure/config'
==> azure-arm.build_image: ToolVersionNode 'Azure CLI' has empty version
==> azure-arm.build_image: At /imagegeneration/SoftwareReport/software-report-base/SoftwareReport.Nodes.psm1:166 char:13
==> azure-arm.build_image: +             throw "ToolVersionNode '$($this.ToolName)' has empty vers …
==> azure-arm.build_image: +             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
==> azure-arm.build_image: + CategoryInfo          : OperationStopped: (ToolVersionNode 'Az…' has empty version:String) [], RuntimeException
==> azure-arm.build_image: + FullyQualifiedErrorId : ToolVersionNode 'Azure CLI' has empty version

[lanni-energinet]

@kishorekumar-anchala

Can you test this PR: #11173

[kishorekumar-anchala]

@kishorekumar-anchala

Can you test this PR: #11173

thank you @lanni-energinet , Tried it but bad luck , getting the same error .

[lanni-energinet]

@kishorekumar-anchala

Sorry about that. I overlooked that the install scripts was running as root, but the tests was running as packer.

I've updated the PR and tests are now passing.

[kishorekumar-anchala]

Hi @lanni-energinet - I tested the changes , working good . Thanks you for your support in implementing the request. PR will be in review. I will update you once it merged .