Initial bicep restore of Bicep extensibility modules fails

[maskati]

Bicep version
0.31.92

Describe the bug
If an extensibility module is not restored, initial restore using bicep restore fails.
The second bicep restore succeeds.
If not restored, initial restore as part of bicep build succeeds.

To Reproduce
bicepconfig.json

{
  "experimentalFeaturesEnabled": {
    "extensibility": true
  }
}

main.bicep

extension 'br:mcr.microsoft.com/bicep/extensions/microsoftgraph/v1.0:0.1.8-preview'

Restore first attempt fails

1. Clear module cache
2. bicep restore main.bicep first attempt fails
3. bicep restore main.bicep second attempt succeeds

Build first attempt succeeds

1. Clear module cache
2. bicep build main.bicep first attempt succeeds

The exception on initial restore failure:

Unhandled exception. System.Collections.Generic.KeyNotFoundException: The given key '[profiledirectory]/.bicep/br/mcr.microsoft.com/bicep$extensions$microsoftgraph$v1.0/0.1.8-preview$/types.tgz' was not present in the dictionary.
   at System.Collections.Immutable.ImmutableDictionary`2.get_Item(TKey)
   at Bicep.Core.Workspaces.SourceFileGrouping.TryGetSourceFile(IArtifactReferenceSyntax reference)
   at Bicep.Core.BicepCompiler.<GetModuleRestoreDiagnosticsByBicepFile>g__DiagnosticForModule|14_0(SourceFileGrouping grouping, IArtifactReferenceSyntax moduleDeclaration)
   at Bicep.Core.BicepCompiler.<GetModuleRestoreDiagnosticsByBicepFile>g__GetDiagnosticsForModulesToRestore|14_1(SourceFileGrouping grouping, ImmutableHashSet`1 originalArtifactsToRestore)+MoveNext()

[stephaniezyen]

@majastrz @shenglol @StephenWeatherford

[shenglol]

It looks like this might be a race condition, but I haven't been able to reproduce it locally. @maskati - are you encountering this error consistently?

[maskati]

@shenglol yes I am encountering the issue consistently. You can reproduce e.g. using Docker:

docker run --rm -it alpine /bin/sh -c "apk add -q --no-progress libstdc++ && wget -qO bicep https://github.com/Azure/bicep/releases/download/v0.31.92/bicep-linux-musl-x64 && chmod +x ./bicep && echo '{\"experimentalFeaturesEnabled\":{\"extensibility\":true}}' > bicepconfig.json && echo \"extension 'br:mcr.microsoft.com/bicep/extensions/microsoftgraph/v1.0:0.1.8-preview'\" > test.bicep && DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1 ./bicep restore test.bicep"

Results in:

Unhandled exception. System.Collections.Generic.KeyNotFoundException: The given key 'file:///root/.bicep/br/mcr.microsoft.com/bicep$extensions$microsoftgraph$v1.0/0.1.8-preview$/types.tgz' was not present in the dictionary.
   at System.Collections.Immutable.ImmutableDictionary`2.get_Item(TKey)
   at Bicep.Core.Workspaces.SourceFileGrouping.TryGetSourceFile(IArtifactReferenceSyntax reference)
   at Bicep.Core.BicepCompiler.<GetModuleRestoreDiagnosticsByBicepFile>g__DiagnosticForModule|14_0(SourceFileGrouping grouping, IArtifactReferenceSyntax moduleDeclaration)
   at Bicep.Core.BicepCompiler.<GetModuleRestoreDiagnosticsByBicepFile>g__GetDiagnosticsForModulesToRestore|14_1(SourceFileGrouping grouping, ImmutableHashSet`1 originalArtifactsToRestore)+MoveNext()
   at System.Linq.Lookup`2.Create[TSource](IEnumerable`1, Func`2, Func`2, IEqualityComparer`1)
   at System.Linq.Enumerable.ToLookup[TSource,TKey,TElement](IEnumerable`1, Func`2, Func`2, IEqualityComparer`1 )
   at System.Linq.Enumerable.ToLookup[TSource,TKey,TElement](IEnumerable`1, Func`2, Func`2)
   at Bicep.Core.BicepCompiler.GetModuleRestoreDiagnosticsByBicepFile(SourceFileGrouping sourceFileGrouping, ImmutableHashSet`1 originalModulesToRestore, Boolean forceModulesRestore)
   at Bicep.Core.BicepCompiler.Restore(Compilation compilation, Boolean forceRestore)
   at Bicep.Cli.Commands.RestoreCommand.RunAsync(RestoreArguments args)
   at Bicep.Cli.Program.RunAsync(String[] args, CancellationToken cancellationToken)
   at Bicep.Cli.Program.<>c__DisplayClass3_0.<<Main>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Bicep.Cli.Program.RunWithCancellationAsync(Func`2 runFunc)
   at Bicep.Cli.Program.Main(String[] args)
   at Bicep.Cli.Program.<Main>(String[] args)

[maskati]

@shenglol note that this only occurs when the extensibility module is not already in the cache, so to reproduce you need an empty cache. This is however often the case when performing restore operations on a CI pipeline such as GitHub hosted actions or in the above example on a fresh container image.

[shenglol]

I did clear the cache. The only difference is that I tested the issue on my Windows machine. I'll give it a try using a container.

[maskati]

The issue seems to occur on both Linux and Windows, see repro GitHub Actions run with both Linux and Windows jobs:
https://github.com/maskati/bicep-issue-15696/actions/runs/12347331438

Linux

Current runner version: '2.321.0'
##[group]Operating System
Ubuntu
24.04.1
LTS
##[endgroup]
##[group]Runner Image
Image: ubuntu-24.04
Version: 20241208.1.0
Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20241208.1/images/ubuntu/Ubuntu2404-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20241208.1
##[endgroup]
##[group]Run bicep restore test.bicep
[36;1mbicep restore test.bicep�[0m
shell: /usr/bin/bash -e {0}
##[endgroup]
Unhandled exception. System.Collections.Generic.KeyNotFoundException: The given key 'file:///home/runner/.bicep/br/mcr.microsoft.com/bicep$extensions$microsoftgraph$v1.0/0.1.8-preview$/types.tgz' was not present in the dictionary.
   at System.Collections.Immutable.ImmutableDictionary`2.get_Item(TKey)
   at Bicep.Core.Workspaces.SourceFileGrouping.TryGetSourceFile(IArtifactReferenceSyntax reference)
   at Bicep.Core.BicepCompiler.<GetModuleRestoreDiagnosticsByBicepFile>g__DiagnosticForModule|14_0(SourceFileGrouping grouping, IArtifactReferenceSyntax moduleDeclaration)
   at Bicep.Core.BicepCompiler.<GetModuleRestoreDiagnosticsByBicepFile>g__GetDiagnosticsForModulesToRestore|14_1(SourceFileGrouping grouping, ImmutableHashSet`1 originalArtifactsToRestore)+MoveNext()
   at System.Linq.Lookup`2.Create[TSource](IEnumerable`1, Func`2, Func`2, IEqualityComparer`1)
   at System.Linq.Enumerable.ToLookup[TSource,TKey,TElement](IEnumerable`1, Func`2, Func`2, IEqualityComparer`1 )
   at System.Linq.Enumerable.ToLookup[TSource,TKey,TElement](IEnumerable`1, Func`2, Func`2)
   at Bicep.Core.BicepCompiler.GetModuleRestoreDiagnosticsByBicepFile(SourceFileGrouping sourceFileGrouping, ImmutableHashSet`1 originalModulesToRestore, Boolean forceModulesRestore)
   at Bicep.Core.BicepCompiler.Restore(Compilation compilation, Boolean forceRestore)
   at Bicep.Cli.Commands.RestoreCommand.RunAsync(RestoreArguments args)
   at Bicep.Cli.Program.RunAsync(String[] args, CancellationToken cancellationToken)
   at Bicep.Cli.Program.<>c__DisplayClass3_0.<<Main>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Bicep.Cli.Program.RunWithCancellationAsync(Func`2 runFunc)
   at Bicep.Cli.Program.Main(String[] args)
   at Bicep.Cli.Program.<Main>(String[] args)
/home/runner/work/_temp/4bd57053-862d-49d5-be5e-82577424243c.sh: line 1:  1829 Aborted                 (core dumped) bicep restore test.bicep
##[error]Process completed with exit code 134.

Windows

Current runner version: '2.321.0'
##[group]Operating System
Microsoft Windows Server 2022
10.0.20348
Datacenter
##[endgroup]
##[group]Runner Image
Image: windows-2022
Version: 20241211.1.0
Included Software: https://github.com/actions/runner-images/blob/win22/20241211.1/images/windows/Windows2022-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/win22%2F20241211.1
##[endgroup]
##[group]Run bicep restore test.bicep
[36;1mbicep restore test.bicep�[0m
shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
##[endgroup]
Unhandled exception. System.Collections.Generic.KeyNotFoundException: The given key 'file:///C:/Users/runneradmin/.bicep/br/mcr.microsoft.com/bicep$extensions$microsoftgraph$v1.0/0.1.8-preview$/types.tgz' was not present in the dictionary.
   at System.Collections.Immutable.ImmutableDictionary`2.get_Item(TKey)
   at Bicep.Core.Workspaces.SourceFileGrouping.TryGetSourceFile(IArtifactReferenceSyntax reference)
   at Bicep.Core.BicepCompiler.<GetModuleRestoreDiagnosticsByBicepFile>g__DiagnosticForModule|14_0(SourceFileGrouping grouping, IArtifactReferenceSyntax moduleDeclaration)
   at Bicep.Core.BicepCompiler.<GetModuleRestoreDiagnosticsByBicepFile>g__GetDiagnosticsForModulesToRestore|14_1(SourceFileGrouping grouping, ImmutableHashSet`1 originalArtifactsToRestore)+MoveNext()
   at System.Linq.Lookup`2.Create[TSource](IEnumerable`1, Func`2, Func`2, IEqualityComparer`1)
   at System.Linq.Enumerable.ToLookup[TSource,TKey,TElement](IEnumerable`1, Func`2, Func`2, IEqualityComparer`1 )
   at System.Linq.Enumerable.ToLookup[TSource,TKey,TElement](IEnumerable`1, Func`2, Func`2)
   at Bicep.Core.BicepCompiler.GetModuleRestoreDiagnosticsByBicepFile(SourceFileGrouping sourceFileGrouping, ImmutableHashSet`1 originalModulesToRestore, Boolean forceModulesRestore)
   at Bicep.Core.BicepCompiler.Restore(Compilation compilation, Boolean forceRestore)
   at Bicep.Cli.Commands.RestoreCommand.RunAsync(RestoreArguments args)
   at Bicep.Cli.Program.RunAsync(String[] args, CancellationToken cancellationToken)
   at Bicep.Cli.Program.<>c__DisplayClass3_0.<<Main>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Bicep.Cli.Program.RunWithCancellationAsync(Func`2 runFunc)
   at Bicep.Cli.Program.Main(String[] args)
   at Bicep.Cli.Program.<Main>(String[] args)
##[error]Process completed with exit code 1.