NXDOMAIN results for the psg-prod-eastus.azureedge.net domain are breaking package downloads

[ShaunLawrie]

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest version
• Search the existing issues.

Steps to reproduce

Install any module.
Install-Module AWS.Tools.ECR-Verbose
Occasionally you will get a failure to download the package.

Expected behavior

PS> $ErrorView = 'DetailedView'
PS> $requiredModules = @("AWS.Tools.Common", "AWS.Tools.ECR")
PS> $requiredModules | Foreach-Object {
    Write-Host "Installing PowerShell Module '$module'"
    Install-Module -Name $module -Force -Verbose
}

Both modules install successfully.

Actual behavior

PS> $ErrorView = 'DetailedView'
PS> $requiredModules = @("AWS.Tools.Common", "AWS.Tools.ECR")
PS> $requiredModules | Foreach-Object {
    Write-Host "Installing PowerShell Module '$module'"
    Install-Module -Name $module -Force -Verbose
}

One occasionally fails, sometimes the first, sometimes the second.

Error details

PS> $ErrorView = 'DetailedView'
PS> $requiredModules = @("AWS.Tools.Common", "AWS.Tools.ECR")
PS> $requiredModules | Foreach-Object {
    Write-Host "Installing PowerShell Module '$module'"
    Install-Module -Name $module -Force -Verbose
}

Installing PowerShell Module 'AWS.Tools.Common'
VERBOSE: Acquiring providers for assembly: /opt/microsoft/powershell/7/Modules/PackageManagement/coreclr/netstandard2.0/Microsoft.PackageManagement.MetaProvider.PowerShell.dll
VERBOSE: Acquiring providers for assembly: /opt/microsoft/powershell/7/Modules/PackageManagement/coreclr/netstandard2.0/Microsoft.PackageManagement.ArchiverProviders.dll
VERBOSE: Acquiring providers for assembly: /opt/microsoft/powershell/7/Modules/PackageManagement/coreclr/netstandard2.0/Microsoft.PackageManagement.NuGetProvider.dll
VERBOSE: Acquiring providers for assembly: /opt/microsoft/powershell/7/Modules/PackageManagement/coreclr/netstandard2.0/Microsoft.PackageManagement.CoreProviders.dll
VERBOSE: Using the provider 'PowerShellGet' for searching packages.
VERBOSE: The -Repository parameter was not specified.  PowerShellGet will use all of the registered repositories.
VERBOSE: Getting the provider object for the PackageManagement Provider 'NuGet'.
VERBOSE: The specified Location is 'https://www.powershellgallery.com/api/v2' and PackageManagementProvider is 'NuGet'.
VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='AWS.Tools.Common'' for ''.
VERBOSE: Total package yield:'1' for the specified package 'AWS.Tools.Common'.
VERBOSE: Performing the operation "Install-Module" on target "Version '4.1.734' of module 'AWS.Tools.Common'".
VERBOSE: The installation scope is specified to be 'CurrentUser'.
VERBOSE: The specified module will be installed in '/home/runner/.local/share/powershell/Modules'.
VERBOSE: The specified Location is 'NuGet' and PackageManagementProvider is 'NuGet'.
VERBOSE: Downloading module 'AWS.Tools.Common' with version '4.1.734' from the repository 'https://www.powershellgallery.com/api/v2'.
VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='AWS.Tools.Common'' for ''.
VERBOSE: InstallPackage' - name='AWS.Tools.Common', version='4.1.734',destination='/tmp/1065875406'
VERBOSE: DownloadPackage' - name='AWS.Tools.Common', version='4.1.734',destination='/tmp/1065875406/AWS.Tools.Common.4.1.734/AWS.Tools.Common.4.1.734.nupkg', uri='https://www.powershellgallery.com/api/v2/package/AWS.Tools.Common/4.1.734'
VERBOSE: Downloading 'https://www.powershellgallery.com/api/v2/package/AWS.Tools.Common/4.1.734'.
VERBOSE: Completed downloading 'https://www.powershellgallery.com/api/v2/package/AWS.Tools.Common/4.1.734'.
VERBOSE: Completed downloading 'AWS.Tools.Common'.
VERBOSE: InstallPackageLocal' - name='AWS.Tools.Common', version='4.1.734',destination='/tmp/1065875406'
VERBOSE: Validating the 'AWS.Tools.Common' module contents under '/tmp/1065875406/AWS.Tools.Common.4.1.734' path.
VERBOSE: Test-ModuleManifest successfully validated the module manifest file '/tmp/1065875406/AWS.Tools.Common.4.1.734'.
VERBOSE: Module 'AWS.Tools.Common' was installed successfully to path '/home/runner/.local/share/powershell/Modules/AWS.Tools.Common/4.1.734'.

Installing PowerShell Module 'AWS.Tools.ECR'
VERBOSE: Using the provider 'PowerShellGet' for searching packages.
VERBOSE: The -Repository parameter was not specified.  PowerShellGet will use all of the registered repositories.
VERBOSE: Getting the provider object for the PackageManagement Provider 'NuGet'.
VERBOSE: The specified Location is 'https://www.powershellgallery.com/api/v2' and PackageManagementProvider is 'NuGet'.
VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='AWS.Tools.ECR'' for ''.
VERBOSE: Total package yield:'1' for the specified package 'AWS.Tools.ECR'.
VERBOSE: Performing the operation "Install-Module" on target "Version '4.1.734' of module 'AWS.Tools.ECR'".
VERBOSE: The installation scope is specified to be 'CurrentUser'.
VERBOSE: The specified module will be installed in '/home/runner/.local/share/powershell/Modules'.
VERBOSE: The specified Location is 'NuGet' and PackageManagementProvider is 'NuGet'.
VERBOSE: Downloading module 'AWS.Tools.ECR' with version '4.1.734' from the repository 'https://www.powershellgallery.com/api/v2'.
VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='AWS.Tools.ECR'' for ''.
VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='AWS.Tools.Common'' for ''.
VERBOSE: InstallPackage' - name='AWS.Tools.Common', version='4.1.734',destination='/tmp/295771909'
VERBOSE: DownloadPackage' - name='AWS.Tools.Common', version='4.1.734',destination='/tmp/295771909/AWS.Tools.Common.4.1.734/AWS.Tools.Common.4.1.734.nupkg', uri='https://www.powershellgallery.com/api/v2/package/AWS.Tools.Common/4.1.734'
VERBOSE: Downloading 'https://www.powershellgallery.com/api/v2/package/AWS.Tools.Common/4.1.734'.
VERBOSE: Name or service not known (psg-prod-eastus.azureedge.net:443)
VERBOSE: Retry downloading 'https://www.powershellgallery.com/api/v2/package/AWS.Tools.Common/4.1.734' for '2' more times
VERBOSE: Name or service not known (psg-prod-eastus.azureedge.net:443)
VERBOSE: Retry downloading 'https://www.powershellgallery.com/api/v2/package/AWS.Tools.Common/4.1.734' for '1' more times
VERBOSE: Name or service not known (psg-prod-eastus.azureedge.net:443)
VERBOSE: Retry downloading 'https://www.powershellgallery.com/api/v2/package/AWS.Tools.Common/4.1.734' for '0' more times
VERBOSE: Downloading package 'AWS.Tools.Common' failed, please make sure 'https://www.powershellgallery.com/api/v2/package/AWS.Tools.Common/4.1.734' is accessable.
WARNING: Source Location 'https://www.powershellgallery.com/api/v2/package/AWS.Tools.Common/4.1.734' is not valid.

Install-Package: Package 'AWS.Tools.Common' failed to download.
Error: Process completed with exit code 1.

Environment data

I've had this occur in github actions on github hosted runners and can see the domain name failing to resolve
periodically from our office in New Zealand.

Version

Multiple versions (one is 7.4.6)

Visuals

With verbose output while installing the modules you can see the domain resolution failures:

I wrote a crappy script to check all the domains I could find listed that are required for psgallery package search and downloads and the eastus is the only one failing to resolve. It's failing to resolve from multiple DNS servers (I used google and cloudflare in this test):

# Crappy script to check all psgallery domains
$domains = @("psg-prod-eastus.azureedge.net", "psg-prod-centralus.azureedge.net", "onegetcdn.azureedge.net", "az818661.vo.msecnd.net", "powershellgallery.azureedge.net")
$resolvers = @("8.8.8.8", "1.1.1.1")

while ($true) {
  Start-Sleep -Seconds 1
  foreach ($resolver in $resolvers) {
    $results = [ordered]@{}
    foreach($domain in $domains) {
      $results[$domain] = $false
      try {
        Resolve-DnsName -Name $domain -Server $resolver -ErrorAction Stop | Out-Null
        $results[$domain] = $true
      } catch {
        if ($_.Exception.Message -notlike "*name does not exist*") {
          throw
        }
      }
    }
    Write-Host "$resolver > " -NoNewline
    $results.GetEnumerator() | ForEach-Object {
      $color = ($false -eq $_.Value) ? "Red" : "Green"
      Write-Host -ForegroundColor $color -Message $_.Name -NoNewline
      Write-Host ", " -NoNewline
    }
    Write-Host ""
  }
}

[ShaunLawrie]

Not sure if this is related PowerShell/PowerShell#24734

[o-l-a-v]

AWS seems to host links to the modules themselves. Great as a plan B / mirror.

• Announcement: AWS.Tools Offline Installer aws/aws-tools-for-powershell#107 (comment)
• Links to latest:
  • https://sdk-for-net.amazonwebservices.com/ps/v4/latest/AWS.Tools.zip
  • https://sdk-for-net.amazonwebservices.com/ps/v4/latest/AWSPowerShell.NetCore.zip
  • https://sdk-for-net.amazonwebservices.com/ps/v4/latest/AWSPowerShell.zip

Feature request for other modules to do the same or something similar:

• Az: [Feature]: Create mirror for when PowerShell Gallery is down Azure/azure-powershell#27001
• Microsoft.Graph: Create mirror for when PowerShell Gallery is down microsoftgraph/msgraph-sdk-powershell#3070
• PnP.PowerShell: Add `.nupkg` to GitHub releases as PowerShell Gallery mirror pnp/powershell#4689

Each Az release on GitHub also have a .tar.gz containing most of the Az.<submodule>.<version>.nupkg files.

• https://github.com/Azure/azure-powershell/releases

[mahmoudghorbelMG]

Hi all,
I confirm that i have also a package download failure occasionally on my runner (azure VM) since last night. I use the command "pwsh -c Install-Module" to install several modules, but it randomly fails with some modules. ex of error message: Install-Package: Package 'Az.Accounts' failed to download.

[ShaunLawrie]

AWS seems to host links to the modules themselves. Great as a plan B / mirror.

Thanks @o-l-a-v that was the only module in the critical path for us so I added a fallback to use the direct download links. DNS appears more stable today so it hasn't needed it 🤞