Powershell - Why is Using Invoke-WebRequest Much Slower Than a Browser Download?
PowershellAmazon Ec2Amazon S3Powershell 3.0Powershell Problem Overview
I use Powershell's Invoke-WebRequest method to download a file from Amazon S3 to my Windows EC2 instance.
If I download the file using Chrome, I am able to download a 200 MB file in 5 seconds. The same download in PowerShell using Invoke-WebRequest takes up to 5 minutes.
Why is using Invoke-WebRequest slower and is there a way to download at full speed in a PowerShell script?
Powershell Solutions
Solution 1 - Powershell
Without switching away from Invoke-WebRequest, turning off the progress bar did it for me. I found the answer from this thread: https://github.com/PowerShell/PowerShell/issues/2138 (jasongin commented on Oct 3, 2016)
$ProgressPreference = 'SilentlyContinue'
Invoke-WebRequest <params>
For my 5MB file on localhost, the download time went from 30s to 250ms.
Note that to get the progress bar back in the active shell, you need to call $ProgressPreference = 'Continue'.
Solution 2 - Powershell
I was using
Invoke-WebRequest $video_url -OutFile $local_video_url
I changed the above to
$wc = New-Object net.webclient
$wc.Downloadfile($video_url, $local_video_url)
This restored the download speed to what I was seeing in my browsers.
Solution 3 - Powershell
$ProgressPreference = 'SilentlyContinue'
I got this down from 52min down to 14sec, for a file of 450 M.
Spectacular.
Solution 4 - Powershell
One-liner to download a file to the temp directory:
(New-Object Net.WebClient).DownloadFile("https://www.google.com", "$env:temp\index.html")
Solution 5 - Powershell
I just hit this issue today, if you change the ContentType argument to application/octet-stream it is much faster (as fast as using webclient). The reason is because the Invoke-Request command will not try and parse the response as JSON or XML.
Invoke-RestMethod -ContentType "application/octet-stream" -Uri $video_url -OutFile $local_video_url