Save a large file using the Python requests library

PythonFileHttpDownloadRequest

Python Problem Overview


> Possible Duplicate:
> How to download image using requests

I know that fetching a url is as simple as requests.get and I can get at the raw response body and save it to a file, but for large files, is there a way to stream directly to a file? Like if I'm downloading a movie with it or something?

Python Solutions


Solution 1 - Python

Oddly enough, requests doesn't have anything simple for this. You'll have to iterate over the response and write those chunks to a file:

response = requests.get('http://www.example.com/image.jpg', stream=True)
    
# Throw an error for bad status codes
response.raise_for_status()
    
with open('output.jpg', 'wb') as handle:
    for block in response.iter_content(1024):
        handle.write(block)

I usually just use urllib.urlretrieve(). It works, but if you need to use a session or some sort of authentication, the above code works as well.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionMatt WilliamsonView Question on Stackoverflow
Solution 1 - PythonBlenderView Answer on Stackoverflow