How do I download multiple files or an entire folder from Google Colab?
Python 3.xGoogle ColaboratoryPython 3.x Problem Overview
Currently, I can download files as individual files with the command
files.download(file_name)
I also tried uploading them to the drive with the below code snippet but it is uploading them as individual files.
uploaded = drive.CreateFile({'title': file_name})
uploaded.SetContentString('Sample upload file content')
uploaded.Upload()
print('Uploaded file with ID {}'.format(uploaded.get('id')))
How can I download multiple files as a folder to my local computer? Or how can I upload these files as a folder to my google drive?
Python 3.x Solutions
Solution 1 - Python 3.x
I have created a zip file:
!zip -r /content/file.zip /content/Folder_To_Zip
Than I have downloded that zip file:
from google.colab import files
files.download("/content/file.zip")
Solution 2 - Python 3.x
For example, if you have to download log folder:
!zip -r log.zip log/
-r
represent recursive
while log.zip
is destination zip file and
log/
is source folder path
Solution 3 - Python 3.x
I found that:
!zip -r ./myresultingzippedfolderwithallthefiles.zip ./myoriginalfolderwithallthefiles/
worked for me in colab.
Here .
can be your home directory or the directory where your original myoriginalfolderwithallthefiles
is and where myresultingzippedfolderwithallthefiles.zip
will be created. Change the directories as needed.
Solution 4 - Python 3.x
You may use code to zip folders and download them using files
.
#@title Utility to zip and download a directory
#@markdown Use this method to zip and download a directory. For ex. a TB logs
#@markdown directory or a checkpoint(s) directory.
from google.colab import files
import os
dir_to_zip = 'dir_name' #@param {type: "string"}
output_filename = 'file.zip' #@param {type: "string"}
delete_dir_after_download = "No" #@param ['Yes', 'No']
os.system( "zip -r {} {}".format( output_filename , dir_to_zip ) )
if delete_dir_after_download == "Yes":
os.system( "rm -r {}".format( dir_to_zip ) )
files.download( output_filename )
Solution 5 - Python 3.x
In my case, I had to download an entire folder containing h5 files(for submitting a college project) of each model my notebook built. Easiest way I found to download this folder and hence all files in the folder is to drag and drop the folder into the "My Drive" folder in the same folder tree.
Obviously I later downloaded the folder from Google Drive.
Solution 6 - Python 3.x
Copy this code into a cell, and change the 2 fields filename and folders_or_files_to_save. It will zip all of the folders or files into a zipfile and save it in your Google drive
#@title save yo data to drive
filename = "kerasmodel" #@param {type:"string"}
folders_or_files_to_save = "keras_model.h5" #@param {type:"string"}
from google.colab import files
from google.colab import auth
from googleapiclient.http import MediaFileUpload
from googleapiclient.discovery import build
def save_file_to_drive(name, path):
file_metadata = {
'name': name,
'mimeType': 'application/octet-stream'
}
media = MediaFileUpload(path,
mimetype='application/octet-stream',
resumable=True)
created = drive_service.files().create(body=file_metadata, media_body=media, fields='id').execute()
print('File ID: {}'.format(created.get('id')))
return created
extension_zip = ".zip"
zip_file = filename + extension_zip
# !rm -rf $zip_file
!zip -r $zip_file {folders_or_files_to_save} # FOLDERS TO SAVE INTO ZIP FILE
auth.authenticate_user()
drive_service = build('drive', 'v3')
destination_name = zip_file
path_to_file = zip_file
save_file_to_drive(destination_name, path_to_file)
Solution 7 - Python 3.x
!zip -r /content/sample_data.zip /content/sample_data
# change sample_data.zip to your desired download name Ex: nothing.zip
# change sample_data to your desired download folder name Ex: ner_data
Solution 8 - Python 3.x
Use tar
to group files in a directory into a single file.
For example, here's a snippet that creates a directory, 3 files inside it, and a .tar
archive containing the group:
!mkdir demo
!echo a > demo/a
!echo b > demo/b
!echo c > demo/c
!tar -cvf demo.tar demo/
The file to download would be demo.tar
in this case. For more tips, search for creating and expanding tar archives.