Read file data without saving it in Flask

I am writing my first flask application. I am dealing with file uploads, and basically what I want is to read the data/content of the uploaded file without saving it and then print it on the resulting page. Yes, I am assuming that the user uploads a text file always.

Here is the simple upload function i am using:

@app.route('/upload/', methods=['GET', 'POST'])
def upload():
    if request.method == 'POST':
        file = request.files['file']
        if file:
            filename = secure_filename(file.filename)
            file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
            a = 'file uploaded'

    return render_template('upload.html', data = a)

Right now, I am saving the file, but what I need is that 'a' variable to contain the content/data of the file .. any ideas?

FileStorage contains stream field. This object must extend IO or file object, so it must contain read and other similar methods. FileStorage also extend stream field object attributes, so you can just use file.read() instead file.stream.read(). Also you can use save argument with dst parameter as StringIO or other IO or file object to copy FileStorage.stream to another IO or file object.

See documentation: http://flask.pocoo.org/docs/api/#flask.Request.files and http://werkzeug.pocoo.org/docs/datastructures/#werkzeug.datastructures.FileStorage.

If you want to use standard Flask stuff - there's no way to avoid saving a temporary file if the uploaded file size is > 500kb. If it's smaller than 500kb - it will use "BytesIO", which stores the file content in memory, and if it's more than 500kb - it stores the contents in TemporaryFile() (as stated in the werkzeug documentation). In both cases your script will block until the entirety of uploaded file is received.

The easiest way to work around this that I have found is:

1. Create your own file-like IO class where you do all the processing of the incoming data

2. In your script, override Request class with your own:

   class MyRequest( Request ): def _get_file_stream( self, total_content_length, content_type, filename=None, content_length=None ): return MyAwesomeIO( filename, 'w' )

3. Replace Flask's request_class with your own:

   app.request_class = MyRequest

4. Go have some beer :)

I share my solution (assuming everything is already configured to connect to google bucket in flask)

from google.cloud import storage

@app.route('/upload/', methods=['POST'])
def upload():
    if request.method == 'POST':
        # FileStorage object wrapper
        file = request.files["file"]                    
        if file:
            os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = app.config['GOOGLE_APPLICATION_CREDENTIALS']
            bucket_name = "bucket_name" 
            storage_client = storage.Client()
            bucket = storage_client.bucket(bucket_name)
            # Upload file to Google Bucket
            blob = bucket.blob(file.filename) 
            blob.upload_from_string(file.read())

My post

https://stackoverflow.com/questions/60843700/direct-to-google-bucket-in-flask

I was trying to do the exact same thing, open a text file (a CSV for Pandas actually). Don't want to make a copy of it, just want to open it. The form-WTF has a nice file browser, but then it opens the file and makes a temporary file, which it presents as a memory stream. With a little work under the hood,

form = UploadForm() 
 if form.validate_on_submit(): 
      filename = secure_filename(form.fileContents.data.filename)  
      filestream =  form.fileContents.data 
      filestream.seek(0)
      ef = pd.read_csv( filestream  )
      sr = pd.DataFrame(ef)  
      return render_template('dataframe.html',tables=[sr.to_html(justify='center, classes='table table-bordered table-hover')],titles = [filename], form=form) 

I share my solution, using pandas

@app.route('/upload/', methods=['POST'])
def upload():
    if request.method == 'POST':
        # FileStorage object wrapper
        file = request.files["file"]                    
        if file:
            df = pd.read_excel(files_excel["file"])

Building on a great answer by @tbicr the simplest form of that boils down to:

for line in request.files.get('file'):
   print("Next line: " + line)

in function

def handleUpload():
    if 'photo' in request.files:
        photo = request.files['photo']
        if photo.filename != '':      
            image = request.files['photo']  
            image_string = base64.b64encode(image.read())
            image_string = image_string.decode('utf-8')
            #use this to remove b'...' to get raw string
            return render_template('handleUpload.html',filestring = image_string)
    return render_template('upload.html')

in html file

<html>
<head>
    <title>Simple file upload using Python Flask</title>
</head>
<body>
    {% if filestring %}
      <h1>Raw image:</h1>
      <h1>{{filestring}}</h1>
      <img src="data:image/png;base64, {{filestring}}" alt="alternate" />.
    {% else %}
      <h1></h1>
    {% endif %}
</body>

In case we want to dump the in memory file to disk. This code can be used

  if isinstanceof(obj,SpooledTemporaryFile):
    obj.rollover()