Flask throwing 'working outside of request context' when starting sub thread

PythonFlaskMongoengine

Python Problem Overview


I am trying to start a new thread in Python inside of a Flask application. I am doing background work that gets triggered by the request, but I don't need to wait for the work to be done to respond to the request.

Is it possible to set the flask request in this sub-threat to the request that came in? Reason being, our ACL on our queries to our DB (mongoengine in front of mongoDB) relies on the request's user (it grabs it from flask's request object) to see if they have access to the objects, and its blowing up because the request is not available in the sub-thread.

Any thoughts would be much appreciated.

Here's pseudo code of how I am handling it now, but it is not working.

@app.route('/my_endpoint', methods=['POST'])
def my_endpoint_handler():
    #do tracking in sub-thread so we don't hold up the page
    def handle_sub_view(req):
        from flask import request
        request = req
        # Do Expensive work
    thread.start_new_thread(handle_sub_view, (request))
    return "Thanks"

Python Solutions


Solution 1 - Python

Wrap your thread code in a test_request_context so you have access to context locals:

@app.route('/my_endpoint', methods=['POST'])
def my_endpoint_handler():
    #do tracking in sub-thread so we don't hold up the page
    def handle_sub_view(req):
        with app.test_request_context():
            from flask import request
            request = req
            # Do Expensive work
    thread.start_new_thread(handle_sub_view, (request))
    return "Thanks"

Edit: it's worth pointing out that the thread will have a different context than the original request. You need to extract any interesting request data, such as the user ID, before spawning the thread. You can then grab a (different) user object in the sub-thread using the ID.

Solution 2 - Python

Since version 0.10 there is a supported way of doing this: http://flask.pocoo.org/docs/api/#flask.copy_current_request_context

If you want the before_request hooks to run you must call current_app.preprocess_request() inside of the decorated function.

Solution 3 - Python

As @runfalk pointed out, you'll need to use @copy_current_request_context. Here's a working code snippet:

import threading

from flask import request, jsonify, copy_current_request_context


@app.route('/foo')
def get_foo():
    @copy_current_request_context
    def foo_main():
        # insert your code here
        print(request.url)

    threading.Thread(target=foo_main).start()

    return jsonify({'status': 'started'})

Solution 4 - Python

You can copy the desired info and pass it on:

@app.route('/my_endpoint', methods=['POST'])
def my_endpoint_handler():
    #do tracking in sub-thread so we don't hold up the page
    def handle_sub_view(data):
        # Use the data in subprocess
    data = request.get_json()  # copy the data
    thread.start_new_thread(handle_sub_view, data)
    return "Thanks"

Solution 5 - Python

A more cleaner way is using Flask built-in executor which wraps app context, see https://flask-executor.readthedocs.io/en/latest/

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionMattoToddView Question on Stackoverflow
Solution 1 - PythonAlex MoregaView Answer on Stackoverflow
Solution 2 - PythonrunfalkView Answer on Stackoverflow
Solution 3 - PythonjrcView Answer on Stackoverflow
Solution 4 - PythonvimView Answer on Stackoverflow
Solution 5 - PythonfrogcdcnView Answer on Stackoverflow