Flask broken pipe with requests

PythonUbuntuPython 2.7Flask

Python Problem Overview


I'd like to send a local REST request in a flask app, like this:

from flask import Flask, url_for, request
import requests

app = Flask(__name__)

@app.route("/<name>/hi", methods=["POST"])
def hi_person(name):
    form = {"name": name}
    return requests.post(url_for("hi", _external=True), data=form)

@app.route("/hi", methods=["POST"])
def hi():
    return 'Hi, %s!' % request.form["name"]

Sending curl -X POST http://localhost:5000/john/hi causes the entire flask app to freeze. When I send a kill signal, I get a broken pipe error. Is there a way to prevent flask from freezing here?

Python Solutions


Solution 1 - Python

Run your flask app under a proper WSGI server capable of handling concurrent requests (perhaps gunicorn or uWSGI) and it'll work. While developing, enable threads in the Flask-supplied server with:

app.run(threaded=True)

but note that the Flask server is not recommended for production use. As of Flask 1.0, threaded is enabled by default, and you'd want to use the flask command on the command line, really, to run your app.

What happens is that using requests you are making a second request to your flask app, but since it is still busy processing the first, it won't respond to this second request until it is done with that first request.

Incidentally, under Python 3 the socketserver implementation handles the disconnect more gracefully and continues to serve rather than crash.

Solution 2 - Python

There are several things at play here, and I'll try to address them one-at-a-time.

First, you're probably using the toy development server. This server has many limitations; chiefly among these limitations is that it can only handle one request at a time. When you create a second request during your first request, you are locking up your application: The requests.post() function is waiting for Flask to respond, but Flask itself is waiting for post() to return! The solution to this particular problem is to run your WSGI application in a multithreaded or multiprocess environment. I prefer http://twistedmatrix.com/trac/wiki/TwistedWeb for this, but there are several other options.

With that out of the way... This is an antipattern. You almost certainly don't want to invoke all of the overhead of an HTTP request just to share some functionality between two views. The correct thing to do is to refactor to have a separate function that does that shared work. I can't really refactor your particular example, because what you have is very simple and doesn't really even merit two views. What did you want to build, exactly?

Edit: A comment asks whether multithreaded mode in the toy stdlib server would be sufficient to keep the deadlock from occurring. I'm going to say "maybe." Yes, if there aren't any dependencies keeping both threads from making progress, and both threads make sufficient progress to finish their networking tasks, then the requests will complete correctly. However, determining whether two threads will deadlock each other is undecidable (proof omitted as obtuse) and I'm not willing to say for sure that the stdlib server can do it right.

Solution 3 - Python

The bug that caused the crash was fixed in Version 0.12, Released on December 21st 2016. Yeah! This is an important fix that many have been waiting for.

From the Flask changelog:

> - Revert a behavior change that made the dev server crash instead of returning a Internal Server Error (pull request #2006).

Solution 4 - Python

I have had the same issue with post method, in general my post method was doing nothing, thats why this issues came

return _socket.socket.send(self._sock, data, flags) urllib3.exceptions.ProtocolError:
('Connection aborted.', BrokenPipeError(32, 'Broken pipe'))

if request.method == 'POST':
    print(len(request.data))
return 'dummy'

this print did the trick

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionjfochtView Question on Stackoverflow
Solution 1 - PythonMartijn PietersView Answer on Stackoverflow
Solution 2 - PythonCorbinView Answer on Stackoverflow
Solution 3 - PythonarotmanView Answer on Stackoverflow
Solution 4 - PythonTaras VaskivView Answer on Stackoverflow