How should I interpret Heroku H18 errors?

Heroku

Heroku Problem Overview


We're seeing quite a lot of Error H18 (Request Interrupted) in the logs. How should these be interpreted, since sock=client in all cases I assume that it is the client that is disconnecting. Should it therefore be safe to ignore these?

Heroku Solutions


Solution 1 - Heroku

This typically indicates either that the user's network was disconnected (e.g. this happens with some frequency for mobile users) or the end-user closed their browser or similar (e.g. pressed Stop, went to a different page, etc).

You can safely ignore those that are tagged as being client-end in nature with "sock=client", but might want to investigate those that are being closed by the Server end ("sock=server").

Solution 2 - Heroku

Edit (Aug 10, 2015): My answer below for H18 errors is out of date. Heroku has changed the behavior of H18 errors making them more specific (and serious) than before. This Answer is now more correct.


H18 Errors

I recently asked Heroku support about a significant number of H18 errors (3-4 an hour sometimes) my app was receiving and being surfaced in the Metrics section of the new Heroku dashboard (screenshot above). I referenced catsby's response on this thread and asked to confirm if they were in fact not actionable. This was the response I received from Heroku Support:

> I just went through the last 24 hours of H18s on your app. They are all sock=client and looking at the User-Agent I see a lot of the usual culprits. Mobile browsers for the most part and also the latest Chrome which I have seen other apps have issue with as well since a few days ago. Unless you see a pattern such as for a particular URL or user then it often is just network issues. > > [That these H18 errors are] Not actionable is not always true. Some apps do care about it, and sometimes it can also mean a client crash rather than a network error. Browsers do crash from time to time but in particular mobile browsers can be pretty fragile. If an app uses a lot of assets and triggers "page not responding" errors you might see spikes in H18s. In that case sometimes there is something you can do about it. Other apps are serviced entirely in wired networks and would never expect it, in that case there could be a faulty switch or firewall. For the sake of transparency we still want to report these errors as there is no way to tell if they are or aren't actionable.

To summarize, most of the time you can ignore them if they are sock=client errors, but that does indicate that clients are disconnecting which could indicate a real issue depending on how your application is networked to its clients (e.g. mobile or really bad network connection)... but probably you can safely ignore them.

Solution 3 - Heroku

FYI: Ben Sheldon's and catsby's answers are no longer correct. They was at the time of writing, but things have changed. H18s are now always sock=server. Heroku added a new error code H27 for sock=client.

Details here: https://devcenter.heroku.com/changelog-items/662

H18 should be taken more seriously now as they are definitely an issue with your service. H27s can generally be safely ignored as they are an issue with the client.

Solution 4 - Heroku

There is another case that can cause this error that is not documented by Heroku.

If your server responds to the request and closes the connection without reading the request body, then the router will respond with the H18 error. The Heroku router logs will show sock=backend.

In this case, your server is not doing anything wrong with respect to the HTTP spec. It's a bug with the Heroku router.

I contacted Heroku technical support about this, and they confirmed the issue. It will be fixed in a new version of the router they are implementing.

The workaround is to always ensure the request body is read on the backend server before closing the connection.

Solution 5 - Heroku

If you're running a golang app, there is a strange case where you need to drain the body of the request. I was able to reproduce this bug in the heroku router if the following are true:

  1. The response body is large
  2. The HTTP request is of type POST
  3. The POST request has a body (even an empty one)

This is the code to reproduce and fix it:

...
// To force a Heroku H18 error
// curl -i \
//         -X POST \
//         -H "content-type: application/json" \
//         -H "content-length: 2" \
//         -d "\"{}\"" \
//         "http://localhost:8081/h18"
func h18(w http.ResponseWriter, r *http.Request) {
	// This fixes it
	drainBody(w, r)

	data := dummyBigResponse()
	jData, _ := json.Marshal(data)
	w.WriteHeader(http.StatusOK)
	w.Write(jData)
}

func dummyBigResponse() map[string]string {
	dummyResponse := map[string]string{}
	for i := 1; i < 1000; i++ {
		dummy := fmt.Sprintf("dummy-%d", i)
		dummyResponse[dummy] = "herokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuherokuheroku"
	}
	return dummyResponse
}

// Heroku's suggestion to drain the body
func drainBody(w http.ResponseWriter, r *http.Request) {
	// check if we have a body too big for buffering
	if r.Body != nil || r.Body != http.NoBody || r.ContentLength >1e6 {
		io.Copy(ioutil.Discard, r.Body)
	}
}
...

Without the drainBody call, you'll get the H18 router error. With it, you will not. I wish they documented this because they know it's an issue! I spoke with support about it.

Solution 6 - Heroku

In case anyone comes across this H18 error situation - the solution that worked for me for my simple Python 3.9.2 Flask app using Postgresql and SQLAlchemy.

I had to make sure that every routing endpoint function explicitly read the full request body. In my case this had NOTHING to do with long requests, timeouts or large data in either requests or response messages.

I fixed this by adding this line to the top of each function based on a post I found suggesting just such a solution.

@app.route("/store/<int:store_id>")
def get_store_by_id(store_id):
    # ALL requests need to read the full request body on Heroku / Flask
    hack = request.data
...

After adding this the app worked flawlessly - and if I remove the request.data call the app will randomly fail with a H18 error with no other diagnostics. Once it starts to fail it will begin to fail for all requests.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionKristofferView Question on Stackoverflow
Solution 1 - HerokucatsbyView Answer on Stackoverflow
Solution 2 - HerokuBen SheldonView Answer on Stackoverflow
Solution 3 - HerokuscosmanView Answer on Stackoverflow
Solution 4 - HerokuWill SewellView Answer on Stackoverflow
Solution 5 - HerokuTonyView Answer on Stackoverflow
Solution 6 - HerokumalsmithView Answer on Stackoverflow