Apache HttpClient timeout

JavaTimeoutApache Httpclient-4.x

Java Problem Overview


Is there a way to specify a timeout for the whole execution of HttpClient?

I have tried the following:

httpClient.getParams().setParameter("http.socket.timeout", timeout * 1000);
httpClient.getParams().setParameter("http.connection.timeout", timeout * 1000);
httpClient.getParams().setParameter("http.connection-manager.timeout", new Long(timeout * 1000));
httpClient.getParams().setParameter("http.protocol.head-body-timeout", timeout * 1000);

It actually works fine, except if a remote host sends back data - even at one byte/second - it will continue to read forever! But I want to interrupt the connection in 10 seconds max, whether or not the host responds.

Java Solutions


Solution 1 - Java

For a newer version of httpclient (e.g. http components 4.3 - https://hc.apache.org/httpcomponents-client-4.3.x/index.html):

int CONNECTION_TIMEOUT_MS = timeoutSeconds * 1000; // Timeout in millis.
RequestConfig requestConfig = RequestConfig.custom()
    .setConnectionRequestTimeout(CONNECTION_TIMEOUT_MS)
    .setConnectTimeout(CONNECTION_TIMEOUT_MS)
    .setSocketTimeout(CONNECTION_TIMEOUT_MS)
    .build();

HttpPost httpPost = new HttpPost(URL);
httpPost.setConfig(requestConfig);

Solution 2 - Java

There is currently no way to set a maximum request duration of that sort: basically you want to say I don't care whether or not any specific request stage times out, but the entire request must not last longer than 15 seconds (for example).

Your best bet would be to run a separate timer, and when it expires fetch the connection manager used by the HttpClient instance and shutdown the connection, which should terminate the link. Let me know if that works for you.

Solution 3 - Java

Works fine, as proposed by Femi. Thanks!

Timer timer = new Timer();
timer.schedule(new TimerTask() {
    public void run() {
        if(getMethod != null) {
            getMethod.abort();
        }
    }
}, timeout * 1000);

Solution 4 - Java

Timer is evil! Using timer or executor or any other mechanism which creates a thread/runnable object per request is a very bad idea. Please think wisely and don't do it. Otherwise you will quickly run into all kind of memory issues with more or less real environment. Imagine 1000 req/min means 1000 threads or workers / min. Poor GC. The solution I propose require only 1 watchdog thread and will save you resources time and nerves. Basically you do 3 steps.

  1. put request in cache.
  2. remove request from cache when complete.
  3. abort requests which are not complete within your limit.

your cache along with watchdog thread may look like this.

import org.apache.http.client.methods.*;
import java.util.*;
import java.util.concurrent.*;
import java.util.stream.*;

public class RequestCache {

private static final long expireInMillis = 300000;
private static final Map<HttpUriRequest, Long> cache = new ConcurrentHashMap<>();
private static final ScheduledExecutorService exe = Executors.newScheduledThreadPool(1);

static {
    // run clean up every N minutes
    exe.schedule(RequestCache::cleanup, 1, TimeUnit.MINUTES);
}

public static void put(HttpUriRequest request) {
    cache.put(request, System.currentTimeMillis()+expireInMillis);
}

public static void remove(HttpUriRequest request) {
    cache.remove(request);
}

private static void cleanup() {
    long now = System.currentTimeMillis();
    // find expired requests
    List<HttpUriRequest> expired = cache.entrySet().stream()
            .filter(e -> e.getValue() > now)
            .map(Map.Entry::getKey)
            .collect(Collectors.toList());

    // abort requests
    expired.forEach(r -> {
        if (!r.isAborted()) {
            r.abort();
        }
        cache.remove(r);
      });
    }
  }

and the following sudo code how to use cache

import org.apache.http.client.methods.*;

public class RequestSample {

public void processRequest() {
    HttpUriRequest req = null;
    try {
        req = createRequest();

        RequestCache.put(req);

        execute(req);

    } finally {
        RequestCache.remove(req);
    }
  }
}

Solution 5 - Java

Building off the the other answers, my solution was to use a HttpRequestInterceptor to add the abort runnable to every request. Also i swapped out the Timer for a ScheduledExecutorService.

public class TimeoutInterceptor implements HttpRequestInterceptor {

private int requestTimeout = 1 * DateTimeConstants.MILLIS_PER_MINUTE;

private ScheduledExecutorService executorService = Executors.newSingleThreadScheduledExecutor();

public TimeoutInterceptor() {  }

public TimeoutInterceptor(final int requestTimeout) {
    this.requestTimeout = requestTimeout;
}

@Override
public void process(final HttpRequest request, final HttpContext context) throws HttpException, IOException {
    if (request instanceof AbstractExecutionAwareRequest) {
        final AbstractExecutionAwareRequest abortableRequest = (AbstractExecutionAwareRequest) request;
        setAbort(abortableRequest);
    } else if (request instanceof HttpRequestWrapper) {
        HttpRequestWrapper wrapper = (HttpRequestWrapper) request;
        this.process(wrapper.getOriginal(), context);
    }
  
}

/**
 * @param abortableRequest
 */
private void setAbort(final AbstractExecutionAwareRequest abortableRequest) {
    final SoftReference<AbstractExecutionAwareRequest> requestRef = new SoftReference<AbstractExecutionAwareRequest>(abortableRequest);
    
    executorService.schedule(new Runnable() {
        
        @Override
        public void run() {
            AbstractExecutionAwareRequest actual = requestRef.get();
            if (actual != null && !actual.isAborted()) {
                actual.abort();
            }
        }
    }, requestTimeout, TimeUnit.MILLISECONDS);

}

public void setRequestTimeout(final int requestTimeout) {
    this.requestTimeout = requestTimeout;
}
}

Solution 6 - Java

In HttpClient 4.3 version you can use below example.. let say for 5 seconds

int timeout = 5;
RequestConfig config = RequestConfig.custom()
  .setConnectTimeout(timeout * 1000)
  .setConnectionRequestTimeout(timeout * 1000)
  .setSocketTimeout(timeout * 1000).build();
CloseableHttpClient client = 
  HttpClientBuilder.create().setDefaultRequestConfig(config).build();
HttpGet request = new HttpGet("http://localhost:8080/service"); // GET Request
response = client.execute(request);

Solution 7 - Java

That's also work:

HttpClient client = new DefaultHttpClient();
HttpConnectionParams.setConnectionTimeout(client.getParams(), timeout * 1000);
HttpConnectionParams.setSoTimeout(client.getParams(), timeout * 1000);

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionRosty KereiView Question on Stackoverflow
Solution 1 - JavaRealManView Answer on Stackoverflow
Solution 2 - JavaFemiView Answer on Stackoverflow
Solution 3 - JavaRosty KereiView Answer on Stackoverflow
Solution 4 - JavaValchkouView Answer on Stackoverflow
Solution 5 - JavamR_fr0gView Answer on Stackoverflow
Solution 6 - JavaMADHAIYAN MView Answer on Stackoverflow
Solution 7 - JavaAlexBView Answer on Stackoverflow