Spring RestTemplate - how to enable full debugging/logging of requests/responses?
JavaDebuggingLoggingResttemplateJava Problem Overview
I have been using the Spring RestTemplate for a while and I consistently hit a wall when I'am trying to debug it's requests and responses. I'm basically looking to see the same things as I see when I use curl with the "verbose" option turned on. For example :
curl -v http://twitter.com/statuses/public_timeline.rss
Would display both the sent data and the received data (including the headers, cookies, etc.).
I have checked some related posts like : https://stackoverflow.com/questions/3892018/how-do-i-log-response-in-spring-resttemplate but I haven't managed to solve this issue.
One way to do this would be to actually change the RestTemplate source code and add some extra logging statements there, but I would find this approach really a last resort thing. There should be some way to tell Spring Web Client/RestTemplate to log everything in a much friendlier way.
My goal would be to be able to do this with code like :
restTemplate.put("http://someurl", objectToPut, urlPathValues);
and then to get the same type of debug information (as I get with curl) in the log file or in the console. I believe this would be extremely useful for anyone that uses the Spring RestTemplate and has problems. Using curl to debug your RestTemplate problems just doesn't work (in some cases).
Java Solutions
Solution 1 - Java
Just to complete the example with a full implementation of ClientHttpRequestInterceptor
to trace request and response:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.HttpRequest;
import org.springframework.http.client.ClientHttpRequestExecution;
import org.springframework.http.client.ClientHttpRequestInterceptor;
import org.springframework.http.client.ClientHttpResponse;
public class LoggingRequestInterceptor implements ClientHttpRequestInterceptor {
final static Logger log = LoggerFactory.getLogger(LoggingRequestInterceptor.class);
@Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
traceRequest(request, body);
ClientHttpResponse response = execution.execute(request, body);
traceResponse(response);
return response;
}
private void traceRequest(HttpRequest request, byte[] body) throws IOException {
log.info("===========================request begin================================================");
log.debug("URI : {}", request.getURI());
log.debug("Method : {}", request.getMethod());
log.debug("Headers : {}", request.getHeaders() );
log.debug("Request body: {}", new String(body, "UTF-8"));
log.info("==========================request end================================================");
}
private void traceResponse(ClientHttpResponse response) throws IOException {
StringBuilder inputStringBuilder = new StringBuilder();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(response.getBody(), "UTF-8"));
String line = bufferedReader.readLine();
while (line != null) {
inputStringBuilder.append(line);
inputStringBuilder.append('\n');
line = bufferedReader.readLine();
}
log.info("============================response begin==========================================");
log.debug("Status code : {}", response.getStatusCode());
log.debug("Status text : {}", response.getStatusText());
log.debug("Headers : {}", response.getHeaders());
log.debug("Response body: {}", inputStringBuilder.toString());
log.info("=======================response end=================================================");
}
}
Then instantiate RestTemplate
using a BufferingClientHttpRequestFactory
and the LoggingRequestInterceptor
:
RestTemplate restTemplate = new RestTemplate(new BufferingClientHttpRequestFactory(new SimpleClientHttpRequestFactory()));
List<ClientHttpRequestInterceptor> interceptors = new ArrayList<>();
interceptors.add(new LoggingRequestInterceptor());
restTemplate.setInterceptors(interceptors);
The BufferingClientHttpRequestFactory
is required as we want to use the response body both in the interceptor and for the initial calling code. The default implementation allows to read the response body only once.
Solution 2 - Java
in Spring Boot you can get the full request/response by setting this in properties (or other 12 factor method)
logging.level.org.apache.http=DEBUG
this outputs
-DEBUG .i.c.DefaultHttpClientConnectionOperator : Connecting to localhost/127.0.0.1:41827
-DEBUG .i.c.DefaultHttpClientConnectionOperator : Connection established 127.0.0.1:39546<->127.0.0.1:41827
-DEBUG o.a.http.impl.execchain.MainClientExec : Executing request POST /v0/users HTTP/1.1
-DEBUG o.a.http.impl.execchain.MainClientExec : Target auth state: UNCHALLENGED
-DEBUG o.a.http.impl.execchain.MainClientExec : Proxy auth state: UNCHALLENGED
-DEBUG org.apache.http.headers : http-outgoing-0 >> POST /v0/users HTTP/1.1
-DEBUG org.apache.http.headers : http-outgoing-0 >> Content-Type: application/json;charset=UTF-8
-DEBUG org.apache.http.headers : http-outgoing-0 >> Content-Length: 56
-DEBUG org.apache.http.headers : http-outgoing-0 >> Host: localhost:41827
-DEBUG org.apache.http.headers : http-outgoing-0 >> Connection: Keep-Alive
-DEBUG org.apache.http.headers : http-outgoing-0 >> User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_102)
-DEBUG org.apache.http.headers : http-outgoing-0 >> Accept-Encoding: gzip,deflate
-DEBUG org.apache.http.wire : http-outgoing-0 >> "POST /v0/users HTTP/1.1[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "Content-Type: application/json;charset=UTF-8[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "Content-Length: 56[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "Host: localhost:41827[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_102)[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "{"id":null,"email":"xenoterracide@gmail.com","new":true}"
and response
-DEBUG .i.c.DefaultHttpClientConnectionOperator : Connecting to localhost/127.0.0.1:41827
-DEBUG .i.c.DefaultHttpClientConnectionOperator : Connection established 127.0.0.1:39546<->127.0.0.1:41827
-DEBUG o.a.http.impl.execchain.MainClientExec : Executing request POST /v0/users HTTP/1.1
-DEBUG o.a.http.impl.execchain.MainClientExec : Target auth state: UNCHALLENGED
-DEBUG o.a.http.impl.execchain.MainClientExec : Proxy auth state: UNCHALLENGED
-DEBUG org.apache.http.headers : http-outgoing-0 >> POST /v0/users HTTP/1.1
-DEBUG org.apache.http.headers : http-outgoing-0 >> Content-Type: application/json;charset=UTF-8
-DEBUG org.apache.http.headers : http-outgoing-0 >> Content-Length: 56
-DEBUG org.apache.http.headers : http-outgoing-0 >> Host: localhost:41827
-DEBUG org.apache.http.headers : http-outgoing-0 >> Connection: Keep-Alive
-DEBUG org.apache.http.headers : http-outgoing-0 >> User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_102)
-DEBUG org.apache.http.headers : http-outgoing-0 >> Accept-Encoding: gzip,deflate
-DEBUG org.apache.http.wire : http-outgoing-0 >> "POST /v0/users HTTP/1.1[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "Content-Type: application/json;charset=UTF-8[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "Content-Length: 56[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "Host: localhost:41827[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_102)[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "[\r][\n]"
-DEBUG org.apache.http.wire : http-outgoing-0 >> "{"id":null,"email":"xenoterracide@gmail.com","new":true}"
or just logging.level.org.apache.http.wire=DEBUG
which seems to contain all of the relevant information
Solution 3 - Java
Extending @hstoerr answer with some code:
Create LoggingRequestInterceptor to log requests responses
public class LoggingRequestInterceptor implements ClientHttpRequestInterceptor {
private static final Logger log = LoggerFactory.getLogger(LoggingRequestInterceptor.class);
@Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
ClientHttpResponse response = execution.execute(request, body);
log(request,body,response);
return response;
}
private void log(HttpRequest request, byte[] body, ClientHttpResponse response) throws IOException {
//do logging
}
}
Setup RestTemplate
RestTemplate rt = new RestTemplate();
//set interceptors/requestFactory
ClientHttpRequestInterceptor ri = new LoggingRequestInterceptor();
List<ClientHttpRequestInterceptor> ris = new ArrayList<ClientHttpRequestInterceptor>();
ris.add(ri);
rt.setInterceptors(ris);
rt.setRequestFactory(new BufferingClientHttpRequestFactory(new SimpleClientHttpRequestFactory());
Solution 4 - Java
Your best bet is to add logging.level.org.springframework.web.client.RestTemplate=DEBUG
to the application.properties
file.
Other solutions like setting log4j.logger.httpclient.wire
will not always work because they assume you use log4j
and Apache HttpClient
, which is not always true.
Note, however, that this syntax will work only on latest versions of Spring Boot.
Solution 5 - Java
You can use spring-rest-template-logger to log RestTemplate
HTTP traffic.
Add a dependency to your Maven project:
<dependency>
<groupId>org.hobsoft.spring</groupId>
<artifactId>spring-rest-template-logger</artifactId>
<version>2.0.0</version>
</dependency>
Then customize your RestTemplate
as follows:
RestTemplate restTemplate = new RestTemplateBuilder()
.customizers(new LoggingCustomizer())
.build()
Ensure that debug logging is enabled in application.properties
:
logging.level.org.hobsoft.spring.resttemplatelogger.LoggingCustomizer = DEBUG
Now all RestTemplate HTTP traffic will be logged to org.hobsoft.spring.resttemplatelogger.LoggingCustomizer
at debug level.
DISCLAIMER: I wrote this library.
Solution 6 - Java
The solution given by xenoterracide to use
logging.level.org.apache.http=DEBUG
is good but the problem is that by default Apache HttpComponents is not used.
To use Apache HttpComponents add to your pom.xml
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpasyncclient</artifactId>
</dependency>
and configure RestTemplate
with :
RestTemplate restTemplate = new RestTemplate();
restTemplate.setRequestFactory(new HttpComponentsAsyncClientHttpRequestFactory());
Solution 7 - Java
None of these answers actually solve 100% of the problem. mjj1409 gets most of it, but conveniently avoids the issue of logging the response, which takes a bit more work. Paul Sabou provides a solution that seems realistic, but doesn't provide enough details to actually implement (and it didn't work at all for me). Sofiene got the logging but with a critical problem: the response is no longer readable because the input stream has already been consumed!
I recommend using a BufferingClientHttpResponseWrapper to wrap the response object to allow reading the response body multiple times:
public class LoggingRequestInterceptor implements ClientHttpRequestInterceptor {
private static final Logger logger = LoggerFactory.getLogger(LoggingRequestInterceptor.class);
@Override
public ClientHttpResponse intercept(final HttpRequest request, final byte[] body,
final ClientHttpRequestExecution execution) throws IOException {
ClientHttpResponse response = execution.execute(request, body);
response = log(request, body, response);
return response;
}
private ClientHttpResponse log(final HttpRequest request, final byte[] body, final ClientHttpResponse response) {
final ClientHttpResponse responseCopy = new BufferingClientHttpResponseWrapper(response);
logger.debug("Method: ", request.getMethod().toString());
logger.debug("URI: ", , request.getURI().toString());
logger.debug("Request Body: " + new String(body));
logger.debug("Response body: " + IOUtils.toString(responseCopy.getBody()));
return responseCopy;
}
}
This will not consume the InputStream because the response body is loaded into memory and can be read multiple times. If you do not have the BufferingClientHttpResponseWrapper on your classpath, you can find the simple implementation here:
For setting up the RestTemplate:
LoggingRequestInterceptor loggingInterceptor = new LoggingRequestInterceptor();
restTemplate.getInterceptors().add(loggingInterceptor);
Solution 8 - Java
I finally found a way to do this in the right way. Most of the solution comes from https://stackoverflow.com/questions/3387441/how-do-i-configure-spring-and-slf4j-so-that-i-can-get-logging
It seems there are two things that need to be done :
- Add the following line in log4j.properties :
log4j.logger.httpclient.wire=DEBUG
- Make sure spring doesn't ignore your logging config
The second issue happens mostly to spring environments where slf4j is used (as it was my case). As such, when slf4j is used make sure that the following two things happen :
-
There is no commons-logging library in your classpath : this can be done by adding the exclusion descriptors in your pom :
<exclusions><exclusion> <groupId>commons-logging</groupId> <artifactId>commons-logging</artifactId> </exclusion> </exclusions>
-
The log4j.properties file is stored somewhere in the classpath where spring can find/see it. If you have problems with this, a last resort solution would be to put the log4j.properties file in the default package (not a good practice but just to see that things work as you expect)
Solution 9 - Java
Logging RestTemplate
Option 1. Open debug logging.
Configurate RestTemplate
-
By default the RestTemplate relies on standard JDK facilities to establish HTTP connections. You can switch to use a different HTTP library such as Apache HttpComponents
@Bean public RestTemplate restTemplate(RestTemplateBuilder builder) { RestTemplate restTemplate = builder.build(); return restTemplate; }
Configurate logging
-
application.yml
logging: level: org.springframework.web.client.RestTemplate: DEBUG
Option 2. Using Interceptor
Wrapper Response
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.client.ClientHttpResponse;
import org.springframework.util.StreamUtils;
public final class BufferingClientHttpResponseWrapper implements ClientHttpResponse {
private final ClientHttpResponse response;
private byte[] body;
BufferingClientHttpResponseWrapper(ClientHttpResponse response) {
this.response = response;
}
public HttpStatus getStatusCode() throws IOException {
return this.response.getStatusCode();
}
public int getRawStatusCode() throws IOException {
return this.response.getRawStatusCode();
}
public String getStatusText() throws IOException {
return this.response.getStatusText();
}
public HttpHeaders getHeaders() {
return this.response.getHeaders();
}
public InputStream getBody() throws IOException {
if (this.body == null) {
this.body = StreamUtils.copyToByteArray(this.response.getBody());
}
return new ByteArrayInputStream(this.body);
}
public void close() {
this.response.close();
}
}
Implement Interceptor
package com.example.logging;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.HttpRequest;
import org.springframework.http.client.ClientHttpRequestExecution;
import org.springframework.http.client.ClientHttpRequestInterceptor;
import org.springframework.http.client.ClientHttpResponse;
public class LoggingRestTemplate implements ClientHttpRequestInterceptor {
private final static Logger LOGGER = LoggerFactory.getLogger(LoggingRestTemplate.class);
@Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body,
ClientHttpRequestExecution execution) throws IOException {
traceRequest(request, body);
ClientHttpResponse response = execution.execute(request, body);
return traceResponse(response);
}
private void traceRequest(HttpRequest request, byte[] body) throws IOException {
if (!LOGGER.isDebugEnabled()) {
return;
}
LOGGER.debug(
"==========================request begin==============================================");
LOGGER.debug("URI : {}", request.getURI());
LOGGER.debug("Method : {}", request.getMethod());
LOGGER.debug("Headers : {}", request.getHeaders());
LOGGER.debug("Request body: {}", new String(body, "UTF-8"));
LOGGER.debug(
"==========================request end================================================");
}
private ClientHttpResponse traceResponse(ClientHttpResponse response) throws IOException {
if (!LOGGER.isDebugEnabled()) {
return response;
}
final ClientHttpResponse responseWrapper = new BufferingClientHttpResponseWrapper(response);
StringBuilder inputStringBuilder = new StringBuilder();
BufferedReader bufferedReader = new BufferedReader(
new InputStreamReader(responseWrapper.getBody(), "UTF-8"));
String line = bufferedReader.readLine();
while (line != null) {
inputStringBuilder.append(line);
inputStringBuilder.append('\n');
line = bufferedReader.readLine();
}
LOGGER.debug(
"==========================response begin=============================================");
LOGGER.debug("Status code : {}", responseWrapper.getStatusCode());
LOGGER.debug("Status text : {}", responseWrapper.getStatusText());
LOGGER.debug("Headers : {}", responseWrapper.getHeaders());
LOGGER.debug("Response body: {}", inputStringBuilder.toString());
LOGGER.debug(
"==========================response end===============================================");
return responseWrapper;
}
}
Configurate RestTemplate
@Bean
public RestTemplate restTemplate(RestTemplateBuilder builder) {
RestTemplate restTemplate = builder.build();
restTemplate.setInterceptors(Collections.singletonList(new LoggingRestTemplate()));
return restTemplate;
}
Configurate logging
-
Check the package of LoggingRestTemplate, for example in
application.yml
:logging: level: com.example.logging: DEBUG
Option 3. Using httpcomponent
Import httpcomponent dependency
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpasyncclient</artifactId>
Configurate RestTemplate
@Bean
public RestTemplate restTemplate(RestTemplateBuilder builder) {
RestTemplate restTemplate = builder.build();
restTemplate.setRequestFactory(new HttpComponentsAsyncClientHttpRequestFactory());
return restTemplate;
}
Configurate logging
-
Check the package of LoggingRestTemplate, for example in
application.yml
:logging: level: org.apache.http: DEBUG
Solution 10 - Java
---- July 2019 ----
(using Spring Boot)
I was surprised that Spring Boot, with all it's Zero Configuration magic, doesn't provide an easy way to inspect or log a simple JSON response body with RestTemplate. I looked through the various answers and comments provided here, and am sharing my own distilled version of what (still) works and seems to me like a reasonable solution, given the current options (I'm using Spring Boot 2.1.6 with Gradle 4.4)
1. Using Fiddler as http proxy
This is actually quite an elegant solution, as it bypasses all the cumbersome efforts of creating your own interceptor or changing the underlying http client to apache (see below).
> Install and run Fiddler
and then
> add -DproxySet=true -Dhttp.proxyHost=localhost -Dhttp.proxyPort=8888
to your VM Options
2. Using Apache HttpClient
Add Apache HttpClient to your Maven or Gradle dependencies.
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.9</version>
</dependency>
Use HttpComponentsClientHttpRequestFactory
as RequestFactory for RestTemplate. The simplest way to do that would be:
> RestTemplate restTemplate = new RestTemplate();
>
> restTemplate.setRequestFactory(new HttpComponentsClientHttpRequestFactory());
Enable DEBUG in your application.properties
file (if you're using Spring Boot)
> logging.level.org.apache.http=DEBUG
If you're using Spring Boot, you'll need to make sure you have a logging framework set up, e.g. by using a spring-boot-starter dependency that includes spring-boot-starter-logging
.
3. Use an Interceptor
I'll let you read through the proposals, counter-proposals, and gotchas in the other answers and comments and decide for yourself if you want to go down that path.
4. Log URL and Response Status without Body
Although this doesn't meet the stated requirements of logging the body, it's a quick and simple way to start logging your REST calls. It displays the full URL and response status.
Simply add the following line to your application.properties
file (assuming you're using Spring Boot, and assuming you are using a spring boot starter dependency that includes spring-boot-starter-logging
)
> logging.level.org.springframework.web.client.RestTemplate=DEBUG
The output will look something like this:
2019-07-29 11:53:50.265 DEBUG o.s.web.client.RestTemplate : HTTP GET http://www.myrestservice.com/Endpoint?myQueryParam=myValue
2019-07-29 11:53:50.276 DEBUG o.s.web.client.RestTemplate : Accept=[application/json]
2019-07-29 11:53:50.584 DEBUG o.s.web.client.RestTemplate : Response 200 OK
2019-07-29 11:53:50.585 DEBUG o.s.web.client.RestTemplate : Reading to [org.mynamespace.MyJsonModelClass]
Solution 11 - Java
As stated in the other responses, the response body needs special treatment so it can be read repeatedly (by default, its contents get consumed on the first read).
Instead of using the BufferingClientHttpRequestFactory
when setting up the request, the interceptor itself can wrap the response and make sure the content is retained and can be repeatedly read (by the logger as well as by the consumer of the response):
My interceptor, which
- buffers the response body using a wrapper
- logs in a more compact way
- logs the status code identifier as well (e.g. 201 Created)
- includes a request sequence number allowing to easily distinguish concurrent log entries from multiple threads
Code:
public class LoggingInterceptor implements ClientHttpRequestInterceptor {
private final Logger log = LoggerFactory.getLogger(getClass());
private AtomicInteger requestNumberSequence = new AtomicInteger(0);
@Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
int requestNumber = requestNumberSequence.incrementAndGet();
logRequest(requestNumber, request, body);
ClientHttpResponse response = execution.execute(request, body);
response = new BufferedClientHttpResponse(response);
logResponse(requestNumber, response);
return response;
}
private void logRequest(int requestNumber, HttpRequest request, byte[] body) {
if (log.isDebugEnabled()) {
String prefix = requestNumber + " > ";
log.debug("{} Request: {} {}", prefix, request.getMethod(), request.getURI());
log.debug("{} Headers: {}", prefix, request.getHeaders());
if (body.length > 0) {
log.debug("{} Body: \n{}", prefix, new String(body, StandardCharsets.UTF_8));
}
}
}
private void logResponse(int requestNumber, ClientHttpResponse response) throws IOException {
if (log.isDebugEnabled()) {
String prefix = requestNumber + " < ";
log.debug("{} Response: {} {} {}", prefix, response.getStatusCode(), response.getStatusCode().name(), response.getStatusText());
log.debug("{} Headers: {}", prefix, response.getHeaders());
String body = StreamUtils.copyToString(response.getBody(), StandardCharsets.UTF_8);
if (body.length() > 0) {
log.debug("{} Body: \n{}", prefix, body);
}
}
}
/**
* Wrapper around ClientHttpResponse, buffers the body so it can be read repeatedly (for logging & consuming the result).
*/
private static class BufferedClientHttpResponse implements ClientHttpResponse {
private final ClientHttpResponse response;
private byte[] body;
public BufferedClientHttpResponse(ClientHttpResponse response) {
this.response = response;
}
@Override
public HttpStatus getStatusCode() throws IOException {
return response.getStatusCode();
}
@Override
public int getRawStatusCode() throws IOException {
return response.getRawStatusCode();
}
@Override
public String getStatusText() throws IOException {
return response.getStatusText();
}
@Override
public void close() {
response.close();
}
@Override
public InputStream getBody() throws IOException {
if (body == null) {
body = StreamUtils.copyToByteArray(response.getBody());
}
return new ByteArrayInputStream(body);
}
@Override
public HttpHeaders getHeaders() {
return response.getHeaders();
}
}
}
Configuration:
@Bean
public RestTemplateBuilder restTemplateBuilder() {
return new RestTemplateBuilder()
.additionalInterceptors(Collections.singletonList(new LoggingInterceptor()));
}
Example log output:
2018-10-08 10:58:53 [main] DEBUG x.y.z.LoggingInterceptor - 2 > Request: POST http://localhost:53969/payment/v4/private/payment-lists/10022/templates
2018-10-08 10:58:53 [main] DEBUG x.y.z.LoggingInterceptor - 2 > Headers: {Accept=[application/json, application/json], Content-Type=[application/json;charset=UTF-8], Content-Length=[986]}
2018-10-08 10:58:53 [main] DEBUG x.y.z.LoggingInterceptor - 2 > Body:
{"idKey":null, ...}
2018-10-08 10:58:53 [main] DEBUG x.y.z.LoggingInterceptor - 2 < Response: 200 OK
2018-10-08 10:58:53 [main] DEBUG x.y.z.LoggingInterceptor - 2 < Headers: {Content-Type=[application/json;charset=UTF-8], Transfer-Encoding=[chunked], Date=[Mon, 08 Oct 2018 08:58:53 GMT]}
2018-10-08 10:58:53 [main] DEBUG x.y.z.LoggingInterceptor - 2 < Body:
{ "idKey" : "10022", ... }
Solution 12 - Java
Besides the HttpClient logging described in the other answer, you can also introduce a ClientHttpRequestInterceptor that reads the body of the request and the response and logs it. You might want to do this if other stuff also uses the HttpClient, or if you want a custom logging format. Caution: you will want to give the RestTemplate a BufferingClientHttpRequestFactory such that you can read the response twice.
Solution 13 - Java
application.properties
logging.level.org.springframework.web.client=DEBUG
application.yml
logging:
level:
root: WARN
org.springframework.web.client: DEBUG
Solution 14 - Java
This might not be the correct way to do it, but I think this is the most simple approach to print requests and responses without filling too much in logs.
By adding below 2 lines application.properties logs all requests and responses 1st line in order to log the requests and 2nd line to log the responses.
logging.level.org.springframework.web.client.RestTemplate=DEBUG
logging.level.org.springframework.web.servlet.mvc.method.annotation.HttpEntityMethodProcessor=DEBUG
Solution 15 - Java
Assuming RestTemplate
is configured to use HttpClient 4.x, you can read up on HttpClient's logging documentation here. The loggers are different than those specified in the other answers.
The logging configuration for HttpClient 3.x is available here.
Solution 16 - Java
So many responses here require coding changes and customized classes and it really is not necessary. Gte a debugging proxy such as fiddler and set your java environment to use the proxy on the command line (-Dhttp.proxyHost and -Dhttp.proxyPort) then run fiddler and you can see the requests and responses in their entirety. Also comes with many ancillary advantages such as the ability to tinker with the results and responses before and after they are sent to run experiments before committing to modification of the server.
Last bit of an issue that can come up is if you must use HTTPS, you will need to export the SSL cert from fiddler and import it into the java keystore (cacerts) hint: default java keystore password is usually "changeit".
Solution 17 - Java
For logging to Logback with help from Apache HttpClient:
You need Apache HttpClient in classpath:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.10</version>
</dependency>
Configure your RestTemplate
to use HttpClient:
restTemplate.setRequestFactory(new HttpComponentsClientHttpRequestFactory());
To log requests and responses, add to Logback configuration file:
<logger name="org.apache.http.wire" level="DEBUG"/>
Or to log even more:
<logger name="org.apache.http" level="DEBUG"/>
Solution 18 - Java
Adding to above discussion this only represents Happy scenarios. probably you will not be able to log the response if an Error comes .
In this case plus all the cases above you must override DefaultResponseErrorHandler and set it like below
restTemplate.setErrorHandler(new DefaultResponseErrorHandlerImpl());
Solution 19 - Java
Strangely, none of these solutions work as RestTemplate does not seem to return the response on some client and server 500x errors. In which case, you will have log those as well by implementing ResponseErrorHandler as follows. Here is a draft code, but you get the point:
You can set the same interceptor as the error handler:
restTemplate.getInterceptors().add(interceptor);
restTemplate.setRequestFactory(new BufferingClientHttpRequestFactory(new SimpleClientHttpRequestFactory()));
restTemplate.setErrorHandler(interceptor);
And the intercept implements both interfaces:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.HashSet;
import java.util.Set;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.HttpRequest;
import org.springframework.http.HttpStatus.Series;
import org.springframework.http.client.ClientHttpRequestExecution;
import org.springframework.http.client.ClientHttpRequestInterceptor;
import org.springframework.http.client.ClientHttpResponse;
import org.springframework.web.client.DefaultResponseErrorHandler;
import org.springframework.web.client.ResponseErrorHandler;
public class LoggingRequestInterceptor implements ClientHttpRequestInterceptor, ResponseErrorHandler {
static final Logger log = LoggerFactory.getLogger(LoggingRequestInterceptor.class);
static final DefaultResponseErrorHandler defaultResponseErrorHandler = new DefaultResponseErrorHandler();
final Set<Series> loggableStatuses = new HashSet();
public LoggingRequestInterceptor() {
}
public LoggingRequestInterceptor(Set<Series> loggableStatuses) {
loggableStatuses.addAll(loggableStatuses);
}
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
this.traceRequest(request, body);
ClientHttpResponse response = execution.execute(request, body);
if(response != null) {
this.traceResponse(response);
}
return response;
}
private void traceRequest(HttpRequest request, byte[] body) throws IOException {
log.debug("===========================request begin================================================");
log.debug("URI : {}", request.getURI());
log.debug("Method : {}", request.getMethod());
log.debug("Headers : {}", request.getHeaders());
log.debug("Request body: {}", new String(body, "UTF-8"));
log.debug("==========================request end================================================");
}
private void traceResponse(ClientHttpResponse response) throws IOException {
if(this.loggableStatuses.isEmpty() || this.loggableStatuses.contains(response.getStatusCode().series())) {
StringBuilder inputStringBuilder = new StringBuilder();
try {
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(response.getBody(), "UTF-8"));
for(String line = bufferedReader.readLine(); line != null; line = bufferedReader.readLine()) {
inputStringBuilder.append(line);
inputStringBuilder.append('\n');
}
} catch (Throwable var5) {
log.error("cannot read response due to error", var5);
}
log.debug("============================response begin==========================================");
log.debug("Status code : {}", response.getStatusCode());
log.debug("Status text : {}", response.getStatusText());
log.debug("Headers : {}", response.getHeaders());
log.debug("Response body: {}", inputStringBuilder.toString());
log.debug("=======================response end=================================================");
}
}
public boolean hasError(ClientHttpResponse response) throws IOException {
return defaultResponseErrorHandler.hasError(response);
}
public void handleError(ClientHttpResponse response) throws IOException {
this.traceResponse(response);
defaultResponseErrorHandler.handleError(response);
}
}
Solution 20 - Java
The trick of configuring your RestTemplate
with a BufferingClientHttpRequestFactory
doesn't work if you are using any ClientHttpRequestInterceptor
, which you will if you are trying to log via interceptors. This is due to the way that InterceptingHttpAccessor
(which RestTemplate
subclasses) works.
Long story short... just use this class in place of RestTemplate
(note this uses the SLF4J logging API, edit as needed):
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.Constructor;
import java.nio.charset.StandardCharsets;
import java.util.List;
import java.util.Map;
import javax.annotation.PostConstruct;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpRequest;
import org.springframework.http.HttpStatus;
import org.springframework.http.client.ClientHttpRequestExecution;
import org.springframework.http.client.ClientHttpRequestInterceptor;
import org.springframework.http.client.ClientHttpResponse;
import org.springframework.web.client.RestTemplate;
/**
* A {@link RestTemplate} that logs every request and response.
*/
public class LoggingRestTemplate extends RestTemplate {
// Bleh, this class is not public
private static final String RESPONSE_WRAPPER_CLASS = "org.springframework.http.client.BufferingClientHttpResponseWrapper";
private Logger log = LoggerFactory.getLogger(this.getClass());
private boolean hideAuthorizationHeaders = true;
private Class<?> wrapperClass;
private Constructor<?> wrapperConstructor;
/**
* Configure the logger to log requests and responses to.
*
* @param log log destination, or null to disable
*/
public void setLogger(Logger log) {
this.log = log;
}
/**
* Configure the logger to log requests and responses to by name.
*
* @param name name of the log destination, or null to disable
*/
public void setLoggerName(String name) {
this.setLogger(name != null ? LoggerFactory.getLogger(name) : null);
}
/**
* Configure whether to hide the contents of {@code Authorization} headers.
*
* <p>
* Default true.
*
* @param hideAuthorizationHeaders true to hide, otherwise false
*/
public void setHideAuthorizationHeaders(boolean hideAuthorizationHeaders) {
this.hideAuthorizationHeaders = hideAuthorizationHeaders;
}
/**
* Log a request.
*/
protected void traceRequest(HttpRequest request, byte[] body) {
this.log.debug("xmit: {} {}\n{}{}", request.getMethod(), request.getURI(), this.toString(request.getHeaders()),
body != null && body.length > 0 ? "\n\n" + new String(body, StandardCharsets.UTF_8) : "");
}
/**
* Log a response.
*/
protected void traceResponse(ClientHttpResponse response) {
final ByteArrayOutputStream bodyBuf = new ByteArrayOutputStream();
HttpStatus statusCode = null;
try {
statusCode = response.getStatusCode();
} catch (IOException e) {
// ignore
}
String statusText = null;
try {
statusText = response.getStatusText();
} catch (IOException e) {
// ignore
}
try (final InputStream input = response.getBody()) {
byte[] b = new byte[1024];
int r;
while ((r = input.read(b)) != -1)
bodyBuf.write(b, 0, r);
} catch (IOException e) {
// ignore
}
this.log.debug("recv: {} {}\n{}{}", statusCode, statusText, this.toString(response.getHeaders()),
bodyBuf.size() > 0 ? "\n\n" + new String(bodyBuf.toByteArray(), StandardCharsets.UTF_8) : "");
}
@PostConstruct
private void addLoggingInterceptor() {
this.getInterceptors().add(new ClientHttpRequestInterceptor() {
@Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution)
throws IOException {
// Log request
if (LoggingRestTemplate.this.log != null && LoggingRestTemplate.this.log.isDebugEnabled())
LoggingRestTemplate.this.traceRequest(request, body);
// Perform request
ClientHttpResponse response = execution.execute(request, body);
// Log response
if (LoggingRestTemplate.this.log != null && LoggingRestTemplate.this.log.isDebugEnabled()) {
final ClientHttpResponse bufferedResponse = LoggingRestTemplate.this.ensureBuffered(response);
if (bufferedResponse != null) {
LoggingRestTemplate.this.traceResponse(bufferedResponse);
response = bufferedResponse;
}
}
// Done
return response;
}
});
}
private ClientHttpResponse ensureBuffered(ClientHttpResponse response) {
try {
if (this.wrapperClass == null)
this.wrapperClass = Class.forName(RESPONSE_WRAPPER_CLASS, false, ClientHttpResponse.class.getClassLoader());
if (!this.wrapperClass.isInstance(response)) {
if (this.wrapperConstructor == null) {
this.wrapperConstructor = this.wrapperClass.getDeclaredConstructor(ClientHttpResponse.class);
this.wrapperConstructor.setAccessible(true);
}
response = (ClientHttpResponse)this.wrapperConstructor.newInstance(response);
}
return response;
} catch (Exception e) {
this.log.error("error creating {} instance: {}", RESPONSE_WRAPPER_CLASS, e);
return null;
}
}
private String toString(HttpHeaders headers) {
final StringBuilder headerBuf = new StringBuilder();
for (Map.Entry<String, List<String>> entry : headers.entrySet()) {
if (headerBuf.length() > 0)
headerBuf.append('\n');
final String name = entry.getKey();
for (String value : entry.getValue()) {
if (this.hideAuthorizationHeaders && name.equalsIgnoreCase(HttpHeaders.AUTHORIZATION))
value = "[omitted]";
headerBuf.append(name).append(": ").append(value);
}
}
return headerBuf.toString();
}
}
I agree it's silly that it takes this much work just to do this.
Solution 21 - Java
As @MilacH pointed out, there is an error in the implementation. If an statusCode > 400 is returned a IOException is thrown, as the errorHandler is not invoked, from interceptors. The exception can be ignored and is then caught again in the handler method.
package net.sprd.fulfillment.common;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.HttpRequest;
import org.springframework.http.client.ClientHttpRequestExecution;
import org.springframework.http.client.ClientHttpRequestInterceptor;
import org.springframework.http.client.ClientHttpResponse;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import static java.nio.charset.StandardCharsets.UTF_8;
public class LoggingRequestInterceptor implements ClientHttpRequestInterceptor {
final static Logger log = LoggerFactory.getLogger(LoggingRequestInterceptor.class);
@SuppressWarnings("HardcodedLineSeparator")
public static final char LINE_BREAK = '\n';
@Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
try {
traceRequest(request, body);
} catch (Exception e) {
log.warn("Exception in LoggingRequestInterceptor while tracing request", e);
}
ClientHttpResponse response = execution.execute(request, body);
try {
traceResponse(response);
} catch (IOException e) {
// ignore the exception here, as it will be handled by the error handler of the restTemplate
log.warn("Exception in LoggingRequestInterceptor", e);
}
return response;
}
private void traceRequest(HttpRequest request, byte[] body) {
log.info("===========================request begin================================================");
log.info("URI : {}", request.getURI());
log.info("Method : {}", request.getMethod());
log.info("Headers : {}", request.getHeaders());
log.info("Request body: {}", new String(body, UTF_8));
log.info("==========================request end================================================");
}
private void traceResponse(ClientHttpResponse response) throws IOException {
StringBuilder inputStringBuilder = new StringBuilder();
try (BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(response.getBody(), UTF_8))) {
String line = bufferedReader.readLine();
while (line != null) {
inputStringBuilder.append(line);
inputStringBuilder.append(LINE_BREAK);
line = bufferedReader.readLine();
}
}
log.info("============================response begin==========================================");
log.info("Status code : {}", response.getStatusCode());
log.info("Status text : {}", response.getStatusText());
log.info("Headers : {}", response.getHeaders());
log.info("Response body: {}", inputStringBuilder);
log.info("=======================response end=================================================");
}
}
Solution 22 - Java
org.apache.http.wire
gives too unreadable logs, so I use logbook to log application Servlet and RestTemplate requests & responses with payloads.
build.gradle
:
compile group: 'org.zalando', name: 'logbook-spring-boot-starter', version: '2.6.2'
or Maven dependency:
<dependency>
<groupId>org.zalando</groupId>
<artifactId>logbook-spring-boot-starter</artifactId>
<version>2.6.2</version>
</dependency>
application.properties
(or trough YAML):
logging.level.org.zalando.logbook = TRACE
RestTemplate.java
:
import java.util.function.Supplier;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.springframework.boot.web.client.RestTemplateBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.client.ClientHttpRequestFactory;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.web.client.RestTemplate;
import org.zalando.logbook.httpclient.LogbookHttpRequestInterceptor;
import org.zalando.logbook.httpclient.LogbookHttpResponseInterceptor;
@Configuration
public class RestTemplateConfiguration {
private final LogbookHttpRequestInterceptor logbookHttpRequestInterceptor;
private final LogbookHttpResponseInterceptor logbookHttpResponseInterceptor;
public RestTemplateConfiguration(LogbookHttpRequestInterceptor logbookHttpRequestInterceptor,
LogbookHttpResponseInterceptor logbookHttpResponseInterceptor) {
this.logbookHttpRequestInterceptor = logbookHttpRequestInterceptor;
this.logbookHttpResponseInterceptor = logbookHttpResponseInterceptor;
}
@Bean
public RestTemplate restTemplate(RestTemplateBuilder restTemplateBuilder) {
return restTemplateBuilder
.requestFactory(new MyRequestFactorySupplier())
.build();
}
class MyRequestFactorySupplier implements Supplier<ClientHttpRequestFactory> {
@Override
public ClientHttpRequestFactory get() {
// Using Apache HTTP client
CloseableHttpClient client = HttpClientBuilder.create()
.addInterceptorFirst(logbookHttpRequestInterceptor)
.addInterceptorFirst(logbookHttpResponseInterceptor)
.build();
return new HttpComponentsClientHttpRequestFactory(client);
}
}
}
Solution 23 - Java
Best solution now, just add dependency :
<dependency>
<groupId>com.github.zg2pro</groupId>
<artifactId>spring-rest-basis</artifactId>
<version>v.x</version>
</dependency>
It contains a LoggingRequestInterceptor class you can add that way to your RestTemplate:
integrate this utility by adding it as an interceptor to a spring RestTemplate, in the following manner:
restTemplate.setRequestFactory(LoggingRequestFactoryFactory.build());
and add an slf4j implementation to your framework like log4j.
or directly use "Zg2proRestTemplate". The "best answer" by @PaulSabou looks so so, since httpclient and all apache.http libs are not necessarily loaded when using a spring RestTemplate.
Solution 24 - Java
Wanted to add my implementation of this as well. I apologize for all the missing semi-colons, this is written in Groovy.
I needed something more configurable than the accepted answer provided. Here's a rest template bean that's very agile and will log everything like the OP is looking for.
Custom Logging Interceptor Class:
import org.springframework.http.HttpRequest
import org.springframework.http.client.ClientHttpRequestExecution
import org.springframework.http.client.ClientHttpRequestInterceptor
import org.springframework.http.client.ClientHttpResponse
import org.springframework.util.StreamUtils
import java.nio.charset.Charset
class HttpLoggingInterceptor implements ClientHttpRequestInterceptor {
private final static Logger log = LoggerFactory.getLogger(HttpLoggingInterceptor.class)
@Override
ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
logRequest(request, body)
ClientHttpResponse response = execution.execute(request, body)
logResponse(response)
return response
}
private void logRequest(HttpRequest request, byte[] body) throws IOException {
if (log.isDebugEnabled()) {
log.debug("===========================request begin================================================")
log.debug("URI : {}", request.getURI())
log.debug("Method : {}", request.getMethod())
log.debug("Headers : {}", request.getHeaders())
log.debug("Request body: {}", new String(body, "UTF-8"))
log.debug("==========================request end================================================")
}
}
private void logResponse(ClientHttpResponse response) throws IOException {
if (log.isDebugEnabled()) {
log.debug("============================response begin==========================================")
log.debug("Status code : {}", response.getStatusCode())
log.debug("Status text : {}", response.getStatusText())
log.debug("Headers : {}", response.getHeaders())
log.debug("Response body: {}", StreamUtils.copyToString(response.getBody(), Charset.defaultCharset()))
log.debug("=======================response end=================================================")
}
}
}
Rest Template Bean Definition:
@Bean(name = 'myRestTemplate')
RestTemplate myRestTemplate(RestTemplateBuilder builder) {
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(10 * 1000) // 10 seconds
.setSocketTimeout(300 * 1000) // 300 seconds
.build()
PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager()
connectionManager.setMaxTotal(10)
connectionManager.closeIdleConnections(5, TimeUnit.MINUTES)
CloseableHttpClient httpClient = HttpClients.custom()
.setConnectionManager(connectionManager)
.setDefaultRequestConfig(requestConfig)
.disableRedirectHandling()
.build()
RestTemplate restTemplate = builder
.rootUri("https://domain.server.com")
.basicAuthorization("username", "password")
.requestFactory(new BufferingClientHttpRequestFactory(new HttpComponentsClientHttpRequestFactory(httpClient)))
.interceptors(new HttpLoggingInterceptor())
.build()
return restTemplate
}
Implementation:
@Component
class RestService {
private final RestTemplate restTemplate
private final static Logger log = LoggerFactory.getLogger(RestService.class)
@Autowired
RestService(
@Qualifier("myRestTemplate") RestTemplate restTemplate
) {
this.restTemplate = restTemplate
}
// add specific methods to your service that access the GET and PUT methods
private <T> T getForObject(String path, Class<T> object, Map<String, ?> params = [:]) {
try {
return restTemplate.getForObject(path, object, params)
} catch (HttpClientErrorException e) {
log.warn("Client Error (${path}): ${e.responseBodyAsString}")
} catch (HttpServerErrorException e) {
String msg = "Server Error (${path}): ${e.responseBodyAsString}"
log.error(msg, e)
} catch (RestClientException e) {
String msg = "Error (${path})"
log.error(msg, e)
}
return null
}
private <T> T putForObject(String path, T object) {
try {
HttpEntity<T> request = new HttpEntity<>(object)
HttpEntity<T> response = restTemplate.exchange(path, HttpMethod.PUT, request, T)
return response.getBody()
} catch (HttpClientErrorException e) {
log.warn("Error (${path}): ${e.responseBodyAsString}")
} catch (HttpServerErrorException e) {
String msg = "Error (${path}): ${e.responseBodyAsString}"
log.error(msg, e)
} catch (RestClientException e) {
String msg = "Error (${path})"
log.error(msg, e)
}
return null
}
}
Solution 25 - Java
Refer the Q/A for logging the request and response for the rest template by enabling the multiple reads on the HttpInputStream
Solution 26 - Java
An easy way to solve the problem:
- Create a Bean of RestTemplate using RestTemplateBuilder: It will give you more control over connection time and reading time.
@Configuration
public class RestTemplateConfig {
@Bean
public RestTemplate restTemplate(RestTemplateBuilder builder) {
return builder
.setConnectTimeout(Duration.ofMillis(60000))
.setReadTimeout(Duration.ofMillis(60000))
.build();
}
}
- Add this line to the
resources/application.properties
file:logging.level.org.springframework.web.client.RestTemplate=DEBUG
Hope the problem will be resolved!
Solution 27 - Java
Related to the response using ClientHttpInterceptor, I found a way of keeping the whole response without Buffering factories. Just store the response body input stream inside byte array using some utils method that will copy that array from body, but important, surround this method with try catch because it will break if response is empty (that is the cause of Resource Access Exception) and in catch just create empty byte array, and than just create anonymous inner class of ClientHttpResponse using that array and other parameters from the original response. Than you can return that new ClientHttpResponse object to the rest template execution chain and you can log response using body byte array that is previously stored. That way you will avoid consuming InputStream in the actual response and you can use Rest Template response as it is. Note, this may be dangerous if your's response is too big
Solution 28 - Java
my logger config used xml
<logger name="org.springframework.web.client.RestTemplate">
<level value="trace"/>
</logger>
then you will get something like below:
DEBUG org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:92) : Reading [com.test.java.MyClass] as "application/json" using [org.springframework.http.converter.json.MappingJackson2HttpMessageConverter@604525f1]
through HttpMessageConverterExtractor.java:92,you need continue to debug,and in my case,i got this:
genericMessageConverter.write(requestBody, requestBodyType, requestContentType, httpRequest);
and this:
outputMessage.getBody().flush();
outputMessage.getBody() contains the message http(post type) sends