DataBufferLimitException: Exceeded limit on max bytes to buffer webflux error

While sending a file I receive an array of bytes. I always have a problem with webflux to receive an array. the error thrown as below :

org.springframework.core.io.buffer.DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144
	at org.springframework.core.io.buffer.LimitedDataBufferList.raiseLimitException(LimitedDataBufferList.java:101)
	Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException

Do you now how to resolve that in webflux ?

This worked for me:

1. Create a @Bean in one of your configuration classes or the main SpringBootApplication class:

   @Bean
   public WebClient webClient() {
       final int size = 16 * 1024 * 1024;
       final ExchangeStrategies strategies = ExchangeStrategies.builder()
           .codecs(codecs -> codecs.defaultCodecs().maxInMemorySize(size))
           .build();
       return WebClient.builder()
           .exchangeStrategies(strategies)
           .build();
   }
   

2. Next, go to your desired class where you want to use the WebClient:

   @Service
   public class TestService {
   
       @Autowired
       private WebClient webClient;
   
       public void test() {
           String out = webClient
               .get()
               .uri("/my/api/endpoint")
               .retrieve()
               .bodyToMono(String.class)
               .block();
   
           System.out.println(out);
       }
   }
   

I suppose this issue is about adding a new spring.codec.max-in-memory-size configuration property in Spring Boot. Add it to the application.yml file like:

spring:
  codec:
    max-in-memory-size: 10MB

Set the maximum bytes (in megabytes) in your Spring Boot application.properties configuration file like below:

spring.codec.max-in-memory-size=20MB

i was getting this error for a simple RestController (i post a large json string).

here is how i successfully changed the maxInMemorySize

import org.springframework.context.annotation.Configuration;
import org.springframework.http.codec.ServerCodecConfigurer;
import org.springframework.web.reactive.config.ResourceHandlerRegistry;
import org.springframework.web.reactive.config.WebFluxConfigurer;

@Configuration
public class WebfluxConfig implements WebFluxConfigurer {

    @Override
    public void addResourceHandlers(ResourceHandlerRegistry registry) {
        
        registry.addResourceHandler("/swagger-ui.html**")
            .addResourceLocations("classpath:/META-INF/resources/");

        registry.addResourceHandler("/webjars/**")
            .addResourceLocations("classpath:/META-INF/resources/webjars/");
    }

    @Override
    public void configureHttpMessageCodecs(ServerCodecConfigurer configurer) {
        configurer.defaultCodecs().maxInMemorySize(16 * 1024 * 1024);
    }
}

this was surprisingly hard to find

Instead of retrieving data at once, you can stream:

Mono<String> string = webClient.get()
	.uri("end point of an API")
	.retrieve()
	.bodyToFlux(DataBuffer.class)
	.map(buffer -> {
		String string = buffer.toString(Charset.forName("UTF-8"));
		DataBufferUtils.release(buffer);
		return string;
	});

Alternatively convert to stream:

	.map(b -> b.asInputStream(true))
	.reduce(SequenceInputStream::new)
	.map(stream -> {
		// consume stream
		stream.close();
		return string;
	});

In most cases you don't want to really aggregate the stream, rather than processing it directly. The need to load huge amount of data in memory is mostly a sign to change the approach to more reactive one. JSON- and XML-Parsers have streaming interfaces.

worked for me

webTestClient.mutate().codecs(configurer -> configurer
  .defaultCodecs()
  .maxInMemorySize(16 * 1024 * 1024)).build().get()
  .uri("/u/r/l")
  .exchange()
  .expectStatus()
  .isOk()

This worked for me

val exchangeStrategies = ExchangeStrategies.builder()
                .codecs { configurer: ClientCodecConfigurer -> configurer.defaultCodecs().maxInMemorySize(16 * 1024 * 1024) }.build()
        return WebClient.builder().exchangeStrategies(exchangeStrategies).build()

Another alternative could be creating a custom CodecCustomizer, which is going to be applied to both WebFlux and WebClient at the same time:

@Configuration
class MyAppConfiguration {

    companion object {
        private const val MAX_MEMORY_SIZE = 50 * 1024 * 1024 // 50 MB
    }

    @Bean
    fun codecCustomizer(): CodecCustomizer {
        return CodecCustomizer {
            it.defaultCodecs()
                .maxInMemorySize(MAX_MEMORY_SIZE)
        }
    }
}

As of Spring Boot 2.3.0, there is now a dedicated configuration property for the Reactive Elasticsearch REST client.

You can use the following configuration property to set a specific memory limit for the client.

spring.data.elasticsearch.client.reactive.max-in-memory-size= The already existing spring.codec.max-in-memory-size property is separate and only affects other WebClient instances in the application.