Spring-Batch without persisting metadata to database?

JavaSpringSpring Batch

Java Problem Overview

I want to create a spring-batch job, but I want to run it without any database persistence. Unfortunately spring-batch requires to write metadata ob the job cycles to a database somehow, thus procing me to provide at least some kind of db with transactionmanager and entitymanager.

It it possible to prevent the metadata and run independently from txmanagers and databases?

Update:

ERROR org.springframework.batch.core.job.AbstractJob: Encountered fatal error executing job
java.lang.NullPointerException
	at org.springframework.batch.core.repository.dao.MapJobExecutionDao.synchronizeStatus(MapJobExecutionDao.java:158) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
	at org.springframework.batch.core.repository.support.SimpleJobRepository.update(SimpleJobRepository.java:161) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]
	at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:98) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:262) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:95) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at com.sun.proxy.$Proxy134.update(Unknown Source) ~[?:?]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]
	at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at com.sun.proxy.$Proxy134.update(Unknown Source) ~[?:?]
	at org.springframework.batch.core.job.AbstractJob.updateStatus(AbstractJob.java:416) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
	at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:299) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
	at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
	at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]
	at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
	at com.sun.proxy.$Proxy50.run(Unknown Source) [?:?]

Java Solutions

Solution 1 - Java

Simply create a configuration without datasource for Batch configuration :

@Configuration
@EnableAutoConfiguration
@EnableBatchProcessing
public class BatchConfiguration extends DefaultBatchConfigurer {

	@Override
	public void setDataSource(DataSource dataSource) {
		// override to do not set datasource even if a datasource exist.
		// initialize will use a Map based JobRepository (instead of database)
	}

}

It will initialize JobRepository and JobExplorer with a in memory map based implementation. https://github.com/spring-projects/spring-batch/blob/342d27bc1ed83312bdcd9c0cb30510f4c469e47d/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/DefaultBatchConfigurer.java#L84

and you can use you're production datasource as well even if auto configured with spring boot.

Solution 2 - Java

> I want to run it without any database persistence

You can use MapJobRepositoryFactoryBean and ResourcelessTransactionManager

sample configuration:

<bean id="transactionManager"
	class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />

<bean id="jobRepository"
	class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
	<property name="transactionManager" ref="transactionManager" />
</bean>

<bean id="jobLauncher"
	class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
	<property name="jobRepository" ref="jobRepository" />
</bean>

For Spring 4.X, the annotation based configuration would be as follows:

@Bean
public PlatformTransactionManager getTransactionManager() {
	return new ResourcelessTransactionManager();
}

@Bean
public JobRepository getJobRepo() {
	return new MapJobRepositoryFactoryBean(getTransactionManager()).getObject();
}

Solution 3 - Java

After tweaking @Braj's answer, my working configuration looks as follows:

@Bean
public ResourcelessTransactionManager transactionManager() {
	return new ResourcelessTransactionManager();
}

@Bean
public JobRepository jobRepository(ResourcelessTransactionManager transactionManager) throws Exception {
	MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean = new MapJobRepositoryFactoryBean(transactionManager);
	mapJobRepositoryFactoryBean.setTransactionManager(transactionManager);
	return mapJobRepositoryFactoryBean.getObject();
}

@Bean
public SimpleJobLauncher jobLauncher(JobRepository jobRepository) {
	SimpleJobLauncher simpleJobLauncher = new SimpleJobLauncher();
	simpleJobLauncher.setJobRepository(jobRepository);
	return simpleJobLauncher;
}

Solution 4 - Java

I came back to my own question, as the solution did not work anymore. As of spring-batch-1.5.3 use as follows:

@SpringBootApplication(exclude = {DataSourceAutoConfiguration.class})
...
	@Bean
	public PlatformTransactionManager transactionManager() {
	    return new ResourcelessTransactionManager();
	}
}

Solution 5 - Java

If you don't want to store job's metadata in the database, configuration looks as follows

@Configuration
public class InMemoryJobRepositoryConfiguration {
@Bean
public ResourcelessTransactionManager transactionManager() {
    return new ResourcelessTransactionManager();
}

@Bean
public MapJobRepositoryFactoryBean mapJobRepositoryFactory(ResourcelessTransactionManager transactionManager)
        throws Exception {
    MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(transactionManager);
    factory.afterPropertiesSet();
    return factory;
}

@Bean
public JobRepository jobRepository(MapJobRepositoryFactoryBean repositoryFactory) throws Exception {
    return repositoryFactory.getObject();
}

@Bean
public JobExplorer jobExplorer(MapJobRepositoryFactoryBean repositoryFactory) {
    return new SimpleJobExplorer(repositoryFactory.getJobInstanceDao(), repositoryFactory.getJobExecutionDao(),
            repositoryFactory.getStepExecutionDao(), repositoryFactory.getExecutionContextDao());
}

@Bean
public SimpleJobLauncher jobLauncher(JobRepository jobRepository) {
    SimpleJobLauncher launcher = new SimpleJobLauncher();
    launcher.setJobRepository(jobRepository);
    return launcher;
}
}

If job needs to read/write business data to the database and the datasource is configured as follows.

@Autowired
private DataSource dataSource;

spring batch creates internal schema (BATCH_* tables and sequences) using that datasource. To prevent this from happening set flag spring.batch.initializer.enabled=false in application.properties.

Solution 6 - Java

I tried all of the solutions above, but it does not work with my particular scenario because my spring batch job has advanced features such as multi-threading. It needs a database. So here is what I did to solve the problem:

@Configuration
public class FakeBatchConfig implements BatchConfigurer {

  PlatformTransactionManager transactionManager;
  JobRepository jobRepository;
  JobLauncher jobLauncher;
  JobExplorer jobExplorer;

  @Override
  public JobRepository getJobRepository() {
    return jobRepository;
  }

  @Override
  public PlatformTransactionManager getTransactionManager() {
    return transactionManager;
  }

  @Override
  public JobLauncher getJobLauncher() {
    return jobLauncher;
  }

  @Override
  public JobExplorer getJobExplorer() {
    return jobExplorer;
  }

  @PostConstruct
  void initialize() throws Exception {

    if (this.transactionManager == null) {
      this.transactionManager = new ResourcelessTransactionManager();
    }

    MapJobRepositoryFactoryBean jobRepositoryFactory = new MapJobRepositoryFactoryBean(this.transactionManager);
    jobRepositoryFactory.afterPropertiesSet();
    this.jobRepository = jobRepositoryFactory.getObject();

    MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(jobRepositoryFactory);
    jobExplorerFactory.afterPropertiesSet();
    this.jobExplorer = jobExplorerFactory.getObject();
    this.jobLauncher = createJobLauncher();
  }

  private JobLauncher createJobLauncher() throws Exception {
    SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
    jobLauncher.setJobRepository(jobRepository);
    jobLauncher.afterPropertiesSet();
    return jobLauncher;
  }

}

Solution 7 - Java

In addition to @Braj-s response I had to exclude Batch autoconfiguration: @SpringBootApplication(exclude = {BatchAutoConfiguration.class})

It was not working otherwise. This includes Spring Boot application

Solution 8 - Java

After reading all the answers provided I got it working. I used mix of membersound and Aure77 solution. I found out there was no need to override setDataSource method. DefaultBatchConfigurer automatically takes care of PlatformTransactionManager and DataSource is not required. Please note I am using Spring boot 2.0.3.RELEASE. Following is the method I used

@SpringBootApplication(exclude = { DataSourceAutoConfiguration.class })
@EnableBatchProcessing
public class DemoApplication extends DefaultBatchConfigurer {

	@Autowired
	private JobBuilderFactory jobs;

	@Autowired
	private StepBuilderFactory steps;

	@Bean
	protected Tasklet tasklet() {

		return new Tasklet() {
			@Override
			public RepeatStatus execute(StepContribution contribution, ChunkContext context) {
				return RepeatStatus.FINISHED;
			}
		};
	}

	@Bean
	public Job job() throws Exception {
		return this.jobs.get("job").start(step1()).build();
	}

	@Bean
	protected Step step1() throws Exception {
		return this.steps.get("step1").tasklet(tasklet()).build();
	}

	public static void main(String[] args) throws Exception {
		//System.exit(SpringApplication.exit(SpringApplication.run(DemoApplication.class, args)));
		SpringApplication.run(DemoApplication.class, args);
	}
}

Solution 9 - Java

The simplest of all is add an embedded database to classpath. Naturally in production is not recommended but if you're using it to learn, save time.

Add to your pom.xml the embedded H2 database dependency:

<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <scope>runtime</scope>
</dependency>

That's all.

Solution 10 - Java

Following changes can be made to use the Spring Batch without having to create spring batch related metadata database structures.

This worked for me:

  1. Added bean of type ResourcelessTransactionManager and configured MapJobRepositoryFactoryBean.
  2. Using the configured bean MapJobRepositoryFactoryBean created a SimpleJoblauncher bean.
  3. Added annotation @EnableBatchProcessing below @SpringBootApplication.
  4. To override the defaults I had to add spring.main.allow-bean-definition-overriding=true in application.properties.
Categories
Recommended Java Solutions
Recommended Spring Solutions
Recommended Spring Batch Solutions

Previous Solution:

Get the mean across multiple Pandas DataFrames

PythonRNumpyPandas

Next Solution:

What does <f:facet> do and when should I use it?

JsfPrimefacesXhtmlFacet

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionmembersoundView Question on Stackoverflow
Solution 1 - JavaAure77View Answer on Stackoverflow
Solution 2 - JavaBrajView Answer on Stackoverflow
Solution 3 - JavaCleankodView Answer on Stackoverflow
Solution 4 - JavamembersoundView Answer on Stackoverflow
Solution 5 - JavaMKazView Answer on Stackoverflow
Solution 6 - JavaTracy XiaView Answer on Stackoverflow
Solution 7 - JavagmodeView Answer on Stackoverflow
Solution 8 - JavaAkhil KhandelwalView Answer on Stackoverflow
Solution 9 - Javabl4ckr0seView Answer on Stackoverflow
Solution 10 - Javarv.commView Answer on Stackoverflow