Java Web Application Configuration Patterns

Are there any patterns or best practices that can be used to simplify changing configuration profiles for java web applications across multiple environments. e.g. JDBC URLs, SOAP end-points, etc.

As a bit of background to help clarify my question, I work with several large java web applications that during any given release cycle move through 6 different environments; development, integration, QA, performance and eventually get deployed to multiple production servers. In each environment, configuration needs to change. Right now, most configuration changes for each deployment are done manually which is both time consuming and open to errors.
Is there any way to take the manual intervention out of this process?

I'm surprised that no one cited the Jakarta Commons Configuration API (http://commons.apache.org/configuration/) to answer this question. It allows you to have a hierarchy of files (or other configuration sources like XML, JNDI, JDBC, etc.). That's what Jeremy Seghi was talking about and it gives you a good way to have both defaults and local overrides as well.

The best part is that it is a tested working solution so you don't have to go craft something yourself.

I tend to work more with .NET lately, so my Java is fairly rusty. I'm pretty sure this would work in any language with a little tweaking.

We use an extension of the .NET configuration system that allows us to use environment and/or application specific settings in conjunction with a more global configuration. The configuration system uses a Global setting to each machine identifies it as dev, beta, or production (the default). A set of files loaded in order, and the setting from the last file overrides any setting that was defined in a previously loaded file. Files are loaded in the following order:

1. Global settings
2. Application specific settings
3. Application specific environment overrides

All the files are in source control, and since the environment is defined on the machine the application is running on; since it won't access the "beta" configuration unless the machine configuration identifies it as "beta", we can promote all of the configuration files without fear of inadvertently pointing our production application to a dev database.

Here are some possible practices I've used or encountered. Combining these is usually needed in practice.

Substituting the variable values in conffiles when building

Here's an example of how this can be done with Apache Ant. Ant properties (${var.name}) can be controlled with the build configuration files:

<filterset id="variables.to.replace">
    <filter token="APPNAME" value="${app.name}"/>
    <filter token="WEBAPP-PATH" value="${webapp.path}"/>
    <filter token="ENCRYPT-ALGORITHM" value="${encrypt.algorithm}"/>
    <filter token="ERROR-MAILTO" value="${error.mailTo}"/>
    <!--...-->
</filterset>

<!-- Then, when building & copying the conf, replace the variables: -->
<copy todir="${properties.target.dir}">
    <!-- env specific conf files -->
    <fileset dir="${basedir}/env/${run.env}/webapp/WEB-INF/classes" />
    <filterset refid="variables.to.replace"/>
</copy>

The good thing is that you get a fine control over the different configurations at build time. What is bad is that the system tends to grow very complex and hard to maintain if you use this method extensively for a large number of different configurations. Also, having to build the conffiles, too, means slower development cycles.

Substituting the variables from conf inside war at webapp startup

This is what I usually do when using Spring Framework, even if there is just one possble configuration, getting the benefits of the separation of concerns. With Spring, you can have the conf values replaced with PlaceholderPropertyConfigurer inside Spring context at webapp startup. In this case, you have to anyway pick the right configuration, which can be configured for example on build time.

Compared to the build time replacing, it's easier to temporarily manipulate the values in an uncompressed webapp, if needed. Of course, the webapp needs to be rebooted if you change anything, and the manual changes won't persist across webapp redeployments. Spring is also limited to the Spring context, so this doesnt' work e.g. in web.xml (but having variables in web.xml should probably be avoided anyway because of its limitations).

Reading the local conf from a pre-defined file

This approach is probably the easiest one to set up: just invent a configuration file path, e.g. $HOME/mywebapp/conf.properties and make your webapp somehow read it at startup.

The good thing here is that you don't have to care about the conf when building/deploying the webapp. Anyway, you should have some sensible conf defaults that can then be overridden by the local conf.

Having the conf in a database

This is the most flexible solution for overriding conf parameters, but can also get complicated in some cases. Having the conf in a table with name and value columns should work for the most cases.

Of course, you can't configure the JDBC connection urls in a database table, but this is a good solution for simple textual/numerical conf that affects the webapp's operation after the db connection has been set up. To avoid a performance penalty, make sure you cache the conf somehow if it will be frequently accessed.

Extra practices

As pointed out by kgiannakakis, it also helps to set up a configuration diagnosis page of some kind for your app.

This is heavily going to depend on what options the web application servers give you. We have multiple environments for JBoss with different JDBC URLs, the JNDI name remains the same across all servers, just the configuration on the local instance changes, so nothing goes wrong from build to build.

I guess the short answer is that the best practice is to externalize the configurations and keep a good file in place with the correct settings for each server, and have the web app read that configuration. The exact nature of the externalization and reading is going to depend on the specific configuration and application server.

EDIT: These configurations do not exist as part of the war (ear in our case) that way they are not overwritten.

At first have all the configuration settings that frequently change in one place. It is really difficult, if you need to set up JNDI, edit database values and modify property files, all at the same time, in order to complete the configuration. Prefer the medium that is easiest to edit and also easier to verify that everything is set up properly. I would say that property files is the best solution. You can easily edit them and you only need a quick look through them to see that everything is alright. If you opt for property files, carefully select a standard location for them and assign an environmental variable for the path.

It also helps if you have a simple test that verifies that everything is set up properly. For example you can have a test page that displays the configuration parameters and performs some basic tests, like trying to connect to database or remote servers.

What we do works pretty well.

On startup, our programs read a configuration file in a hardcoded path. Let's say it's:

/config/fun_prog/config.xml

Each program has a different hard coded path (FunProgram is in fun_prog, Super Server is in sup_serv, whatever), so we don't have to worry about them walking over each other.

The XML files are read by a little configuration library we created. The XML file contains the DB connection information, usually mail server configuration data, email addresses to send notifications to, whether it should operate in test mode, URLs of external services, etc.

So when we need to make changes, we copy the config file, edit what we want, and restart the program. Since we have a standard server setup, any program can be deployed on any server by just copying these files around (and the neccessary httpd.conf tinkering).

It's not fancy, but it works very well. It's extremely simple to understand, add new configuration options, backup, and edit. Works on all platforms (unix is obvious, Windows translates paths starting with / into c:\ so it works without edits there too).

Our workstations basically run the same software as the server, just with a few changes in that config file.

The good example what you want is used in Seam or Grails (borrowed from Rails). There are profiles, by default three: prod, dev, test, but you can define more if you want.

In Seam project build is done by Ant files. Eeach file that contents can vary is defined for every profile e.g. datasource, sql scripts or properties files.

import-dev.sql import-prod.sql import-test.sql

When ant file is run with choosen profile, appropriate file is taken and profile name is truncated from that file name.

Below is code snippet that you can place in your targets

<copy tofile="${war.dir}/WEB-INF/classes/import.sql" 
      file="${basedir}/resources/import-${profile}.sql"/>

JDBC url, driver names can be externalized to properties files (of course with profile names as suffixes)

<filterset id="persistence">
     <filter token="transactionManagerLookupClass" value="${transactionManagerLookupClass}"/>

<copy tofile="${war.dir}/WEB-INF/classes/META-INF/persistence.xml" 
     file="${basedir}/resources/META-INF/persistence-${profile}.xml">
     <filterset refid="persistence"/>
</copy>

or values of properies you can pass to ant build call from command line. This is short example what is in Seam done.

Another option is to used Maven. In maven way it done by properties and by profiles, but you can use also separate modules to split configuration and create other modules with main functionality. Typical use case examples of maven properties and profiles are run configuration for multiple databases, deployment servers etc. It's even harder when you want to create configuration for different vendors, but for Maven that isn't a problem :)

Great example of using maven profiles is this post form Carlos Sanchez blog.

To sum up I strongly recomend to look up Ant/Seam an Maven parametrization (profiles). Those solutions have another advantage: ant or maven script can be run in CI server (like Hudson) and let run/test simultaneously all your profiles.

You can use the Component Configuration Pattern in your language of choice

It is described in the POSA books (I think that in the 4th volume)

(in java you can use the commons-configuration component ).

There are a few possible ways to approach this:

• use property files like you do, but add a "meta properties" file that is used to select the property file used by defining a map between an environment value (for instance localhost hostname) onto the property file name to load.

• put your properties into a database and define the database connection to the property tables in your application server as resource that is picked up by your web-app.

• don't put the property files in your .war or .ear, but create a properties-deployhost.jar archives containing the property files per target host. bind the appropriate .jar file to the deployed web-app by adding it to the class path (for instance via shared libraries in the application server configuration per web-app.)

Only the first of these does not need extra manual steps when deploying at the expense of having to update your config source and building new deploy files when your target systems are renamed.

I'm sure lots of variantions on these and your approach are possible, what is the best choice depends on your situation.

Please take a look at this URL: http://issues.apache.org/jira/browse/CONFIGURATION-394

The Configuration framework which we're looking for it is something on top of Apache Commons Configuration and must support Concurrency Issues, JMX issues and most of stores(e.g .properties file, .xml files or PreferencesAPI).

What weblogic team provides on 'Administration Console' is intersting which through it you can have transactional(atomic) updates on configurations so that are registered listeners be notified.

The Apache guys insist that this project is out of scopes of Commons Configuration, maybe!

I've attached a simple configuration framework, take look please.