How to reference another property in java.util.Properties?

Can Java properties file reference other properties file?

## define a default directory for Input files  
dir.default=/home/data/in/

dir.proj1=${dir.default}p1
dir.proj2=${dir.default}p2
dir.proj3=${dir.default}p3

Is this possible?

Chris Mair's XProperties class may be a good starting point.

> You can substitute a constant anywhere in the property value, and even have more than one constant within a value, as in the following example:

CONST_1 = shoes and ships
CONST_2 = sealing wax
SomeValue = {CONST_1} and {CONST_2} 

> In this example, the "SomeValue" property evaluates to "shoes and ships and sealing wax."

Eproperties is the open source project which provides variable substitution along with a few other features - although substitution may arguably be the most useful. It is a subclass of java.util.Properties, and will can be used by any other class that may take configuration information as Properties.

Standard properties files are just key-value pairs. In the text format, Properties just separates key from value and does some simple things such as allowing escaped characters. You might be able to define entities in the verbose XML syntax.

If you want your own substitution syntax, then you can manipulate a returned value as you would with any other string. Alternatively, you could write your own version of Properties or do the substitution when generating the file.

The java.util.Properties class won't do this for you. It wouldn't be too difficult to subclass Properties, override the load() method and do the substitution yourself.

The Commons Config lib can also do this. http://commons.apache.org/configuration/userguide/overview.html#Using_Configuration

However, as pointed out already, have a look at the EProperties library; http://code.google.com/p/eproperties/

It supports a number of neat features (like substitution, nesting, lists) including inclusion, extends Java Properties and is a little more light weight than Commons Config (which also allows you to include properties using the include syntax).

Since eproperties is sort of not maintained and commons configuration has a dependency on logging (which ironically means you can't use it to configure logging) I use this code snippet which only requires commons-lang(3) to load interpolated properties:

@SuppressWarnings("serial")
public static Map<String,String> loadPropertiesMap(InputStream s) throws IOException {
	final Map<String, String> ordered = new LinkedHashMap<String, String>();
	//Hack to use properties class to parse but our map for preserved order
	Properties bp = new Properties() {
		@Override
		public synchronized Object put(Object key, Object value) {
			ordered.put((String)key, (String)value);
			return super.put(key, value);
		}
	};
	bp.load(s);
	final Map<String,String> resolved = new LinkedHashMap<String, String>(ordered.size());
	StrSubstitutor sub = new StrSubstitutor(new StrLookup<String>() {
		@Override
		public String lookup(String key) {
			String value = resolved.get(key);
			if (value == null)
				return System.getProperty(key);
			return value;
		}
	});
	for (String k : ordered.keySet()) {
		String value = sub.replace(ordered.get(k));
		resolved.put(k, value);
	}
	return resolved;
}

Input:

blah=${user.dir}
one=1
two=2
five=5
fifteen=${one}${five}
twoonefive=${two}${fifteen}
six=6

Output:

blah=/current/working/dir
one=1
two=2
five=5
fifteen=15
twoonefive=215
six=6

Obviously you can convert the Map<String,String> back to a Properties object if you need it. I resolve based on previously declared properties and system properties but you could obviously adjust that in the StrSubstitutor.lookup.

The configuration file consists of statements in the format key=value or key:value. Their are possible way where a key value can refer the another key value. The string between an opening "${" and closing "}" is interpreted as a key. The value of the substituted variable can be defined as a system property or in the configuration file itself.

Because Properties inherits from Hashtable, theput and putAll methods can be applied to a Properties object.

Map<String, String> map = new LinkedHashMap<String, String>();
map.put("key", "vlaue");
Properties props = new Properties();
props.putAll( map );

elaborating the post of @Adam Gent in-detailed. commons-text-1.1.jar

import org.apache.commons.text.StrLookup;
import org.apache.commons.text.StrSubstitutor;

public class Properties_With_ReferedKeys {
	public static void main(String[] args) {
		
		ClassLoader classLoader = Properties_With_ReferedKeys.class.getClassLoader();
		
		String propertiesFilename = "keys_ReferedKeys.properties";
		Properties props = getMappedProperties(classLoader, propertiesFilename);
		
		System.out.println( props.getProperty("jdk") );
		
	}
	
	
	public static Properties getMappedProperties( ClassLoader classLoader, String configFilename ) {
		Properties fileProperties = new Properties();
		
		try {
			InputStream resourceAsStream = classLoader.getResourceAsStream( configFilename );
			
			Map<String, String> loadPropertiesMap = loadPropertiesMap( resourceAsStream );
			Set<String> keySet = loadPropertiesMap.keySet();
			System.out.println("Provided 'Key':'Value' pairs are...");
			for (String key : keySet) {
				System.out.println( key + " : " + loadPropertiesMap.get(key) );
			}
			
			fileProperties.putAll( loadPropertiesMap );
		} catch ( IOException e ) {
			e.printStackTrace();
		}
		
		return fileProperties;
	}
	public static Map<String,String> loadPropertiesMap( InputStream inputStream ) throws IOException {
		final Map<String, String> unResolvedProps = new LinkedHashMap<String, String>();
		
		/*Reads a property list (key and element pairs) from the input byte stream. 
		 * The input stream is in a simple line-oriented format.
		 */
		@SuppressWarnings("serial")
		Properties props = new Properties() {
			@Override
			public synchronized Object put(Object key, Object value) {
				unResolvedProps.put( (String)key, (String)value );
				return super.put( key, value );
			}
		};
		props.load( inputStream );
		
		final Map<String,String> resolvedProps = new LinkedHashMap<String, String>( unResolvedProps.size() );
		
		// Substitutes variables within a string by values.
		StrSubstitutor sub = new StrSubstitutor( new StrLookup<String>() {
			@Override
			public String lookup( String key ) {
				
				/*The value of the key is first searched in the configuration file,
				 * and if not found there, it is then searched in the system properties.*/
				String value = resolvedProps.get( key );
				
				if (value == null)
					return System.getProperty( key );
				
				return value;
			}
		} );
		
		for ( String key : unResolvedProps.keySet() ) {
			
			/*Replaces all the occurrences of variables with their matching values from the resolver using the given 
			 * source string as a template. By using the default ${} the corresponding value replaces the ${variableName} sequence.*/
			String value = sub.replace( unResolvedProps.get( key ) );
			resolvedProps.put( key, value );
		}
		return resolvedProps;
	}
}

> Configuration File « If you want reference to be ignored and won't be replaced then you can use below format. > > $${${name}} must be used for output ${ Yash }. EX: jdk = ${jre-1.8}

File: keys_ReferedKeys.properties

# MySQL Key for each developer for their local machine
dbIP       = 127.0.0.1
dbName     = myApplicationDB
dbUser     = scott
dbPassword = tiger

# MySQL Properties 
# To replace fixed-keys with corresponding build environment values. like « predev,testing,preprd.
config.db.driverClassName : com.mysql.jdbc.Driver
config.db.url             : jdbc:mysql://${dbIP}:3306/${dbName}
config.db.username        : ${dbUser}
config.db.password        : ${dbPassword}

# SystemProperties
userDir      = ${user.dir}
os.name      = ${os.name}
java.version = ${java.version}
java.specification.version = ${java.specification.version}

# If you want reference to be ignored and won't be replaced.
# $${${name}} must be used for output ${ Yash }.  EX: jdk = ${jre-1.8}
jdk = $${jre-${java.specification.version}}

---

Java properties (key=value) format example log4j.properties

In this particular case (and in others too), you'd better resolve the duplication by defining different properties:

1. change: dir.proj1=dir.default /p1 into dir.proj1_extension=/p1
2. prepend: dir.default to dir.proj1_extension to get the full location of proj1 in your application code.

Do the same for the other projects.

None of the given solutions I really liked. EProperties is not maintained, and it is not available in Maven Central. Commons Config is too big for this. StrSubstitutor in commons-lang is deprecated.

My solution just relies on common-text:

public static Properties interpolateProperties(Properties rawProperties) {
    Properties newProperties = new Properties();
    interpolateProperties(rawProperties, newProperties);
    return newProperties;
}

public static void interpolateProperties(Properties rawProperties, Properties dstProperties) {
    StringSubstitutor sub = new StringSubstitutor((Map)rawProperties);
    for (Map.Entry<Object, Object> e : rawProperties.entrySet()) {
        dstProperties.put(e.getKey(), sub.replace(e.getValue()));
    }
}

ie:

Properties props = new Properties();
props.put("another_name", "lqbweb");
props.put("car", "this is a car from ${name}");
props.put("name", "${another_name}");
System.out.println(interpolateProperties(props));

prints out:

> {car=this is a car from ruben, name=ruben, another_name=ruben}

A 'pure' Java implementation:

    static final Pattern PATTERN = Pattern.compile("\\$\\{([^}]+)}");

    private static void macro(final Properties properties)
    {
        properties.replaceAll((k, v) -> PATTERN.matcher((String) v).replaceAll(mr -> properties.getProperty(mr.group(1), mr.group(0)).replace("$", "\\$")));
    }

Which can be incorporated into a trivial subclass of Properties, like this:

import java.io.IOException;
import java.io.InputStream;
import java.io.Reader;
import java.util.Properties;
import java.util.regex.Pattern;

public class MacroProperties extends Properties
{
    static final Pattern PATTERN = Pattern.compile("\\$\\{([^}]+)}", 0);

    @Override
    public synchronized void load(final Reader reader) throws IOException
    {
        super.load(reader);
        macro(this);
    }

    @Override
    public synchronized void load(final InputStream inStream) throws IOException
    {
        super.load(inStream);
        macro(this);
    }

    private static void macro(final Properties properties)
    {
        properties.replaceAll((k, v) -> PATTERN.matcher((String) v).replaceAll(mr -> properties.getProperty(mr.group(1), mr.group(0)).replace("$", "\\$")));
    }
}

---

How does it work?

PATTERN is a regexp that matches simple ${foo} patterns, and captures the text between the braces as a group.

Properties.replaceAll applies a function to replace each value with the result of the function.

Matcher.replaceAll applies a function to replace each match of PATTERN.

Our implementation of this function looks up match group 1 in the properties or defaults to the match (i.e. does not actually do a replacement).

Matcher.replaceAll also interprets the replacement string looking for group references so we also need to use String.replace to backslash escape $.

Below is a code snippet in Java for reading properties that reference other properties. Specifically, these are are reusable queries but can be other stuff as well.

LinkedHashMap<String, String> sqlsRaw = loadPropertiesFromFile();
LinkedHashMap<String, String> sqls = new LinkedHashMap<>();
StrSubstitutor substitutor = new StrSubstitutor(sqls);

for (Map.Entry<String, String> entry : sqlsRaw.entrySet()) {
	String sql = entry.getValue();
	try {
		sql = substitutor.replace(sql);
	} catch (Exception e) {
		throw new RuntimeException("Found an sql with a non replaced reference to another. Please validate that the required key was defined before this sql: " + entry.getValue(), e);
	}
	sqls.put(entry.getKey(), sql);
}

Example properties:

key1=value1
key21=value2 ${key1}

After running this, key21 will have the value value2 value1.

* Using apache's StrSubstitutor.

I like the idea of the solutions above, but I really wanted something to replace Properties. The class below builds on those ideas above. It still uses the Apache Commons-text StringSubstitutor, and looks for keys in the Properties class, the Java System defines or the System's Env. This class extends Properties and overrides getProperty(...) methods, so it is a drop in replacement. You can get the original key's value with the 'lookup()' method, but it will return a value from one of those 3 locations. If you want to determine if the key exists at all in the properties, then use the Map's underlying get().

The Apache commons dependency:

        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-text</artifactId>
            <version>1.3</version>
        </dependency>

Class source.

import java.util.Properties;
import org.apache.commons.text.StringSubstitutor;
import org.apache.commons.text.lookup.StringLookup;

/**
 * This extends Properties to provide macros substitution and includes getting properties from
 *     the Java System properties or the System's Environment.  This could be used to consolidate
 *     getting a system variable regardless if it is defined in the Java system or in the System's
 *     environment, without any other actual properties.
 *
 * The macro substitution is recursive so that given the following properties:
    <code>
      myProg=Program1
      outDir=./target
      inputs'./data/${defman }Templates
      outputs=${outDir}/${myProg}
      log=${outDir}/${myProg}/build_report.txt
      homeLog=${HOMEDRIVE}/${log}
    </code>
 * And assuming the environment variable HOMEDRIVE=C:
 * the getProperties("homeLog") would result in:  C:/./target/Program1/build_report.txt
 * 
 * Although based on the article below, this version substitutes during the getProperty() functions
 * instead of the loading functions explained in the article.
 * 
 * Based on this article: 
 *     https://stackoverflow.com/questions/872272/how-to-reference-another-property-in-java-util-properties
 *
 * @author Tim Gallagher
 * @license - You are free to use, alter etc. for any reason
 */
public class MacroProperties extends Properties implements StringLookup {

  // Substitutes variables within a string by values.
  public final StringSubstitutor macroSubstiitutor;

  public MacroProperties() {
    this.macroSubstiitutor = new StringSubstitutor(this);
  }

  public MacroProperties(Properties prprts) {
    super(prprts);
    this.macroSubstiitutor = new StringSubstitutor(this);
  }

  /**
   * The value of the key is first searched in the properties, and if not found there, it is then
   * searched in the system properties, and if still not found, then it is search in the 
   * system Env.
   *
   * @param key non-null string.
   * @return may be null.
   */
  @Override
  public String lookup(String key) {
    // get the Property first - this must look up in the parent class
    // or we'll get into an endless loop
    String value = super.getProperty(key);
    if (value == null) {      
      // if not found, get the Java system property which may have been defined on the 
      // Java command line with '-D...'
      value = System.getProperty(key);

      if (value == null) {
        // if not found, get the System's environment variable.
        value = System.getenv(key);
      }
    }

    return value;
  }

  @Override
  public String getProperty(String key, String defaultValue) {
    /*
       * Replaces all the occurrences of variables with their matching values from the resolver 
       * using the given source string as a template. By using the default ${} the corresponding
       * value replaces the ${variableName} sequence.
     */
    String value = lookup(key);
    if (value != null) {
      value = macroSubstiitutor.replace(value);
    } else {
      value = defaultValue;
    }
    return value;
  }

  @Override
  public String getProperty(String key) {
    return getProperty(key, null);
  }
}