Committing Machine Specific Configuration Files

A common scenario when I develop is that the codebase will have several config files which require machine specific settings. These files will be checked into Git and other developers will always accidentally check them back in and break someone else's configuration.

A simple solution to this would be to just not check them in to Git, or even to additionally add a .gitignore entry for them. However, I find that it is much more elegant to have some sensible defaults in the file which the developer can modify to suit his needs.

Is there an elegant way to make Git play nicely with such files? I would like to be able to modify a machine-specific configuration file and then be able to run "git commit -a" without checking that file in.

Have your program read a pair of configuration files for its settings. First, it should read a config.defaults file that would be included in the repository. Then, it should read a config.local file that should be listed in .gitignore

With this arrangement, new settings appear in the defaults file and take effect as soon as it's updated. They will only vary on particular systems if they're overridden.

As a variation on this, you could have just a general config file that you ship in version control, and have it do something like include config.local to bring in the machine-specific values. This introduces a more general mechanism (versus policy) in you code, and consequently enables more complicated configurations (if that's desirable for your application). The popular extension from this, seen in many large-scale open-source software, is to include conf.d, which reads configuration from all the files in a directory.

Also see my answer to a similar question.

You can try git update-index --skip-worktree filename . This will tell git to pretend that local changes to filename don't exist, so git commit -a will ignore it. It has the added advantage of also resisting git reset --hard, so you won't accidentally lose your local changes. Also, automatic merges will fail gracefully if the file is changed upstream (unless the working directory copy matches the index copy, in which case it will be automatically updated). The downside is the command has to be run on all machines involved, and it's difficult to do this automatically. See also git update-index --assume-unchanged for a subtly different version of this idea. Details on both can be found with git help update-index .

Another approach is to maintain local changes to common configuration files in another private branch. I do this for some projects that require several local changes. This technique may not be applicable to all situations, but it works for me in some cases.

First I create a new branch based on the master branch (in this particular case I'm using git-svn so I need to commit from master but that's not terribly important here):

git checkout -b work master

Now modify the configuration file(s) as necessary and commit. I usually put something distinctive in the commit message like "NOCOMMIT" or "PRIVATE" (this will be useful later). At this point, you can work away on your private branch using your own config file.

When you want to push your work back upstream, cherry-pick each change from your work branch to the master. I have a script to help do this, which looks something like this:

#!/bin/sh

BRANCH=`git branch | grep ^\\* | cut -d' ' -f2`
if [ $BRANCH != "master" ]; then
  echo "$0: Current branch is not master"
  exit 1
fi

git log --pretty=oneline work...master | grep -v NOCOMMIT: | cut -d' ' -f1 | tac | xargs -l git cherry-pick

This first checks to make sure I'm on the master branch (sanity check). Then, it lists each commit in work, filters out the ones that mention the NOCOMMIT keyword, reverses the order, and finally cherry-picks each commit (now from the oldest first) into master.

Finally, after pushing the changes in master upstream, I switch back to work and rebase:

git checkout work
git rebase master

Git will reapply each of the commits in the work branch, effectively skipping over the one(s) that have already been applied in master through the cherry-picking. What you should be left with is only the NOCOMMIT local commits.

This technique makes the push process a bit more time-consuming, but it solved a problem for me so I thought I'd share.

One possibility is to have the actual files in your .gitignore, but check in default configurations with a different extension. A typical example for a Rails app would be the config/database.yml file. We would check in config/database.yml.sample, and each developer creates their own config/database.yml which is already .gitignored.

Check in a default configuration with a different extension (say .default), use a symlink to symlink the default to the correct location, add the correct location to .gitignore, and add everything else related to the configuration to .gitignore (so the only thing that gets checked in is config.default).

Additionally, write a quick install script that sets up the symlinks for your application-wide.

We used a similar approach at a previous company. The install script autodetected what environment you were running in (sandbox, development, QA, production), and would automatically do the right thing. If you had a config.sandbox file, and were running from the sandbox, it would link that (otherwise it would just link the .defaults file). Common procedure was to copy .defaults and change settings as necessary.

Writing the install script is easier than you might imagine, and gives you a lot of flexibility.

I agree with the best answer but also would like add something. I use an ANT script to strip & modify files from the GIT repo so I'm sure no production files get overwritten. There is a nice option in ANT to modify java-property files. This means putting your local test variables in a java-style property file and adding some code to process it, but it gives you the opportunity to automate building your site before you FTP it online. Typically you would put your production information in the site.default.properties file, and let ANT manage the settings. Your local settings would be in the site.local.properties.

    <?php
/**
 * This class will read one or two files with JAVA style property files. For instance site.local.properties & site.default.properties
 * This will enable developers to make config files for their personal development environment, while maintaining a config file for 
 * the production site. 
 * Hint: use ANT to build the site and use the ANT <propertyfile> command to change some parameters while building.
 * @author martin
 *
 */
class javaPropertyFileReader {
	
	private $_properties;
	private $_validFile;
	
	/**
	 * Constructor
	 * @return javaPropertyFileReader
	 */
	public function   __construct(){
		$this->_validFile = false;
		return $this;
	}//__construct
	
	/**
	 * Reads one or both Java style property files
	 * @param String $filenameDefaults
	 * @param String $filenameLocal
	 * @throws Exception
	 * @return javaPropertyFileReader
	 */
	public function readFile($filenameDefaults, $filenameLocal = ""){
		
		$this->handleFile($filenameDefaults);
		if ($filenameLocal != "") $this->handleFile($filenameLocal);
	}//readFile
	
	/**
	 * This private function will do all the work of reading the file and  setting up the properties
	 * @param String $filename
	 * @throws Exception
	 * @return javaPropertyFileReader
	 */
	private function handleFile($filename){
		
  	$file = @file_get_contents($filename);
  	
  	if ($file === false) {
  		 throw (New Exception("Cannot open property file: " . $filename, "01"));
  	}
  	else {
  		# indicate a valid file was opened
  		$this->_validFile = true;
  		
  		// if file is Windows style, remove the carriage returns
  		$file = str_replace("\r", "", $file);
  		
  		// split file into array : one line for each record
  		$lines = explode("\n", $file);
  		
  		// cycle lines from file
  		foreach ($lines as $line){
  			$line = trim($line);
  			
  			if (substr($line, 0,1) == "#" || $line == "") {
  				#skip comment line
  			}
  			else{
  				// create a property via an associative array
  				$parts   = explode("=", $line);
  				$varName = trim($parts[0]);
  				$value   = trim($parts[1]);

  				// assign property
  				$this->_properties[$varName] = $value;
  			}
  		}// for each line in a file
  	}
  	return $this;
	}//readFile
	
	/**
	 * This function will retrieve the value of a property from the property list.
	 * @param String $propertyName
	 * @throws Exception
	 * @return NULL or value of requested property
	 */
	function getProperty($propertyName){
		if (!$this->_validFile) throw (new Exception("No file opened", "03"));
		
		if (key_exists($propertyName, $this->_properties)){
			return $this->_properties[$propertyName];
		}
		else{
		  return NULL;
		}
	}//getProperty
	
	/**
	 * This function will retreive an array of properties beginning with a certain prefix.
	 * @param String $propertyPrefix
	 * @param Boolean $caseSensitive
	 * @throws Exception
	 * @return Array
	 */
	function getPropertyArray($propertyPrefix, $caseSensitive = true){
		if (!$this->_validFile) throw (new Exception("No file opened", "03"));
		
		$res = array();
		
		if (! $caseSensitive) $propertyPrefix= strtolower($propertyPrefix);
		
		foreach ($this->_properties as $key => $prop){
			$l = strlen($propertyPrefix);
			
			if (! $caseSensitive) $key = strtolower($key);
			
			if (substr($key, 0, $l ) == $propertyPrefix) $res[$key] = $prop;
		}//for each proprty
		
		return $res;
	}//getPropertyArray
	
	function createDefineFromProperty($propertyName){
		$propValue = $this->getProperty($propertyName);
		define($propertyName, $propValue);
	}//createDefineFromProperty
	
	
	/**
	 * This will create a number of 'constants' (DEFINE) from an array of properties that have a certain prefix.
	 * An exception is thrown if 
	 * @param  String $propertyPrefix
	 * @throws Exception
	 * @return Array The array of found properties is returned.
	 */
	function createDefinesFromProperties($propertyPrefix){
		// find properties
		$props = $this->getPropertyArray($propertyPrefix);

		// cycle all properties 
		foreach($props as $key => $prop){
			
			// check for a valid define name
			if (preg_match("'[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*'", $key)) {
				define($key, $prop);
			}	
			else{
				throw (new Exception("Invalid entry in property file: cannot create define for {" . $key . "}", "04"));
			}	
		}// for each property found
		
		return $props;
	}//createDefineFromProperty
	
}//class javaPropertyFileReader

then use it:

  $props = new javaPropertyFileReader();
  $props->readFile($_SERVER["DOCUMENT_ROOT"] . "/lib/site.default.properties",$_SERVER["DOCUMENT_ROOT"] . "/lib/site.local.properties");

  #create one DEFINE
  $props->createDefineFromProperty("picture-path");

  # create a number of DEFINEs for enabled modules
  $modules = $props->createDefinesFromProperties("mod_enabled_");

Your site.default.properties would look like:

release-date=x
environment=PROD
picture-path=/images/

SITE_VERSION_PRODUCTION=PROD
SITE_VERSION_TEST=TEST
SITE_VERSION_DEVELOP=DEV

# Available Modules
mod_enabled_x=false
mod_enabled_y=true
mod_enabled_z=true

and your site.local.properties would look like (notice the difference environment and enabled modules):

release-date=x
environment=TEST
picture-path=/images/

SITE_VERSION_PRODUCTION=PROD
SITE_VERSION_TEST=TEST
SITE_VERSION_DEVELOP=DEV

# Available Modules
mod_enabled_x=true
mod_enabled_y=true
mod_enabled_z=true

And your ANT instructions: ($d{deploy} being your deployment target directory)

<propertyfile
    file="${deploy}/lib/site.properties"
    comment="Site properties">
    <entry  key="environment" value="PROD"/>
    <entry  key="release-date" type="date" value="now" pattern="yyyyMMddHHmm"/>
</propertyfile>

Nowadays (2019) I use ENV vars for example in python/django, you can also add defaults to them. In the context of docker I can save the ENV vars in a docker-compose.yml file or an extra file which is ignored in version control.

# settings.py
import os
DEBUG = os.getenv('DJANGO_DEBUG') == 'True'
EMAIL_HOST = os.environ.get('DJANGO_EMAIL_HOST', 'localhost')

The simplest solution is to edit the file to defaults, commit it, then add it to your .gitignore. This way, developers will not accidentally commit it when doing git commit -a, but they can still commit it in the (presumably rare) case where you want to change your defaults with git add --force.

However, having a .default and .local config file is ultimately the best solution, since this allows anyone with a machine-specific configuration to change the defaults, without having to break their own configuration.

I do it like it's recommended here with default and local config files. To manage my local config files wich are in the projects .gitignore, I made a git repo ~/settings. There I manage all my local settings from all projects. You create, for example a folder project1 in ~/settings and put all the local config stuff for this project into it. After that you can symlink that files/folder to your project1.

With that approach you can track your local config files, and don't put them into to the normal source code repository.

Building on @Greg Hewgill's answer, you could add a specific commit with your local changes and tag it as localchange:

git checkout -b feature master
vim config.local
git add -A && git commit -m "local commit" && git tag localchange

Then proceed to add your feature's commits. After finishing the work, you can merge this branch back to master without the localchange commit by doing this:

git rebase --onto master localchange feature
git fetch . feature:master
git cherry-pick localchange
git tag localchange -f

These commands will:

1. Rebase your feature branch to master, ignoring the localchange commit.
2. Fast forward master without leaving feature branch
3. Add localchange commit back to the top of the feature branch so you can continue working on it. You can do this to any other branch you want to continue working on.
4. Reset localchange tag to this cherry-picked commit so we can use rebase --onto again in the same way.

This isn't meant to replace the accepted answer as the best general solution, but as a way of thinking out of the box about the problem. You basically avoid accidentally merging local changes to master by only rebasing from localchange to feature and fast forwarding master.