Distributed Version Control Systems and the Enterprise - a Good mix?

I can see why distributed source control systems (DVCS - like Mercurial) make sense for open source projects.

But do they make sense for an enterprise? (over a centralized Source Control System such as TFS)

What features of a DVCS make it better or worse suited for an enterprise with many developers? (over a centralized system)

I have just introduced a DVCS (Git in this case) in a large banking company, where Perforce, SVN or ClearCase was the centralized VCS of choices:
I already knew of the challenges (see my previous answer "Can we finally move to DVCS in Corporate Software? Is SVN still a 'must have' for development?")

I have been challenged on three fronts:

• centralization: while the decentralized model has its merits (and allows for private commits or working without the network while having access to the full history), there still needs to be a clear set of centralized repos, acting as the main reference for all developers.

• authentication: a DVCS allows you to "sign-off" (commit) your code as... pretty much anyone (author "foo", email "[email protected]").
  You can do a git config user.name foo, or git config user.name whateverNameIFeelToHave, and have all your commits with bogus names in it.
  That doesn't mix well with the unique centralized "Active Directory" user referential used by big enterprises.

• authorization: by default, you can clone, push from or pull to any repository, and modify any branch, or any directory.
  For sensitive projects, that can be a blocking issue (the banking world is usually very protective of some pricing or quants algorithms, which require strict read/write access for a very limited number of people)

The answer (for a Git setup) was:

• centralization: a unique server has been set up for any repository having to be accessible by all users.
  Backup has been taking care of (incremental every day, full every week).
  DRP (Disaster Recovery Plan) has been implemented, with a second server on another site, and with real-time data replication through SRDF.
  This setup in itself is independent of the type of referential or tool you need (DVCS, or Nexus repo, or main Hudson scheduler, or...): any tool which can be critical for a release into production needs to be installed on servers with backup and DR.

.

• authentication: only two protocols allow users to access the main repos:
  • ssh based, with public/private key:
    • useful for users external to the organization (like off-shore development),
    • and useful for generic accounts that Active Directory manager don't want to create (because it would be an "anonymous" account): a real person has to be responsible for that generic account, and that would be the one owning the private key
  • https-based, with an Apache authenticating the users through a LDAP setting: that way, an actual login must be provided for any git operation on those repos.
    Git offers it with its smart http protocol, allowing not just pull (read) through http, but also push (write) through http.

The authentication part is also reinforced at the Git level by a post-receive hook which makes sure that at least one of the commits you are pushing to a repo has a "committer name" equals to the user name detected through shh or http protocol.
In other words, you need to set up your git config user.name correctly, or any push you want to make to a central repo will be rejected.

.

• authorization: both previous settings (ssh or https) are wired to call the same set of perl script, named gitolite, with as parameters:
  • the actual username detected by those two protocols
  • the git command (clone, push or pull) that user wants to do

The gitolite perl script will parse a simple text file where the authorizations (read/write access for a all repository, or for branches within a given repository, or even for directories within a repository) have been set.
If the access level required by the git command doesn't match the ACL defined in that file, the command is rejected.

---

The above describes what I needed to implement for a Git setting, but more importantly, it lists the main issues that need to be addressed for a DVCS setting to make sense in a big corporation with a unique user base.

Then, and only then, a DVCS (Git, Mercurial, ...) can add values because of:

• data exchange between multiple sites: while those users are all authenticated through the same Active Directory, they can be located across the world (the companies I have worked for have developments usually between teams across two or three countries). A DVCS is naturally made for exchanging efficiently data between those distributed teams.

• replication across environments: a setting taking care of authentication/authorization allows for cloning those repositories on other dedicated servers (for integration testing, UAT testing, pre-production, and pre-deployment purposes)

• process automation: the ease with which you can clone a repo can also be used locally on one user's workstation, for unit-testing purposes with the "guarded commits" techniques and other clever uses: see "What is the cleverest use of source repository that you have ever seen?".
  In short, you can push to a second local repo in charge of:

  • various tasks (unit test or static analysis of the code)
  • pushing back to the main repo if those tasks are successful
  • while you are still working in the first repo without having to wait for the result of those tasks.

.

• killer features: Any DVCS comes with those, the main one being merging (ever tried to do a complex merge workflow with SVN? Or sloooowly merge 6000 files with ClearCase?).
  That alone (merging) means you can really take advantage of branching, while being able at all time to merge back your code to another "main" line of development because you would do so:

  • first locally within your own repo, without disturbing anybody
  • then on the remote server, pushing the result of that merge on the central repo.

Absolutely a distributed source model can make sense in an enterprise, but it does depend on the structure of your teams.

Distributed source control gives you the flexibility to create your own workflows.

Imagine, if you will, a larger team, within which are smaller teams working on separate feature branches.

• These teams can all have their own central repositories, with their own build automation/checkin control mechanisms.
• They can work anywhere, and backup their local work whenever they so desire.
• They can then choose what checkins they'd like to share between groups.
• They can have a single individual integrator, working on their own machine, performing merging, without impacting others.

These are things you could achieve with a traditional centralised server, but as @Brook points out, the centralised model has to scale, whereas the distributed model is already sharded, so no (or at least, less) need to vertically scale any servers.

To add to the other comments, I would observe that there's no reason you can't have a Corporate Central Repository. Technically it's just another repository, but it's the one you ship production from. I've been using one form or another of VCS for over 30 years and I can say that switching to Mercurial was like a city boy breathing clean country air for the first time.

DSCS have a better story (generally) than centralized systems for offline or slow networks. They tend to be faster, which is really noticable for developers (using TDD) who do lots of check-ins.

Centralized systems are somewhat easier to grasp initially and might be a better choice for less experienced developers. DVCSes allow you to create lots of mini-branches and isolate new features while still doing red-gree-refactor checkin on green style of coding. Again this is very powerful but only attractive to fairly savvy development teams.

Having a single central repository for support for exclusive locks makes sense if you deal with files that are not mergable like digital assets and non-text documents (PDFs and Word etc) as it prevents you getting yourself into a mess and manually merging.

I don't think the number of developers or codebase size plays into it that much, both systems have been show to support large source trees and numbers of committers. However for large code bases and projects DVCS gives a lot of flexibility in quickly creating decentralized remote branches. You can do this with centralized systems but you need to be more deliberate about it which is both good and bad.

In short there are some technical aspects to consider but you should also think about the maturity of your team and their current process around SCCS.

At least with tfs 2013 you do have the ability to work disconnected with local workspaces. Distributed vs centralized is defined by the business and depends on the needs and requirements of the projects under development.

For enterprise projects the ability to connect workflow and documents to code changes can be critical in connecting business requirements and higher order elements to specific code changes that address a specific change, bug or feature addition.

This connection between workflow and code repository separates TFS from code repository only solutions. For some places where a higher order of project auditing is required only a product like TFS would satisfy more of the project auditing requirements.

An overview of the application lifecycle management process can be found here.

http://msdn.microsoft.com/en-us/library/vstudio/fda2bad5(v=vs.110).aspx

The biggest issue we face with Git in enterprise setting is the lack of path based read-access control. It is inherent in Git's architecture (and I would assume most DVCSs) that if you got read access to repository you get the whole thing. But sometimes a project would require a sparse checkout (i.e. you want to version control sensitive data close to the source, or you want to give third party a selective view of part of the project).

Out of the box, Git provides no permissions - you've got hooks to write your own.

Most of the popular repo managers GithubEnterprise, Gitlab, Bitbucket provide branch based write restrictions. Gitolite allows to be finer grained, providing path (and more) based write restrictions.

The only repo manager I've heard of supporting read access is the Perforce Helix, which reimplements git protocol on top of perforce backend, but I have no hands-on experience with it. It is promising, but I would be concerned how compatible it is with "plain" git.

To me the biggest thing they offer is Speed. They're orders of magnitude faster for the most common operations than centralized source control.

Working disconnected is also a huge plus.

Our team used TFS for about 3 years before switching to Mercurial. HG's branch/merge support is so much better than TFS. This is because a DVCS relies on painless merging.

Better synchronization across remote / disconnected locations.