Git symbolic links in Windows

Our developers use a mix of Windows and Unix-based OSes. Therefore, symbolic links created on Unix machines become a problem for Windows developers. In Windows (MSysGit), the symbolic link is converted to a text file with a path to the file it points to. Instead, I'd like to convert the symbolic link into an actual Windows symbolic link.

The (updated) solution I have to this is:

• Write a post-checkout script that will recursively look for "symbolic link" text files.
• Replace them with a Windows symbolic link (using mklink) with the same name and extension as dummy "symbolic link"
• Ignore these Windows symbolic links by adding an entry into file .git/info/exclude

I have not implemented this, but I believe this is a solid approach to this problem.

1. What, if any, downsides do you see to this approach?
2. Is this post-checkout script even implementable? I.e., can I recursively find out the dummy "symlink" files Git creates?

I was asking this exact same question a while back (not here, just in general), and ended up coming up with a very similar solution to OP's proposition. I'll post the solution I ended up using.

But first I'll provide direct answers to OP's 3 questions:

>Q: "What, if any, downsides do you see to this approach?" > >A: There are indeed a few downsides to the proposed solution, mainly regarding an increased potential for repository pollution, or accidentally adding duplicate files while they're in their "Windows symlink" states. (More on this under "limitations" below.)

>Q: "Is this post-checkout script even implementable? i.e. can I recursively find out the dummy "symlink" files git creates?" > >A: Yes, a post-checkout script is implementable! Maybe not as a literal post-git checkout step, but the solution below has met my needs well enough that a literal post-checkout script wasn't necessary.

>Q: "Has anybody already worked on such a script?" > >A: Yes!

The Solution:

Our developers are in much the same situation as OP's: a mixture of Windows and Unix-like hosts, repositories and submodules with many git symlinks, and no native support (yet) in the release version of MsysGit for intelligently handling these symlinks on Windows hosts.

Thanks to Josh Lee for pointing out the fact that git commits symlinks with special filemode 120000. With this information it's possible to add a few git aliases that allow for the creation and manipulation of git symlinks on Windows hosts.

1. Creating git symlinks on Windows

    git config --global alias.add-symlink '!'"$(cat <<'ETX'
    __git_add_symlink() {
      if [ $# -ne 2 ] || [ "$1" = "-h" ]; then
        printf '%b\n' \
            'usage: git add-symlink <source_file_or_dir> <target_symlink>\n' \
            'Create a symlink in a git repository on a Windows host.\n' \
            'Note: source MUST be a path relative to the location of target'
        [ "$1" = "-h" ] && return 0 || return 2
      fi
   
      source_file_or_dir=${1#./}
      source_file_or_dir=${source_file_or_dir%/}
   
      target_symlink=${2#./}
      target_symlink=${target_symlink%/}
      target_symlink="${GIT_PREFIX}${target_symlink}"
      target_symlink=${target_symlink%/.}
      : "${target_symlink:=.}"
   
      if [ -d "$target_symlink" ]; then
        target_symlink="${target_symlink%/}/${source_file_or_dir##*/}"
      fi
   
      case "$target_symlink" in
        (*/*) target_dir=${target_symlink%/*} ;;
        (*) target_dir=$GIT_PREFIX ;;
      esac
   
      target_dir=$(cd "$target_dir" && pwd)
   
      if [ ! -e "${target_dir}/${source_file_or_dir}" ]; then
        printf 'error: git-add-symlink: %s: No such file or directory\n' \
            "${target_dir}/${source_file_or_dir}" >&2
        printf '(Source MUST be a path relative to the location of target!)\n' >&2
        return 2
      fi
   
      git update-index --add --cacheinfo 120000 \
          "$(printf '%s' "$source_file_or_dir" | git hash-object -w --stdin)" \
          "${target_symlink}" \
        && git checkout -- "$target_symlink" \
        && printf '%s -> %s\n' "${target_symlink#$GIT_PREFIX}" "$source_file_or_dir" \
        || return $?
    }
    __git_add_symlink
    ETX
    )"
   

   Usage: git add-symlink <source_file_or_dir> <target_symlink>, where the argument corresponding to the source file or directory must take the form of a path relative to the target symlink. You can use this alias the same way you would normally use ln.

   E.g., the repository tree:

    dir/
    dir/foo/
    dir/foo/bar/
    dir/foo/bar/baz      (file containing "I am baz")
    dir/foo/bar/lnk_file (symlink to ../../../file)
    file                 (file containing "I am file")
    lnk_bar              (symlink to dir/foo/bar/)
   

   Can be created on Windows as follows:

    git init
    mkdir -p dir/foo/bar/
    echo "I am baz" > dir/foo/bar/baz
    echo "I am file" > file
    git add -A
    git commit -m "Add files"
    git add-symlink ../../../file dir/foo/bar/lnk_file
    git add-symlink dir/foo/bar/ lnk_bar
    git commit -m "Add symlinks"
   

2. Replacing git symlinks with NTFS hardlinks+junctions

    git config --global alias.rm-symlinks '!'"$(cat <<'ETX'
    __git_rm_symlinks() {
      case "$1" in (-h)
        printf 'usage: git rm-symlinks [symlink] [symlink] [...]\n'
        return 0
      esac
      ppid=$$
      case $# in
        (0) git ls-files -s | grep -E '^120000' | cut -f2 ;;
        (*) printf '%s\n' "$@" ;;
      esac | while IFS= read -r symlink; do
        case "$symlink" in
          (*/*) symdir=${symlink%/*} ;;
          (*) symdir=. ;;
        esac
   
        git checkout -- "$symlink"
        src="${symdir}/$(cat "$symlink")"
   
        posix_to_dos_sed='s_^/\([A-Za-z]\)_\1:_;s_/_\\\\_g'
        doslnk=$(printf '%s\n' "$symlink" | sed "$posix_to_dos_sed")
        dossrc=$(printf '%s\n' "$src" | sed "$posix_to_dos_sed")
   
        if [ -f "$src" ]; then
          rm -f "$symlink"
          cmd //C mklink //H "$doslnk" "$dossrc"
        elif [ -d "$src" ]; then
          rm -f "$symlink"
          cmd //C mklink //J "$doslnk" "$dossrc"
        else
          printf 'error: git-rm-symlink: Not a valid source\n' >&2
          printf '%s =/=> %s  (%s =/=> %s)...\n' \
              "$symlink" "$src" "$doslnk" "$dossrc" >&2
          false
        fi || printf 'ESC[%d]: %d\n' "$ppid" "$?"
   
        git update-index --assume-unchanged "$symlink"
      done | awk '
        BEGIN { status_code = 0 }
        /^ESC\['"$ppid"'\]: / { status_code = $2 ; next }
        { print }
        END { exit status_code }
      '
    }
    __git_rm_symlinks
    ETX
    )"
   
    git config --global alias.rm-symlink '!git rm-symlinks'  # for back-compat.
   

   Usage:

    git rm-symlinks [symlink] [symlink] [...]
   

   This alias can remove git symlinks one-by-one or all-at-once in one fell swoop. Symlinks will be replaced with NTFS hardlinks (in the case of files) or NTFS junctions (in the case of directories). The benefit of using hardlinks+junctions over "true" NTFS symlinks is that elevated UAC permissions are not required in order for them to be created.

   To remove symlinks from submodules, just use git's built-in support for iterating over them:

    git submodule foreach --recursive git rm-symlinks
   

   But, for every drastic action like this, a reversal is nice to have...

3. Restoring git symlinks on Windows

    git config --global alias.checkout-symlinks '!'"$(cat <<'ETX'
    __git_checkout_symlinks() {
      case "$1" in (-h)
        printf 'usage: git checkout-symlinks [symlink] [symlink] [...]\n'
        return 0
      esac
      case $# in
        (0) git ls-files -s | grep -E '^120000' | cut -f2 ;;
        (*) printf '%s\n' "$@" ;;
      esac | while IFS= read -r symlink; do
        git update-index --no-assume-unchanged "$symlink"
        rmdir "$symlink" >/dev/null 2>&1
        git checkout -- "$symlink"
        printf 'Restored git symlink: %s -> %s\n' "$symlink" "$(cat "$symlink")"
      done
    }
    __git_checkout_symlinks
    ETX
    )"
   
    git config --global alias.co-symlinks '!git checkout-symlinks'
   

   Usage: git checkout-symlinks [symlink] [symlink] [...], which undoes git rm-symlinks, effectively restoring the repository to its natural state (except for your changes, which should stay intact).

   And for submodules:

    git submodule foreach --recursive git checkout-symlinks
   

4. Limitations:

   • Directories/files/symlinks with spaces in their paths should work. But tabs or newlines? YMMV… (By this I mean: don’t do that, because it will not work.)

   • If yourself or others forget to git checkout-symlinks before doing something with potentially wide-sweeping consequences like git add -A, the local repository could end up in a polluted state.

     Using our "example repo" from before:

     echo "I am nuthafile" > dir/foo/bar/nuthafile
     echo "Updating file" >> file
     git add -A
     git status
     # On branch master
     # Changes to be committed:
     #   (use "git reset HEAD <file>..." to unstage)
     #
     #       new file:   dir/foo/bar/nuthafile
     #       modified:   file
     #       deleted:    lnk_bar           # POLLUTION
     #       new file:   lnk_bar/baz       # POLLUTION
     #       new file:   lnk_bar/lnk_file  # POLLUTION
     #       new file:   lnk_bar/nuthafile # POLLUTION
     #
     

     Whoops...

     For this reason, it's nice to include these aliases as steps to perform for Windows users before-and-after building a project, rather than after checkout or before pushing. But each situation is different. These aliases have been useful enough for me that a true post-checkout solution hasn't been necessary.

Hope that helps!

References:

http://git-scm.com/book/en/Git-Internals-Git-Objects

http://technet.microsoft.com/en-us/library/cc753194

Last Update: 2019-03-13

• POSIX compliance (well, except for those mklink calls, of course) — no more Bashisms!
• Directories and files with spaces in them are supported.
• Zero and non-zero exit status codes (for communicating success/failure of the requested command, respectively) are now properly preserved/returned.
• The add-symlink alias now works more like ln(1) and can be used from any directory in the repository, not just the repository’s root directory.
• The rm-symlink alias (singular) has been superseded by the rm-symlinks alias (plural), which now accepts multiple arguments (or no arguments at all, which finds all of the symlinks throughout the repository, as before) for selectively transforming git symlinks into NTFS hardlinks+junctions.
• The checkout-symlinks alias has also been updated to accept multiple arguments (or none at all, == everything) for selective reversal of the aforementioned transformations.

Final Note: While I did test loading and running these aliases using Bash 3.2 (and even 3.1) for those who may still be stuck on such ancient versions for any number of reasons, be aware that versions as old as these are notorious for their parser bugs. If you experience issues while trying to install any of these aliases, the first thing you should look into is upgrading your shell (for Bash, check the version with CTRL+X, CTRL+V). Alternatively, if you’re trying to install them by pasting them into your terminal emulator, you may have more luck pasting them into a file and sourcing it instead, e.g. as

. ./git-win-symlinks.sh

Good luck!

You can find the symlinks by looking for files that have a mode of 120000, possibly with this command:

git ls-files -s | awk '/120000/{print $4}'

Once you replace the links, I would recommend marking them as unchanged with git update-index --assume-unchanged, rather than listing them in .git/info/exclude.

The most recent version of git scm (testet 2.11.1) allows to enable symbolic links. But you have to clone the repository with the symlinks again git clone -c core.symlinks=true <URL>. You need to run this command with administrator rights. It is also possible to create symlinks on Windows with mklink. Check out the wiki.

2020+ TL;DR Answer

1. Enable "Developer Mode" in Windows 10/11 -- gives mklink permissions
2. Ensure symlinks are enabled in git with (at least) one of
   • System setting: check the checkbox when installing msysgit
   • Global setting: git config --global core.symlinks true
   • Local setting: git config core.symlinks true

---

Be careful, support for symlinks in git on Windows is relatively new. There are some bugs that still affect some git clients. Notably, symlinks with relative (..) paths are mangled in some programs because of a (fixed) regression in libgit2. For instance, GitKraken is affected by this because they are waiting on nodegit to update libgit2 from v0.x (regression) to v1.x (fixed).

---

Recreate missing/broken symlinks

Various levels of success have been reported across multiple git clients with one of these (increasingly forceful and "dangerous") options

• Checkout: git checkout -- path/to/symlink
• Restore (since git v2.23.0): git restore -- path/to/symlink
• Switch branches (away and back)
• Hard Reset: git reset --hard
• Delete local repository and clone again

Troubleshooting

git config --show-scope --show-origin core.symlinks will show you the level (aka "scope") the setting is set, where the configuration file (aka "origin") that is persisting it is, and the current value of the setting. Most likely a "local" configuration is overriding the "global" or "system" setting. git config --unset core.symlinks will clear a "local" setting allowing a higher level setting to take effect.

so as things have changed with GIT since alot of these answers were posted here is the correct instructions to get symlinks working correctly in windows as of

AUGUST 2018

---

1. Make sure git is installed with symlink support

[![During the install of git on windows][1]][1]

2. Tell Bash to create hardlinks instead of symlinks

EDIT -- (git folder)/etc/bash.bashrc

ADD TO BOTTOM - MSYS=winsymlinks:nativestrict

3. Set git config to use symlinks

git config core.symlinks true

or

git clone -c core.symlinks=true <URL>

NOTE: I have tried adding this to the global git config and at the moment it is not working for me so I recommend adding this to each repo...

4. pull the repo

NOTE: Unless you have enabled developer mode in the latest version of Windows 10, you need to run bash as administrator to create symlinks

5. Reset all Symlinks (optional) If you have an existing repo, or are using submodules you may find that the symlinks are not being created correctly so to refresh all the symlinks in the repo you can run these commands.

find -type l -delete
git reset --hard

NOTE: this will reset any changes since last commit so make sure you have committed first [1]: https://i.stack.imgur.com/Am9L1.png

It ought to be implemented in msysgit, but there are two downsides:

• Symbolic links are only available in Windows Vista and later (should not be an issue in 2011, and yet it is...), since older versions only support directory junctions.
• (the big one) Microsoft considers symbolic links a security risk and so only administrators can create them by default. You'll need to elevate privileges of the git process or use fstool to change this behavior on every machine you work on.

I did a quick search and there is work being actively done on this, see issue 224.

Short answer: They are now supported nicely, if you can enable developer mode.

From https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/

> Now in Windows 10 Creators Update, a user (with admin rights) can > first enable Developer Mode, and then any user on the machine can run > the mklink command without elevating a command-line console. > > What drove this change? The availability and use of symlinks is a big > deal to modern developers: > > Many popular development tools like git and package managers like npm > recognize and persist symlinks when creating repos or packages, > respectively. When those repos or packages are then restored > elsewhere, the symlinks are also restored, ensuring disk space (and > the user’s time) isn’t wasted.

Easy to overlook with all the other announcements of the "Creator's update", but if you enable Developer Mode, you can create symlinks without elevated privileges. You might have to re-install git and make sure symlink support is enabled, as it's not by default.

I would suggest you don't use symlinks within the repo'. Store the actual content inside the repo' and then place symlinks out side the repo' that point to the content.

So lets say you are using a repo' to compare hosting your site on *nix with hosting on win. Store the content in your repo', lets say /httpRepoContent and c:\httpRepoContent with this being the folder that is synced via GIT, SVN etc.

Then, replace the content folder of you web server (/var/www and c:\program files\web server\www {names don't really matter, edit if you must}) with a symbolic link to the content in your repo'. The web servers will see the content as actually in the 'right' place, but you get to use your source control.

However, if you need to use symlinks with in the repo', you will need to look into something like some sort of pre/post commit scripts. I know you can use them to do things, such as parse code files through a formatter for example, so it should be possible to convert the symlinks between platforms.

if any one knows a good place to learn how to do these scripts for the common source controls, SVN GIT MG, then please do add a comment.

For those using CygWin on Vista, Win7, or above, the native git command can create "proper" symlinks that are recognized by Windows apps such as Android Studio. You just need to set the CYGWIN environment variable to include winsymlinks:native or winsymlinks:nativestrict as such:

export CYGWIN="$CYGWIN winsymlinks:native"

The downside to this (and a significant one at that) is that the CygWin shell has to be "Run as Administrator" in order for it to have the OS permissions required to create those kind of symlinks. Once they're created, though, no special permissions are required to use them. As long they aren't changed in the repository by another developer, git thereafter runs fine with normal user permissions.

Personally, I use this only for symlinks that are navigated by Windows apps (i.e. non-CygWin) because of this added difficulty.

For more information on this option, see this SO question: https://stackoverflow.com/questions/3648819/how-to-make-symbolic-link-with-cygwin-in-windows-7

I just tried with Git 2.30.0 (released 2020-12-28).

This is NOT a full answer but a few useful tidbits nonetheless. (Feel free to cannibalize for your own answer.)

Git Wiki Entry

There's a documentation link when installing Git for Windows

This link takes you here: https://github.com/git-for-windows/git/wiki/Symbolic-Links -- And this is quite a longish discussion.

FYI: There are at least three "kinds of links"

And just to highlight an important aspect of this wiki entry: I didn't know this, but there are several ways all of which are "kind of" symbolic links on the surface, but on a technical level are VERY DIFFERENT:

• git bash's "ln -s"
  Which just COPIES things. Oh, boy. That was unexpected to me.
  (FYI: Plain Cygwin does NOT do this. Mobaxterm does NOT do this. Instead they both create something that their stat command actually recognizes as "symbolic link".)
• cmd.exe's builtin "mklink" command with the "/D" parameter
  Which creates a directory symbolic link. (See MS Docs)
• cmd.exe's builtin "mklink" command with the "/J" parameter
  Which creates a directory junction AKA soft link AKA reparse point. (See MS Docs)

Release Notes Entry

Also symbolic links keep popping up in the release notes. As of 2.30.0 this here is still listed as a "Known issue":

>On Windows 10 before 1703, or when Developer Mode is turned off, special permissions are required when cloning repositories with symbolic links, therefore support for symbolic links is disabled by default. Use git clone -c core.symlinks=true <URL> to enable it, see details here.

Here is a batch script for converting symlinks in repository, for files only, based on Josh Lee's answer. Script with some additional check for administrator rights is at https://gist.github.com/Quazistax/8daf09080bf54b4c7641.

@echo off
pushd "%~dp0"
setlocal EnableDelayedExpansion

for /f "tokens=3,*" %%e in ('git ls-files -s ^| findstr /R /C:"^120000"') do (
     call :processFirstLine %%f
)
REM pause
goto :eof

:processFirstLine
@echo.
@echo FILE:    %1

dir "%~f1" | find "<SYMLINK>" >NUL && (
  @echo FILE already is a symlink
  goto :eof
)

for /f "usebackq tokens=*" %%l in ("%~f1") do (
  @echo LINK TO: %%l
  
  del "%~f1"
  if not !ERRORLEVEL! == 0 (
    @echo FAILED: del
    goto :eof
  )

  setlocal
  call :expandRelative linkto "%1" "%%l"
  mklink "%~f1" "!linkto!"
  endlocal
  if not !ERRORLEVEL! == 0 (
    @echo FAILED: mklink
    @echo reverting deletion...
    git checkout -- "%~f1"
    goto :eof
  )

  git update-index --assume-unchanged "%1"
  if not !ERRORLEVEL! == 0 (
    @echo FAILED: git update-index --assume-unchanged
    goto :eof
  )
  @echo SUCCESS
  goto :eof
)
goto :eof

:: param1 = result variable
:: param2 = reference path from which relative will be resolved
:: param3 = relative path
:expandRelative
  pushd .
  cd "%~dp2"
  set %1=%~f3
  popd
goto :eof

I use sym links all the time between my document root and git repo directory. I like to keep them separate. On windows I use mklink /j option. The junction seems to let git behave normally:

>mklink /j <location(path) of link> <source of link>

for example:

>mklink /j c:\gitRepos\Posts C:\Bitnami\wamp\apache2\htdocs\Posts

I was looking for an easy solution to deal with the unix symbolic links on windows. Thank you very much for the above Git aliases. There is one little optimization that can be done to the rm-symlinks so that it doesn't delete the files in the destination folder in case the alias is run a second time accidentally. Please observe the new if condition in the loop to make sure the file is not already a link to a directory before the logic is run.

git config --global alias.rm-symlinks '!__git_rm_symlinks(){
for symlink in $(git ls-files -s | egrep "^120000" | cut -f2); do
    *if [ -d "$symlink" ]; then
      continue
    fi*
    git rm-symlink "$symlink"
    git update-index --assume-unchanged "$symlink"
done
}; __git_rm_symlinksenter 

One simple trick we use is to just call git add --all twice in a row.

For example, our Windows 7 commit script calls:

$ git add --all
$ git add --all

The first add treats the link as text and adds the folders for delete.

The second add traverses the link correctly and undoes the delete by restoring the files.

It's less elegant than some of the other proposed solutions but it is a simple fix to some of our legacy environments that got symlinks added.

Here's a powershell script to replace unix symlinks with windows

# This fixes Permission denied errors you might get when
#  there are git symlinks being used on repositories that
#  you share in both POSIX (usually the host) and Windows (VM).
#
# This is not an issue if you are checking out the same
#  repository separately in each platform. This is only an issue
#  when it's the same working set (aka make a change w/out
#  committing on OSX, go to Windows VM and git status would show
#  you that change).
#
# Based on this answer on stack overflow: http://stackoverflow.com/a/5930443/18475
#
# No warranties, good luck
#
# MUST BE RUN IN ELEVATED POWER SHELL

$ROOT = $PWD

$symlinks = &git ls-files -s | gawk '/120000/{print $4}'
foreach ($symlink in $symlinks) {
  $content = &Get-Content $symlink
  $content = $content.Replace("/", "\")
  $filename = $symlink.Split("/")[-1]
  cd (dirname $symlink)
  rm $filename
  echo Linking $content -> $filename
  New-Item -ItemType SymbolicLink -Path $filename -Target $content
  &git update-index --assume-unchanged $symlink
  cd $ROOT
}