Is it possible for git-merge to ignore line-ending differences?

Is it possible for git merge to ignore line-ending differences?

Maybe I'm asking the wrong question ... but:

I tried uisng config.crlf input but things got a bit messy and out of control, specially when I applied it after the fact.

For one thing, applying this config after the fact doesn't seem to affect files that were committed to the repository before applying this option. Another thing is that suddenly all commits now result in lots of annoying warning messages about CRLF being converted to LF.

To be honest, I don't really care what line-ending is used, I personally prefer the Unix style \n, but whatever. All I care about, is for git merge to be a bit smarter and ignore the differences in line-endings.

Sometimes I have two identical files, but git would mark them as being in conflict (and the conflict is the whole file) simply because they use a different line ending character.

Update:

I found out that git diff accepts a --ignore-space-at-eol option, would it be possible to let git merge use this option as well?

Update 2013:

More recent git versions authorize using merge with strategy recursive and strategy option (-X):

• from "Git Merge and Fixing Mixed Spaces and Tabs with two Branches":

  git merge -s recursive -Xignore-space-at-eol
  

But using "-Xignore-space-change" is also a possibility

• Fab-V mentions below:

  git merge master -s recursive -X renormalize
  

---

jakub.g also comments that the strategies work also with cherry-picking:

git cherry-pick abcd123456 --strategy=recursive --strategy-option=renormalize 

This works much better than ignore-all-space.

---

Before Git 2.29 (Q4 2020), All "mergy" operations that internally use the merge-recursive machinery should honor the merge.renormalize configuration, but many of them didn't.

See commit 00906d6, commit 8d55225, commit 6f6e7cf, commit fe48efb (03 Aug 2020) by Elijah Newren (newren).
^{(Merged by Junio C Hamano -- gitster -- in commit 4339259, 10 Aug 2020)}

> ## merge: make merge.renormalize work for all uses of merge machinery
> ^{Signed-off-by: Elijah Newren}

> The 'merge' command is not the only one that does merges; other commands like checkout -m or rebase do as well.
> > Unfortunately, the only area of the code that checked for the "merge.renormalize" config setting was in builtin/merge.c, meaning it could only affect merges performed by the "merge" command.
> > Move the handling of this config setting to merge_recursive_config() so that other commands can benefit from it as well.

---

Original answer (May 2009)

The patch for ignoring eol style has been proposed in June 2007, but it only concerns git diff --ignore-space-at-eol, not git merge.

At the time, the question has been askeed:

> Should --ignore-space-at-eol be an option to git-merge ?
Merges are where this functionality matters.
What are the semantics of an auto-resolved merge with those options in effect -- are they only used for rename detection, or do we, e.g., not flag conflicts with only whitespace changes ? And if we don't, which version do we accept automatically ?

Julio C Hamano was not exactly enthusiastic:

> This certainly is tempting, but I suspect that should be left to later rounds.
I suspect that it would introduce a concept of two different kinds of diffs, one to be mechanically processed (i.e. use in merge with "git-merge-recursive", and apply with "git-am"), and another to be inspected by humans to understand.
It often may be useful to munge the input for the latter case, even though the output from comparing munged input files may not be readily usable for mechanical application.

The general idea, when it comes to git merge, is to rely on the third-party merge tool.

For instance, I have setup DiffMerge to be the tool for Git merge, setting a ruleset which allow that merge tool to ignore eol for certain type of files.

---

Setup on Windows, with MSysGit1.6.3, either for DOS or Git bash session, with DiffMerge or KDiff3:

• set a directory into your PATH (here: c:\HOMEWARE\cmd).
• add in that directory the script merge.sh (wrapper for your favorite merge tool)

merge.sh:

#!/bin/sh

# Passing the following parameters to mergetool:
#  local base remote merge_result

alocal=$1
base=$2
remote=$3
result=$4

if [ -f $base ]
then
    #"C:/Program Files/SourceGear/DiffMerge/DiffMerge.exe" "$alocal" "$base" "$remote" -m --result="$result" --title1="Mine" --title2="Merging to: $result" --title3="Theirs"

    # for merge respecting eol, KDiff3 is better than DiffMerge (which will always convert LF into CRLF)
    # KDiff3 will display eol choices (if Windows: CRLF, if Unix LF)
    "C:/Program Files/KDiff3/kdiff3.exe" -m "$base" "$alocal" "$remote" -o "$result"
else
    #there is not always a common ancestor: DiffMerge needing 3 files, BASE will be the result
    #"C:/Program Files/SourceGear/DiffMerge/DiffMerge.exe" "$alocal" "$result" "$remote" -m --result="$result" --title1="Mine" --title2="Merging to: $result" --title3="Theirs"
    
    # KDiff3 however does know how to merge based on 2 files (not just 3)
    "C:/Program Files/KDiff3/kdiff3.exe" -m "$base" "$remote" -o "$result"
fi

• Declare your merge wrapper for Git

Git config commands:

git config --global merge.tool diffmerge
git config --global mergetool.diffmerge.cmd "merge.sh \"$PWD/$LOCAL\" \"$PWD/$BASE\" \"$PWD/$REMOTE\" \"$PWD/$MERGED\"
git config --global mergetool.diffmerge.trustExitCode false
git config --global mergetool.diffmerge.keepBackup false

• Check that autoCRLF is false

git config at system level:

git config ---system core.autoCRLF=false

• Test that, when two lines are identical (but their eol chars), both DiffMerge or KDiff3 will ignore those line during a merge.

DOS script (note: the dos2unix command comes from here, and is used to simulate a Unix eol-style. That command has been copied in the directory mentioned at the beginning of this answer.):

C:\HOMEWARE\git\test>mkdir test_merge
C:\HOMEWARE\git\test>cd test_merge
C:\HOMEWARE\git\test\test_merge>git init
C:\HOMEWARE\git\test\test_merge>echo a1 > a.txt & echo a2 >> a.txt
C:\HOMEWARE\git\test\test_merge>git add a.txt
C:\HOMEWARE\git\test\test_merge>git commit -m "a.txt, windows eol style"
C:\HOMEWARE\git\test\test_merge>git checkout -b windows
Switched to a new branch 'windows'
C:\HOMEWARE\git\test\test_merge>echo a3 >> a.txt & echo a4 >> a.txt
C:\HOMEWARE\git\test\test_merge>git add a.txt
C:\HOMEWARE\git\test\test_merge>git commit -m "add two lines, windows eol style"
C:\HOMEWARE\git\test\test_merge>git checkout master
C:\HOMEWARE\git\test\test_merge>git checkout -b unix
Switched to a new branch 'unix'
C:\HOMEWARE\git\test\test_merge>echo au3 >> a.txt & echo au4 >> a.txt && echo au5 >> a.txt
C:\HOMEWARE\git\test\test_merge>dos2unix a.txt
Dos2Unix: Processing file a.txt ...
C:\HOMEWARE\git\test\test_merge>git add a.txt
C:\HOMEWARE\git\test\test_merge>git commit -m "add 3 lines, all file unix eol style"
[unix c433a63] add 3 lines, all file unix eol style

C:\HOMEWARE\git\test\test_merge>git merge windows
Auto-merging a.txt
CONFLICT (content): Merge conflict in a.txt
Automatic merge failed; fix conflicts and then commit the result.

C:\HOMEWARE\git\test\test_merge>git ls-files -u
100644 39b4c894078a02afb9b1dfeda6f1127c138e38df 1       a.txt
100644 28b3d018872c08b0696764118b76dd3d0b448fca 2       a.txt
100644 3994da66530b4df80189bb198dcfac9b8f2a7b33 3       a.txt

C:\HOMEWARE\git\test\test_merge>git mergetool
Merging the files: a.txt

Normal merge conflict for 'a.txt':
  {local}: modified
  {remote}: modified
Hit return to start merge resolution tool (diffmerge):

At this point (Hitting "return"), DiffMerge or KDiff3 will open, and you will see for yourself what lines are actually merged, and what lines are ignored.

Warning: the result file will always be in Windows eol mode (CRLF) with DiffMerge...
KDiff3 offers to save in one way or another.

I was looking for the same answer and I found out this

> # Merging branches with differing checkin/checkout attributes > > If you have added attributes to a file that cause the canonical > repository format for that file to change, such as adding a > clean/smudge filter or text/eol/ident attributes, merging anything > where the attribute is not in place would normally cause merge > conflicts. > > To prevent these unnecessary merge conflicts, git can be told to run a > virtual check-out and check-in of all three stages of a file when > resolving a three-way merge by setting the merge.renormalize > configuration variable. This prevents changes caused by check-in > conversion from causing spurious merge conflicts when a converted file > is merged with an unconverted file. > > As long as a "smudge→clean" results in the same output as a "clean" > even on files that are already smudged, this strategy will > automatically resolve all filter-related conflicts. Filters that do > not act in this way may cause additional merge conflicts that must be > resolved manually.

So running this command in any repository will do the trick:

git config merge.renormalize true

After reading https://stackoverflow.com/a/12194759/1441706 and https://stackoverflow.com/a/14195253/1441706

for me, this command did the trick perfectly:

git merge master -s recursive -X renormalize

As in this answer: https://stackoverflow.com/a/5262473/943928

You could try: git merge -s recursive -Xignore-space-at-eol

What I did was leave everything as default (i.e. autocrlf=true), touch all files (find . -exec touch {} ;), let git see them as 'modified' and commit them back, and be done with it. Otherwise you'll always either be plagued by annoying messages or surprising differences, or have to turn off all of git's whitespace features.

You'll lose blame information, but it's better to do it sooner rather than later :)

"git merge -Xrenormalize" works like a charm.

It seems to me now that the best way is to normalized the line endings on both branches (and commit) before merging them.

I googled "convert crlf to lf" and found this as the first results:
<http://stahlforce.com/dev/index.php?tool=remcrlf>

I downloaded it and used, seems like a nice tool.

>sfk remcr . .py

Be sure though to specify a directory and a file type (e.g. .py) otherwise it might try to mess with the contents of the .git directory!

AFAICT, (I haven't tried it) you could use git diff to compare the branch you want to merge to the common ancestor, then apply the results with git apply. Both commands have --ignore-whitespace options to ignore line ending and white space errors.

Unfortunately, if the patch doesn't apply cleanly, the whole operation is aborted. You can't fix merge conflicts. There is a --reject option to leave unpatchable hunks in .rej files, which helps, but isn't the same as having the merge conflicts shown in one file.

After reading https://stackoverflow.com/questions/14250257

I finally resolved my version of this issue. I was trying to pull updates from the upstream repo but my current one was having CRLF related issues and was unable to merge as result. It should be noted I had NO LOCAL CHANGES i needed to worry about. The following steps resolved my issue:

As per github's instructions on syncing forks (https://help.github.com/articles/syncing-a-fork/):

1. git fetch upstream

2. git reset --hard upstream/master
   My limited understanding of git tells me this is doing what i want-- rebasing my fork (with no actual uncommitted changes) to gain all the changes made to the upstream source. According to the source page, this step should normally not be required, but the CRLF issue made it required.

3. git merge upstream/master

4. git push

however i suggest to use tool like sed to achieve correct line endings, and then diff files. I spent couple hours on diffing projects with various line endings.

The best way was to:

1. copy only project files (omit .git directory) to another directory create repository in it, then add files and commit them (should be on master branch in new repository).
2. copy files from second project to same folder, but another branch for example dev (git checkout -b dev), commit files on this branch and run (if first project is in master): git diff master..dev --names-only to see names of changed files only