Why can't I push this up-to-date Git subtree?

I am using Git subtree with a couple of projects that I am working on, in order to share some base code between them. The base code gets updated often, and the upgrades can happen in anyone of the projects, with all of them getting updated, eventually.

I have ran into a problem where git reports that my subtree is up to date, but pushing gets rejected. For example:

#! git subtree pull --prefix=public/shared project-shared master
From github.com:****
* branch            master     -> FETCH_HEAD
Already up-to-date.

If I push, I should get a message that there is nothing to push... Right? RIGHT? :(

#! git subtree push --prefix=public/shared project-shared master
git push using:  project-shared master
To [email protected]:***
! [rejected]        72a6157733c4e0bf22f72b443e4ad3be0bc555ce -> master (non-fast-forward)
error: failed to push some refs to '[email protected]:***'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Merge the remote changes (e.g. 'git pull')
hint: before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

What could be the reason for this? Why is pushing failing?

I found the answer on this blog comment https://coderwall.com/p/ssxp5q

> If you come across the "Updates were rejected because the tip of your current branch is > behind. Merge the remote changes (e.g. 'git pull')" problem when you're pushing (due to > whatever reason, esp screwing about with git history) then you'll need to nest git > commands so that you can force a push to heroku. e.g, given the above example:

git push heroku `git subtree split --prefix pythonapp master`:master --force

On windows the nested command doesn't work:

git push heroku `git subtree split --prefix pythonapp master`:master --force

You can just run the nested bit first:

git subtree split --prefix pythonapp master

This will (after a lot of numbers) return a token, e.g.

157a66d050d7a6188f243243264c765f18bc85fb956

Use this in the containing command, e.g:

git push heroku 157a66d050d7a6188f243243264c765f18bc85fb956:master --force

Use the --onto flag:

# DOESN'T WORK: git subtree push --prefix=public/shared project-shared master --onto=project-shared/master

[EDIT: unfortunately subtree push doesn't forward --onto to the underlying split, so the operation has to be done in two commands! With that done, I see that my commands are identical to those in one of the other answers, but the explanation is different so I'll leave it here anyway.]

git push project-shared $(git subtree split --prefix=public/shared --onto=project-shared/master):master

Or if you're not using bash:

git subtree split --prefix=public/shared --onto=project-shared/master
# This will print an ID, say 0123456789abcdef0123456789abcdef,
# which you then use as follows:
git push project-shared 01234567:master

I spent hours poring through the git-subtree source to figure this one out, so I hope you appreciate it ;)

subtree push starts by running subtree split, which rewrites your commit history into a format which should be ready to push. The way it does this is, it strips public/shared/ off the front of any path which has it, and removes any information about files that don't have it. That means even if you pull non-squashed, all the upstream sub-repository commits are disregarded since they name the files by their bare paths. (Commits that don't touch any files under public/shared/, or merge commits that are identical to a parent, are also collapsed. [EDIT: Also, I've since found some squash detection, so now I'm thinking it's only if you pulled non-squashed, and then the simplistic merge commit collapsing described in yet another answer manages to choose the non-squashed path and discard the squashed path.]) The upshot is, the stuff it tries to push ends up containing any work someone committed to the current host repository you're pushing from, but not work people committed directly to the sub-repository or via another host repository.

However, if you use --onto, then all the upstream commits are recorded as OK to use verbatim, so when the rewriting process comes across them as one of the parents of a merge it wants to rewrite, it will keep them instead of trying to rewrite them in the usual way.

For a "GitHub pages" type app, where you deploy a "dist" subtree to a gh-pages branch, the solution might look something like this

git push origin `git subtree split --prefix dist master`:gh-pages --force

I mention this since it looks slightly different from the heroku examples given above. You can see that my "dist" folder exists on the master branch of my repo, and then I push it as a subtree to the gh-pages branch which is also on origin.

Had this issue, too. Here's what I did, based on the top answers:

Given:

#! git subtree push --prefix=public/shared project-shared master
git push using:  project-shared master
To [email protected]:***
! [rejected]        72a6157733c4e0bf22f72b443e4ad3be0bc555ce -> master (non-fast-forward)
error: failed to push some refs to '[email protected]:***'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Merge the remote changes (e.g. 'git pull')
hint: before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

Enter this:

git push <origin> 72a6157733c4e0bf22f72b443e4ad3be0bc555ce:<branch> --force

Note the token that is output by 'git subtree push' is used in 'git push'.

I have encountered this problem before as well, and here is how I solved it.

What I found out was I had a branch that was not attached to the local master branch. This branch exists and its just hanging in the void. In your case, its probably called project-shared. Assuming this is the case and when you do a git branch you can see a local project-shared branch, then you can 'append' new commits to your existing project-shared branch by doing a:

git subtree split --prefix=public/shared --onto public-shared --branch public-shared

The way I understood is git subtree will start creating new branch from --onto, in this case its the local public-shared branch. Then the branch means creating a branch, which just replaces the old public-shared branch.

This will keep all the previous SHA of the public-shared branch. Finally, you can do a

git push project-shared project-shared:master

Assuming that you have a project-shared remote as well; this will push the local hanging in the void project-shared branch to the master branch of the remote project-shared.

This is because of the limitation of original algorithm. When handling merge-commits, the original algorithm uses a simplified criteria for cutting off unrelated parents. In particular, it checks, if there is a parent, which has the same tree. If such a parent found, it would collapse the merge commit and use the parent commit instead, assuming that other parents have changes unrelated to the sub-tree. In some cases this would result in dropping parts of history, which has actual changes to the sub-tree. In particular it would drop sequences of commits, which would touch a sub-tree, but result in the same sub-tree value.

Lets see an example (which you can easily reproduce) to better understand how this works. Consider the following history (the line format is: commit [tree] subject):

% git log --graph --decorate --pretty=oneline --pretty="%h [%t] %s"
*   E [z] Merge branch 'master' into side-branch
|\
| * D [z] add dir/file2.txt
* | C [y] Revert "change dir/file1.txt"
* | B [x] change dir/file1.txt
|/
*   A [w] add dir/file1.txt

In this example, we are splitting on dir. Commits D and E have the same tree z, because we have commit C, which undone commit B, so B-C sequence does nothing for dir even though it has changes to it.

Now lets do splitting. First we split on commit C.

% git log `git subtree split -P dir C` ...
* C' [y'] Revert "change dir/file1.txt"
* B' [x'] change dir/file1.txt
* A' [w'] add dir/file1.txt

Next we split on commit E.

% git log `git subtree split -P dir E` ...
* D' [z'] add dir/file2.txt
* A' [w'] add dir/file1.txt

Yes, we lost two commits. This results in the error when trying to push the second split, since it doesn't have those two commits, which already got into the origin.

Usually you can tolerate this error by using push --force, since dropped commits generally won't have critical information in them. In the long term, the bug needs to be fixed, so the split history would actually have all commits, which touch dir, as expected. I would expect the fix to include a deeper analysis of parent commits for hidden dependencies.

For reference, here is the portion of original code, responsible for the behavior.

copy_or_skip()
  ...
  for parent in $newparents; do
      ptree=$(toptree_for_commit $parent) || exit $?
      [ -z "$ptree" ] && continue
      if [ "$ptree" = "$tree" ]; then
          # an identical parent could be used in place of this rev.
          identical="$parent"
      else
          nonidentical="$parent"
      fi
  ...
  if [ -n "$identical" ]; then
      echo $identical
  else
      copy_commit $rev $tree "$p" || exit $?
  fi

Eric Woodruff's answer did not help me, but the following did:

I usually 'git subtree pull' with the '--squash' option. It seems this did make things harder to reconcile, so I needed to do a subtree pull without squashing this time, resolve some conflicts and then push.

I must add that the squashed pull did not reveal any conflicts, told me everything was OK.

Another [simpler] solution is to advance the head of the remote by making another commit if you can. After you pull this advanced head into the local subtree then you will be able to push from it again.

You can force push local changes to remote subtree repo

git push subtree_remote_address.git `git subtree split --prefix=subtree_folder`:refs/heads/branch --force

A quick powershell for windows users based on Chris Jordan's solution

$id = git subtree split --prefix pythonapp master
Write-Host "Id is: $id"
Invoke-Expression "git push heroku $id`:master --force"

So this is what I wrote based off what @entheh said.

for /f "delims=" %%b in ('git subtree split --prefix [name-of-your-directory-on-the-file-system-you-setup] -b [name-of-your-subtree-branch]') do @set token=%%b
git push [alias-for-your-subtree] %token%:[name-of-your-subtree-branch] --force

pause