Move Git LFS tracked files under regular Git

GitGit Lfs

Git Problem Overview


I have a project where I stored video files with Git LFS. Now I ran into some complications with my build server that doesn't yet support Git LFS. As it's an external service, I can't really affect the build process, and thus would want to move the files from under Git LFS back to "regular" Git. I managed to untrack the file types with git lfs untrack '<file-type>' but git lfs ls-files still gives a list of the files previously added.

I imagine I could remove the files, push the changes and then manually re-add them, but is this really the recommended way of doing things?

Git Solutions


Solution 1 - Git

I have just recently run into this problem where assets were accidentally added to git-lfs on one branch that shouldn't have been. My solution was:

git lfs untrack '<file-type>'
git rm --cached '<file-type>'
git add '<file-type>'
git commit -m "restore '<file-type>' to git from lfs"

The result is a rewrite of the git-lfs oid sha256 pointers with the standard file contents.


(Edit 2019-03): The accepted answer was changed to provide an easy solution for simpler cases. See also the edits in the answer by VonC for alternate solutions in case you have a more complex case on hand.

Solution 2 - Git

As of Git 2.16 (released Jan 17th, 2018), you can do this easily with the --renormalize flag of git add:

git lfs untrack "<pattern>"
git add --renormalize .
git commit -m "Restore file contents that were previously in LFS"

From Git's documentation:

> --renormalize: > Apply the "clean" process freshly to all tracked files to > forcibly add them again to the index. This is useful after > changing core.autocrlf configuration or the text attribute > in order to correct files added with wrong CRLF/LF line endings. > This option implies -u.

The key part here is "all tracked files". Normally, filters are only run when a Git operation changes a file in the work tree. Changing the LFS whitelist in .gitattributes isn't a Git operation, and so the index ends up in an inconsistent state after you run git lfs untrack. Running git add --renormalize . tells Git to re-run filters on every file in the repository, which ensures that all files which should be in LFS are—and that all files which shouldn't be aren't.

Solution 3 - Git

Issue 641 mentions the same issue.

> I tried to stop using Git LFS, but found no way to revert my previous tracked pointer files using git lfs uninit, git lfs untrack, git rm... after I move those files back it still lists as tracked by Git LFS with git lfs ls-files, how can I opt out the whole Git LFS stuff from my repo?

The answer was:

> 1. Remove all filter.lfs.* git config entries with git lfs uninit. 2. Clear any any attributes that use the lfs filter in .gitattributes by running git lfs untrack for each file type, or deleting .gitattributes if LFS is all you ever used it for.

After this, any added files will go straight to git.

But this was not so simple:

> I later end up LFS pointer files in my working directory and have to recover all my pictures from .git/lfs using the sha1 hash stored in those pointers manually.


Update March 2016, the issue 957 illustrates a possible solution by tstephens619:

> I made the same mistake of including several small graphics formats into my git lfs tracking list.
I was able to move this files back into git by doing the following:

> - Create a list of all of the files currently being tracked by git-lfs, filter out *.gz and *.rpm (I want to still track those extensions with git-lfs)

> git lfs ls-files | grep -vE ".gz|.rpm$" | cut -d ' ' -f 3 > ~/temp/lfs-files.txt

> - Stop tracking the small graphics files

> git lfs untrack ".tts" git lfs untrack ".bfx" git lfs untrack ".ttf" git lfs untrack ".xcf" git lfs untrack ".pkm" git lfs untrack ".png"

> - Temporarily uninit git-lfs

> git lfs uninit # Git LFS 2.x+ git lfs uninstall

> - Use the file list to touch each file:

> cat ~/temp/lfs-files.txt | xargs touch

> git status will now show each file as modified

> - Add the changes to git index (I did this via git gui)

> - commit the changes and then re-init git-lfs

> git commit git lfs init


The maintainer ttaylorr adds:

> One way to do this would be:

for file in $FILES_TO_REVERT; do
  git lfs untrack "$file";
  git rm --cached "$file";
  git add --force "$file";
done

git commit -m "..."

> My preference would be not to add a command to Git LFS to the above effect, since it is possible in a number of different way with the porcelain commands provided by Git and Git LFS

Solution 4 - Git

EDIT: After a couple of years of using GIT LFS successfully, and a number of up/down votes on this answer, I think this warning still applies: GIT LFS has a lot of flaws, such as difficult management of which files should be in LFS, performance issues when (accidentally) adding many small files to LFS on Windows, limited support for multiple remotes and remote URL formats, difficulty removing files from LFS, various problems you can run into when merging, etc. GIT LFS is a foreign element in GIT that exists outside the revision tree. I would, however, like to rephrase my original warning as follows:

  1. Only put files into GIT LFS that you would normally put into GIT (e.g., "source files" that you own and change occasionally)
  2. Only put large files into GIT LFS.
  3. If you need a system for managing binary dependencies, consider using a package manager instead.
  4. Don't use Subversion as a replacement for GIT LFS. It is worse.
  5. Be prepared to mess up your working directory. Make sure to back up (push) valuable changes before you do any major changes to LFS.
  6. When merging, always merge .gitattributes first.

EDIT: Here is my original answer:

It is difficult remove anything from GIT LFS, and although the solutions presented here may work (with modifications), they require a lot of effort and may have side effects on your repository.

If you arrived here, it is time to ask yourself whether you want to manage your large files with GIF LFS and whether GIT itself (which is inherently bad at managing large files, because it is a distributed version control system) was a good choice.

If you have many large files and you are a single organization working on your project, something like Subversion may work better for you.

Solution 5 - Git

I had problems doing steps in Windows. To remove all git lfs tracked files and restore original file I did the following in git bash:

  1. Removed .gitattributes

  2. git lfs ls-files | cut -d ' ' -f 3 > lfs-files.txt

  3. Execute following snippet:

Snippet:

while read file; do
  git lfs untrack "$file";
  git rm --cached "$file";
  git add --force "$file";
done <lfs-files.txt

Solution 6 - Git

I tried and was able to use the following command successfully to revert my repo back to a regular one according to their official doc:

git lfs migrate export --include="*.psd" --everything

The link to the original official doc is at https://github.com/git-lfs/git-lfs/blob/main/docs/man/git-lfs-migrate.1.ronn#migrate-local-history.

Hope this is of use to anyone in the same situation!

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionOlli NiskanenView Question on Stackoverflow
Solution 1 - GitmredView Answer on Stackoverflow
Solution 2 - GitTom HebbView Answer on Stackoverflow
Solution 3 - GitVonCView Answer on Stackoverflow
Solution 4 - GitFlorian WinterView Answer on Stackoverflow
Solution 5 - GitJokerView Answer on Stackoverflow
Solution 6 - GitKris SternView Answer on Stackoverflow