What is equivalent to extending Mercurial extended files in Git?

Question

I am migrating some personal project repositories to Git from Mercurial. One of the projects relies on some non-changing, but large, shapefiles and SQLite databases. These files are important and need to live inside the repo so that everyone who viewed the project has access to them. Mercurial was easy to handle; I used the large file extension. largefiles automatically processes file additions / changes, without trying to parse the contents of files larger than X. That is, I could do hg addremove , and everything will work.

Git, like Mercurial, is not designed to track large files. However, I do not see such an extension. I looked at git-annex , but it seems to me that I need to manually track the files (i.e. I can’t just do git add -A arbitrarily), Also, if I read this right, git -annex seems to support large files in a completely separate repo. I want to save large files in my current repo in the directories that they currently live.

How do people deal with this situation? Of course, there are many projects that need to track large files that are integral to the work of the project. Will git -annex do this, or do I need some other extension?

Answer 1

largefiles automatically processes file additions / changes without trying to parse the contents of files larger than X.

How is this different from core.bigFileThreshold ? -

core.bigFileThreshold

Files larger than this size are stored deflated without delta compression. Storing large files without delta compression avoids excessive memory usage with a slight increase in disk usage.
The default is 512 MiB on all platforms. This should be reasonable for most projects, since source code and other text files can still be delta compressed, but large multimedia binaries will not. "

Answer 2

The only git-like system designed to work with large (even very large) files:

bup (see GitMinutes # 24 for details)

The result is an actual git repo that a regular git command can read.

I will explain in detail how bup differs from git in git in large files .

---

Of course, there are many projects that need to track large files that are integral to the work of the project.

No no. This is simply not what git is for, and even git-annex is a workaround that is not entirely satisfactory: see " git-annex with large files ".
I mention other tools in How to handle a large git repository? .

Answer 3

I track the hash of the md5 file of large files, unlike the files themselves. I also have a script that will exit and upload large files that are tracked in the repository.

I'm sure there are much better methods than this, but it works as a last resort.