Duplicate Submodules with Git

Question

I have a project in Git that has several submodules, and I need these submodules that need to be downloaded, and the files available for using the main project, and for the submodules to work, I need their own submodules to be available, etc. . Therefore, to establish this, I recursively initialize submodules using git submodule update --init --recursive .

However, I noticed that many of my submodules share common dependencies, looking something like this in the pseudocode ( alpha -> beta means alpha has the beta submodule)

 my project -> submodule a -> submodule m -> submodule b -> submodule m -> submodule n -> submodule x -> submodule c -> submodule x 

My question is: is there a way to avoid this duplication using only git, saving (at least one copy) files for each submodule?

I can imagine a solution with symbolic links, but it would be preferable if Git handled this for me, and I'm not sure if installing the symbolic links itself will cause problems when updating submodules.

Ideally, I would like to simplify it:

 my project -> submodule a -> symlink(submodule m) -> submodule b -> symlink(submodule m) -> symlink(submodule n) -> submodule c -> symlink(submodule x) -> submodule m -> submodule n -> symlink(submodule x) -> submodule x 

Thanks in advance for any suggestions!

Answer 1

It is not built into git, but you can definitely do it with symbolic links, as you say. Perhaps you should take a look at git new-workdir (from the git contrib directory), which is essentially what it is. He does not know anything in common with submodules, but the submodule does not know this submodule - it is the parent repo that knows about it. I have not tried this, but I'm sure you can use it something like this:

 # remove the target first (new-workdir will refuse to overwrite) rm -rf submodule_b/submodule_m # (original repo) (symlinked repo) git new-workdir submodule_a/submodule_m submodule_b/submodule_m 

It works, essentially symbolizing the entire .git directory; a remarkable thing that is not symbolic, HEAD ; two directories can have different things checked, but they have the same links and objects.

From here you must be good. When you run the git submodule in a supermodule, it simply goes into submodules and runs the corresponding commands there, which will work as expected.

The only thing you usually need to know with symbolic repositories like this is that they have the same set of branches, so if they both have the same branch and you pass it to one, the other will stop syncing . With submodules this, as a rule, will not be a problem, because, since they are essentially always in the disabled state of HEAD, if you do not intervene.

Answer 2

git-new-workdir might not be the best solution, as discussed here: http://comments.gmane.org/gmane.comp.version-control.git/196019

This did not work for me in the git 1.7.10 section.

I solved this for my use using hard links. I am running OS X and the file system allows creating hard links to directories: https://github.com/darwin/hlink

Now I can hardlink the submodule directories and git processes them transparently. Hard binding also has the nice feature that all submodules are completely mirrored, including HEAD, which is the behavior that I prefer in my case.

Well, the idea is to have one “master” repo submodule and rigidly link all the “subordinate” copies back to it. This will make them indistinguishable from each other and fully synchronized.

WARNINGS

1) This works fine if relative paths in .git work. In other words, you can only tightly link submodules sitting on the same directory level in the directory tree. That was my case. I assume that you can easily fix this by modifying .gitfiles with your complex task. Note. This should not be a problem before git 1.7.10, because the .git submodule was previously a standalone directory, not just a plain-text .git file pointing somewhere else.

2) Hard links may introduce some incompatibilities. For example, TimeMachine is confused because it uses hard links internally for version control. Be sure to include your form in the TimeMachine project directory.

Here is an example of my rake task doing the job: https://github.com/binaryage/site/blob/3ef664693cafc972d05c57a64c41e89b1c947bfc/rakefile#L94-115