Git Submodule | Tracking sub-repos within your Git repo
Table of Contents
- What is Git Submodule?
- Alternatives to Git Submodule
- Git Submodule Use Cases
- How to Use Git Submodule?
- Adding a Submodule
- Pushing Updates
- How to Clone Project Repositories with Submodules
- How to Merge Submodule Changes
- Submodule Tricks
- Next steps
When you begin a new project, you may want to use an external git repo to reference preexisting code. Several options exist for accomplishing this, whether that be copying and pasting the code from the external repository or using a specific package management system like NPM.
Despite their convenience, these methods have limitations, such as being unable to track changes made to external git repositories. This is where the
git submodule command comes in. A git submodule references another repository so that you can incorporate external code. It is the preferred option because it can point to a snapshot of a specific commit in another repository.
Continue reading to learn more about how git submodule works, the best use cases for git submodule, and how to use git submodule safely and effectively.
What is Git Submodule?
A Git submodule is essentially an external repository contained in your local repository to leverage existing code in additional projects.
Git submodule allows you to embed a repository within your main or ‘master’ repository. When using git submodule, you are not actually adding any code through the submodule but instead adding information about the submodule. This submodule information acts as a link to a specific commit.
Because the code is not added to the main repository, the submodule update git is not committed when you update your main repository. This is great if you have made changes to the submodule commit but want to use that previous snapshot because you know it works correctly with your current project.
Alternatives to Git Submodule
Instead of using
git submodule to reuse code, you have two options. The first is simply copying code from the external repository and pasting it into your main repository. While this is a quick and easy way to reuse old code, it's not ideal since you will lose any upstream changes unless you copy and paste the updated code too.
The second option is to use a language package management system like NPM for Node or Gems for Ruby. This can be a great option if you’re working in a specific language with a supportive community and already have everything installed and appropriately versioned. However, it is not as powerful as git submodule since you cannot handle changes to the external repository.
Git Submodule Use Cases
Git submodule is a valuable command to have in your toolbox for a variety of situations. Not only does it allow you to reuse code from other projects, but you are grabbing a snapshot of a commit. This means you can continue working on the repository from which you pulled the code without breaking your current project.
Separating code is one of the most common uses of git submodules in large, complex projects. You can break up extensive codebases with multiple components to make them easier to handle or to delegate specific tasks for each component.
Each component can be added to your main project repository as a submodule. This creates a cleaner repository and allows you to continue to work on components without breaking the overall functionality of your project.
Additionally, you can reuse a component you've created across multiple projects. When you add your component as a submodule, you can easily update it in multiple repositories. This is more flexible and straightforward than copying and pasting, as you won’t need to periodically go back and paste in updated code for multiple projects.
How to Use Git Submodule?
Getting the hang of git submodule is relatively straightforward. Let’s discuss some of the most common tasks you will encounter when integrating this command into your git workflow.
Adding a Submodule
You can easily add a submodule to your main repository using the code format ‘git submodule add email@example.com:url+to/first_submodule.git path_to_first_submodule’
Here’s what it looks like in code form:
$ git submodule add <remote_url> <destination_folder>
And let’s take a look at an example. The following code snippet is adding the ‘old-project’ repository as a submodule into a folder named ‘reuse-code’ on your main repository:
$ git submodule add https://github.com/old-project/old-project.git reuse-code Cloning into '/home/user/main/old-project’... remote: Enumerating objects: 1000, done. remote: Total 1000 (delta 0), reused 0 (delta 0), pack-reused 1000 Receiving objects: 100% (1000/1000), 3.03 MiB | 3.38 MiB/s, done. Resolving deltas: 100% (500/500), done.
This creates a folder in your submodule Git repository. The folder is named according to the submodule you added. In the example above, it would be ‘reuse-code.’ It also adds a hidden ‘.gitmodules’ file and updates your git configuration file.
If you use git status at this point, you will see these two new files need to be committed:
$ git status On branch master No commits yet Changes to be committed: (use "git rm --cached <file>..." to unstage) new file: .gitmodules new file: reuse-code
Commit and Push Submodule
Next, you need to use
git commit and git push on your submodule, which is currently in a staged state:
$ git commit -m 'Added reuse-code submodule to repository' $ git push
Retrieving the Submodules Code
Now the submodule needs to be initialized with the
git submodule init command:
$ git submodule init
With this command, you are adding all relevant entries to the git configuration file, so you can then run update git submodule as needed to retrieve the contents of the submodules. In other words, it’s tracking and copying the submodule information from the git modules file (.gitmodules) to the git configuration file (.git/config).
git submodule init command initializes all submodules present in the parent repository because no directory paths were provided. You can provide a specific path to a specific submodule if you only want to initialize a single submodule.
If you need to make changes to the submodule’s code, you can do this as if the submodule is a standalone repository. Then you can push these changes using the normal git workflow.
To update the submodule pointer to a different commit, start by
cd -ing inside the submodule directory:
$ cd reuse-code/
git checkout to point to the intended commit or branch:
$ git checkout -b ‘new-changes’ Switched to a new branch 'new-changes'
After you cd back to the parent repository, you can use the git status command to see the following output:
$ git status Changes not staged for commit: (use ‘git add <file>...’ to update what will be committed) (use ‘git checkout -- <file>...’ to discard changes in working directory) modified: old-project/reuse-code (modified content)
When you make your next commit in the main repository, it will update the old pointer to the new pointer—in this case, ‘new-changes.’
All team members will then need to update the code of their submodules. Using
git pull does not automatically do this for you. Instead,
git pull retrieves the information of the submodule pointing to a new commit. To update the actual code of a submodule, everyone should run the following command:
$ git submodule update
If you or other team members don’t run this command, your submodule will remain checked out to an old commit. You’ll see the same git status output above since the Head is detached until you run the git submodule update command.
How to Clone Project Repositories with Submodules
Cloning a repository that has submodules is a very simple process. Just clone the repository and then follow through with the initialization and update process outlined below.
Use the git clone command within the main repository.
$ git clone
Next, use the
git submodule init command to initialize the submodule in the cloned repository.
$ git submodule init
git submodule update to update the submodule code and reattach the HEAD.
$ git submodule update
How to Merge Submodule Changes
If you’ve made changes to the repository that the code has been pulled from, you can update the submodules with the following command:
$ git submodule --remote --merge
This command updates your detached HEAD to the newest commit in the submodule repository.
We previously discussed how it’s crucial to initiate and update your submodules. As yet another thing to remember, it is easy to forget to do this. Fortunately, git submodule offers a few tricks to help you stay on track.
git clone --recurse-submodules command allows you to clone your repository while also checking out and initializing any submodules contained within the repository.
git pull --recurse-submodules command pulls the main repository and all its submodules.
Additionally, you might want to execute a command for all submodules simultaneously if your project has multiple submodules. You can use the
foreach argument to loop over all of your checked-out submodules with the same command. For example, you would call the git submodule foreach argument:
$ git submodule foreach <command>
Git submodule is a handy tool for reusing code from other projects safely and effectively. Best of all, it doesn’t have to be your own code. The way git submodule snapshots a commit protects your repo from upstream changes so that you can read code from libraries or open-source projects.
Compared to more cumbersome methods like copying a pasting, git submodule is significantly more efficient for reusing existing code. Now that you have all the git submodule training you need to get started, jump in and try out git submodule while you build your next project!
If you're interested in learning more about how Git works under the hood, check out our Decoding Git Guidebook for Developers, which dives into Git's code in an accessible way. We wrote it for curious developers to learn how Git works at the code level. To do this, we documented the first version of Git's code and discuss it in detail.
We hope you enjoyed this post! Feel free to shoot me an email at firstname.lastname@example.org with any questions or comments.
- Git SCM Docs - https://git-scm.com/docs/git-submodule
Recommended product: Decoding Git Guidebook for Developers