Image of Git-Sim 3 Month Dev Update: Community response, New Features, & The Future

Table of Contents

Introduction

These past 3 months have been a blur! On January 22nd, I released Git-Sim, a tool to visually simulate Git commands in your local repo.

As a quick refresher, it can simulate many Git operations, such as simulating merging a branch with git-sim merge <branch>:

Git-Sim clean command

In this dev update, we'll cover the community response to Git-Sim's release 3 months ago, the set of new supported Git subcommands, some notable new features such as implementing a robust branching system, enabling color-coded commits by author, support for simulating merge conflicts, and much more.

Finally, we'll mention what we're working on next (including a totally top-secret, next-gen version of Git-Sim), how you can contribute to Git-Sim, and how you can support the continued development of Git-Sim.

The community response to Git-Sim

Before releasing Git-Sim, I had been working on it for a few months and the basic idea was in place, but I wasn't entirely happy with its capabilities. Regardless, I felt like needed to put something out there to see if anyone would even be interested. If it didn't appeal to folks, I needed to know so that I didn't keep sinking time into it.

Luckily, that wasn't the case at all! The response to Git-Sim in its first 3 months of life was more than I could have imagined:

This is the first open-source project I've started that's gotten some real traction, and I'm grateful to every single person that read my original Git-Sim release post and especially those that installed Git-Sim, tested it out, and provided thoughtful feedback.

The main purpose of this post is to talk about how the release went and provide a general dev update on what's new in Git-Sim over the past 3 months.

Functional shortcomings with Git-Sim's initial release

I was pretty torn about whether to release Git-Sim when I did. On one hand, I thought the idea of simulating the effects of Git commands within the context of a user's repo in presentation-quality was cool, and I wasn't able to find an existing tool that did ... THAT.

On there other hand, there were several known limitations going into the release, and I knew many bugs would be discovered after the release.

Known limitations

I knew about 3 main limitations of Git-Sim going into the release:

1. Lack of full coverage of many commonly used Git subcommands

Obviously the initial release was missing the ability to simulate many core Git subcommands, notably ALL of the networked commands such as clone, fetch, pull, and push, were not included. These commands are used in pretty much all Git workflows, so a tool that is meant to simulate Git operations would be hard-pressed to succeed without including them.

The main reason was that I wasn't sure how best to simulate those commands without modifying the local and/or remote repos in some way (more on my solution to get around this later), and one of my goals for Git-Sim was to make it a read-only program which users can trust to NOT impact (i.e. break) their underlying repos.

Other notable subcommand omissions were rm, mv, switch, checkout, config, diff, clean, and reflog. I just didn't have time to implement these and wanted to get the release out the door to gauge interest in the tool.

2. Lack of ability to specify the number of commits to display in simulated output

Another major drawback in the initial release was that each Git-Sim subcommand could only display 5 commits. No more, no less. That is because the way the program parsed the commit history was dumb - very dumb.

It basically used GitPython API calls to fetch the most recent 5 commits starting at HEAD, put them into a Python list, and iterated through them to snag their details and draw them to the screen using Manim. The only exception to this was the git-sim log command, which had a non-Git option --commits=N to display the desired number of commits, but even those could only be displayed linearly.

3. Lack of ability to show diverse branching structures for arbitrary commands, such as multiple branches

Even worse, the tool had no way to trace the parent-child relationships in the DAG, so it lost out on the ability to illustrate rich commit history structures, such as multiple branches, branch divergence, and merges. Luckily that's all possible now!

But before we get to that, let's move on to the bugs that I didn't know about before the release.

Bugs

There were many bugs - both installation-related and functional - that Git-Sim's first users quickly started reporting

The installation bugs were mainly environment specific bugs that I simply didn't have the bandwidth to prepare for.

The functional bugs were mostly due to the variation of data and structure of the Git repos out there, along with some glaring oversights on my part.

I'm not too hung up on the bugs though. Those are to be expected in the initial release of a medium-sized project with a single dev, and I never would have been able to do enough testing on my own to weed them all out before the release. If nothing else, the initial release can be thought of as a big bug hunt across thousands of environments in the wild - a good way to weed the garden fast!

Deciding to push the release

All things considered, I felt that the state of Git-Sim at the time did a good enough job of communicating its purpose in a way that people could understand, in a stable enough way that many users could at least download it and test it out. So I released it.

New supported git subcommands

Probably the most important addition to the current version of Git-Sim - v0.3.0 at the time of this post is the following set of new supported subcommands:

  • switch
  • checkout
  • clone
  • fetch
  • pull
  • push
  • rm
  • mv
  • clean

This brings the total number of supported Git-Sim subcommands to 22, which in my opinion covers almost all of the commonly used Git commands (minus diff and reflog).

At the end of this post I'll touch on the commands that we'll be implementing in the near future, but for now, let's go through the new supported subcommands one by one to see how Git-Sim simulates them.

git-sim switch

Supporting the git switch command was one of the first feature requests we received on GitHub.

We added a basic version of this command which allows users to simulate switching between their git branches using git-sim switch <branch> and creating a new branch and switching to it using git-sim switch -c <new-branch>.

The following scenarios needed to be handled in the code:

  • Switching between branches that have diverged
  • Switching to a branch which is an ancestor of the active branch
  • Switching to a branch which is a descendant of the active branch

Example output:

Git-Sim switch command

git-sim checkout

Although git switch is often favored nowadays, git checkout is still commonly used out of habit and in legacy scripts. So we added support for that as well. It works pretty much the same as git switch, except to create a new branch and switch to it, the -b flag is used $ git-sim checkout -b <new-branch>.

Example output:

Git-Sim checkout command

git-sim clone

Finally! A supported Git subcommand that requires interaction with a remote repo! You can now run $ git-sim clone <repo-url> to simulate cloning a repo into a new folder in the current directory. And yes, cloning can be simulated using a true remote repo or a local Git directory.

But how do you simulate a clone without actually doing the clone? Trick question - you don't! My solution to the networked Git commands is to actually perform the clone in a local temp directory and use that to generate the simulated output. Once the Git-Sim image or video is created, the temp repo is simply deleted.

Note that this could lead to performance issues for large repos - like the Linux repo for example - since the user needs to wait for the whole repo to be cloned into temp each time. But for now it's the solution I have implemented.

One alternative would be to preserve the local temp copy so it doesn't need to be re-cloned from scratch each time. However, as we'll see when simulating other networked commands like fetch, pull, and push, the temp copy would need to be synced before each new simulation was generated.

For now the easier option is just to delete and re-clone the temp repo each time.

Example output:

Git-Sim clone command

git-sim fetch

Now before you go fetch Git-Sim yourself, you should know that you can now use git-sim fetch <remote> <branch> to simulate fetching changes from a remote repo to your local copy.

This works slightly differently from git-sim clone, because in this case the local repo already exists. However, since we want to simulate the fetch behavior without altering the local repo, we can just make a local temp copy of our existing local repo! Then we can run git fetch on that copy and use it to generate the simulated image/video without touching out real local repo at all.

Example output:

Git-Sim fetch command

git-sim pull

Now I'm not pulling your leg here, you can now use git-sim pull <remote> <branch> to simulate pulling changes from a remote branch into your local repo.

This works similar to fetch, where the whole operation is performed in a local temp copy of the local repo, and the result is used to generate the simulated image/video.

But remember how git pull is a combination of git fetch followed by git merge? This means that git pull can result in two scenarios that need to be handled :

  1. A successful fetch and merge
  2. A successful fetch followed by a failed merge due to merge conflict

Git-Sim is able to handle these situations. In the event of a merge conflict, a table is displayed showing which files are conflicted.

Example output:

Git-Sim pull command

git-sim push

Alright, let's push the previous commands aside, because git-sim push <remote> <branch> is comin' through! Simulating a Git-Push is the most complex yet, because in this case we want to make sure both the local repo AND remote repo aren't touched by our mischief.

This is accomplished by creating local temp copies of BOTH the local repo AND the remote repo, and running a git push between the two. The result is then be displayed in the Git-Sim output image/video.

Git push can also result in various scenarios that need to be handled:

  1. Successful push
  2. Failed push when the remote has new changes (commits) that don't exist locally
  3. Failed push when the tip of the local branch is behind the remote

Git-Sim can handle these scenarios, and in the event of scenario (2) even uses color-coding to show you which commits need to be pulled down from the remote before you can successfully push.

Example output:

Git-Sim push command

git-sim rm

The git-sim rm command was pretty trivial to implement, since it is functionally very similar to the git-sim add command which was included in the initial release.

Example output:

Git-Sim rm command

git-sim mv

Not much to say here folks, you can now use git-sim mv to simulate changing a filename and/or path to a file tracked by Git.

Example output:

Git-Sim mv command

git-sim clean

This was another simple and fun one to implement. You can now use git-sim clean to simulate deleting the untracked files from your repo.

Note that since this is just a simulation, there is no need to specify one of -i, -n, or -f as is required when using Git.

Example output:

Git-Sim clean command

Notable new features

In addition to adding the new subcommands above, here is a rundown of our biggest enhancements:

Implement a robust branching system

This addresses known limitation (2) and (3) that I mentioned above. I updated Git-Sim so it now recursively parses parent-child relationships to make it's way through the Git commit history / DAG (Directed Acyclic Graph).

This allows a much richer display of complex Git repos with multiple intertwining branches and merges.

In addition, there are 2 new global options -n N and --all that are related to this. These are both Git-native options which is a bonus because it should make it easier for users to knowledge transfer.

The -n N option is used to specify the number of commits to display in the output image/video (by default, N = 5). Note that this corresponds to the length of each displayed branch (think of this as "commits per branch"), not the total number of commits on screen.

The --all option is used to display all unrelated branches in the output image, like git log --graph --all.

So for example, the following command simulates merging the dev branch into main, displaying 10 commits per branch and including unrelated branch heads:

$ git-sim -n 10 --all merge dev

This makes the tool so much more robust!

Enabling color-coded commits by author

This is one of my favorite new features. The new global flag --color-by=author color-codes the commits in the output image or video by their author, and specifies how many commits in the displayed graph were made by that author.

Git-Sim clean command

Color is such an intuitive way to express information, that I'm glad Git-Sim has a feature that makes use of it. I hope to implement many more color-based features going forward.

Add "git-dummy" dependency to generate your own dummy Git repos to play with

I quickly realized when building Git-Sim that my Desktop filled up with a gazillion dummy repos that I was testing in.

For efficiency, I created git-dummy, a command-line tool that automates the process of generating dummy/test repos with the desired number of commits, branches, and merges. For example:

$ git-dummy --name=dummy  --branches 4 --commits 10 --diverge-at 2 --merge="1,2"

This creates a new Git repo named "dummy/" with 4 branches (1 main branch and 3 feature branches), 10 commits per branch, with branches diverging from main after 2 commits, and finally merging the 1st and 2nd branches back into main.

Git-Dummy is now a dependency of Git-Sim so that spinning up a repo to test on is a snap. You can even create up a a dummy repo and run Git-Sim on it with a single command:

$ git-dummy --commits 10 --branches 2 && git-sim -n 10 merge branch1

This is great for testing Git-Sim commands on specific repo structures that are annoying to create over and over again, like prepping two branches to be merged.

Support for simulating merge conflicts

Git-Sim now has the ability to detect merge conflicts when simulating merges, and to display a list of conflicted files when merge conflicts occur. This works by creating a local temp copy of the local repo, running the merge there, and processing the result. The local temp copy is then deleted.

In the future we will likely add an option to show the conflicted lines within those files as well.

Add global flags --invert-branches and --hide-merged-branches

Sometimes a user might want to reorient their commit graph to highlight a particular branch. By default, Git-Sim parses commit parents in the order that Git stores parents within each commit.

This ordering can now be reversed using the --invert-branches flag, which will have the effect of reorienting the resulting commit graph so that the second branch is drawn first (i.e. on top) of the first.

As a bonus, in the event you only want to show a linear representation of one branch while omitting merged branches, you can use the --hide-merged-branches flag. This will ensure the simulated output image/video shows a single line of development.

Add --amend option for "commit" subcommand

Based on this feature request on GitHub, we added the --amend flag for the git-sim commit subcommand. This simulates updating the commit message for the previous commit:

$ git-sim commit --amend -m "Amended commit message"

Add support for stash subcommands push/pop/apply

Our very first GitHub feature request was to add support for:

  • git-sim stash push
  • git-sim stash pop
  • git-sim stash apply

This has been implemented thanks to @abhijitnathwani!

Added support for webm output

This feature request was for the option to generate .webm video output format when using the --animate global flag along with the new --video-format=webm option:

$ git-sim --animate --video-format=webm log

The benefits of webm are smaller sized video file output optimized for use in web settings.

Add --img-format option to export images to png

Similarly, we added an --img-format option to support output images in png format:

$ git-sim --img-format=png log

Create Dockerfile

This feature request was to add a Dockerfile for folks to install and run Git-Sim without having to worry about environment-specific installation and configuration issues. We added one!

Add --stdout option to write image data directly to stdout while suppressing other output

Another GitHub feature request was for Git-Sim to redirect simulated image output to stdout for ingestion by other programs.

We added a nifty little global flag --stdout for this, which allows fun stuff like graphical terminals (like iTerm2) to display the simulation image directly in the terminal:

$ git-sim --stdout merge dev | viu -

Use Typer and pydantic in settings to handle arguments and environment variables

Thanks to @paketb0te, we implemented Typer and Pydantic into Git-Sim which automatically allows all of the global options and flags to conveniently be set via environment variables. Each option/flag name is simply prefixed with git_sim_ in order to be accessed as an environment variable.

For example, I add the following exports to my ~/.bashrc:

export git_sim_media_dir=~/git_sim_media
export git_sim_speed=2.5
export git_sim_reverse=true
export git_sim_light_mode=true

This removes the need to manually enter all these options when running Git-Sim commands.

Minor functional enhancements

Here's a list of more minor (but still important!) functional enhancements since the initial release:

  • Allow specification of remote tracking branches in merge and display them as branches in other commands
  • Support -e, --edit option for cherry-pick subcommand to specify edited commit message
  • Add --quiet, -q flag to suppress all output except errors
  • Add -m option to merge subcommand to allow user to set commit message
  • Add global flag --transparent-bg to generate output images with transparent background
  • Print output image/video location after completing simulation

What we're working on next

In a way, it feels like Git-Sim has already come so far in only a few months, but there is still a lot of work to do. These are the things we'll be focusing on going forward:

  1. Supporting even more Git subcommands: Git-Sim can now simulate 22 Git subcommands, but it's still missing some commonly used ones like git config, git diff, git reflog, git submodule, git bisect, git show, git blame, and more.
  2. Increasing coverage of existing Git subcommand options/flags: Many of the implemented subcommands only support one or two subcommand options/flags, many commonly used options/flags are not done yet.
  3. Adding more features that make use of color: We added the --color-by=author option to color-code commits by author in the simulated output. I believe there are many more ways we can use color to communicate interesting info about Git data, such as coloring commits by branch, date range, and more.
  4. Improve performance: Git-Sim's performance relies heavily on Manim's performance, which is documented by Manim's dev team as being slow and needing optimization. This can make Git-Sim quite slow when it has to draw a large number of commits, and especially when rendering this as video using the --animate flag. Currently it doesn't seem possible to leverage multithreading or multiprocessing with Manim, but I'd like to research that in more detail along with other creative ways to improve Git-Sim's performance.
  5. Building out test coverage: Tests... I won't lie, there are none! Because there are so many subcommands in this program that inherit code from a shared superclass, a robust testing framework will be VERY necessary to maintain this project as it grows.

As mentioned at the start of this article, we welcome anyone who wants to contribute to the Git-Sim codebase and help us work on these items!

You can also support Git-Sim by sponsoring our dev team on GitHub.

Contributing to Git-Sim

In the past 3 months we've gone from 1 contributor (little ol' me) to 7! That's a 7x in 3 months! As amazing as that is, I can only imagine how much we could accomplish if we had 10, 20, or gasp... do I dare say it... 30 contributors!

This project is still in its infancy, and now is a great time for new contributors to make a big impact. This applies to both the code itself and the direction of the project.

Even if you've never worked on open-source before, I encourage you to check out our contribution guidelines on GitHub and say hi on our new contributors GitHub discussion thread if you're interested and want to know the best way to help.

Supporting Git-Sim

If you'd like to support Git-Sim development work, sponsor us on GitHub or support us on Patreon!

Or check out our Git-Sim merch!

Totally top-secret, next-gen version of Git-Sim

Imagine being able to experience your Git repos in a completely new way - a much more intuitive and immersive way.

It's still a secret for now, but I've begun work on a new project that literally takes Git to a whole new dimension!

Sign up to be the first to try it out:

Contact me

Feel free to email me if you're interested in contributing to Git-Sim, or if you have any questions, comments, ideas, or complaints!

References

  1. Git-Sim, Official GitHub Page - https://github.com/initialcommit-com/git-sim
  2. Directed Acyclic Graph, Wikipedia - https://en.wikipedia.org/wiki/Directed_acyclic_graph
  3. Git-Dummy, Official GitHub Page - https://github.com/initialcommit-com/git-dummy
  4. iTerm2 - https://iterm2.com/
  5. Pydantic - https://docs.pydantic.dev/
  6. Python Black, The Uncompromising Code Formatter - https://github.com/psf/black
  7. Manim, performance issues - https://docs.manim.community/en/stable/contributing/performance.html
  8. Sponsor Git-Sim on GitHub - https://github.com/sponsors/initialcommit-com
  9. Become a patron on Patreon - https://www.patreon.com/user/membership?u=92322459
  10. Git-Sim Merch - https://initialcommit.com/store?type=merch