Push multiple files under a single commit through GitHub API

Using Git Database API

Intro

Wanted to understand how to push multiple files under a single commit, using GitHub API. Some googling landed me on Git Database API Docs. Wasn’t very clear how it worked so I tried getting my hands dirty! Documenting it here for anyone who’s looking for it.

Disclosure:

1. This API needs atleast 1 commit to work. Empty repositories don’t work.

Git Database API

The Git Database API enables you to read and write raw Git objects to your Git database on GitHub and to list and update Git references (branch heads and tags).

The whole process is as follows: create blobs corresponding to files, create tree, create a commit for the changes and finally updating refs to reflect the commit.

Creating blobs

A Git blob (binary large object) is the object type used to store the contents of each file in a repository. The file’s SHA-1 hash is computed and stored in the blob object.

To create a blob we need to hit the following endpoint:

POST https://api.github.com/repos/{user}/{repo}/git/blobs

Let’s create a blob for a file called main.py with 1 line of python code:

{
    "content":"print(\"hello world !\")",
    "encoding": "utf-8"
}

We can create blobs for each file we want to upload. In the response, we get SHA of the blob which we need to send while creating the tree. Sample response:

{
    "sha": "638eff25696b982124deeb1f3dfcceabfdc81a93",
    "url": "https://api.github.com/repos/BRO3886/git-db-example/git/blobs/638eff25696b982124deeb1f3dfcceabfdc81a93"
}

Creating a tree

Now, we need to create a tree with the files we added. For that, first we need to get the SHA of the base tree. Then we need to create a tree for the files we need to add.

Get SHA for base_tree

The base tree can be thought of as the folder where you'd like to create your commit. If you want to create it at the root folder, you can make request to:

GET https://api.github.com/repos/{user}/{repo}/git/trees/{branch}

It returns a response like:

{
    "sha": "0d43a3b20104b4baa402c09a6c9c6c3298390e4a",
    "url": "{url of tree}",
    "tree": [
        {
            "path": "App",
            "mode": "040000",
            "type": "tree",
            "sha": "{sha}",
            "url": "{url of folder/tree}"
        },
        {
            "path": "README.md",
            "mode": "100644",
            "type": "blob",
            "sha": "{some SHA}",
            "size": 885,
            "url": "{some URL}"
        },
        ...
    ],
    "truncated": false
}

Here, we can extract the SHA from response.sha or SHA of the folder under response.tree[i].sha.

Create tree

Then, we need to hit the create tree api:

POST https://api.github.com/repos/{user}/{repo}/git/trees

With the following body:

{
    "tree":[
        {
            "path":"helloworld/main.py",
            "mode":"100644",
            "type":"blob",
            "sha":"638eff25696b982124deeb1f3dfcceabfdc81a93"
        },
        {
            "path":"helloworld/main2.py",
            "mode":"100644",
            "type":"blob",
            "sha":"638eff25696b982124deeb1f3dfcceabfdc81a93"
        }
        ...
    ],
    "base_tree":"3c408bafa55eda6b1c51de5df0fc36304f37414c"
}

Here request.tree would have an array of blobs that you want to push. For each blob we need the SHA that we got as a response from create blob API.

For the mode:
The file mode; one of 100644 for file (blob), 100755 for executable (blob), 040000 for subdirectory (tree), 160000 for submodule (commit), or 120000 for a blob that specifies the path of a symlink.

GitHub doesn’t store folders, so the only way to make a folder is to set the path for the blob as {folderName}/{fileName}

Here, we get another SHA in the response, which we need when we create a commit corresponding to the changes:

{
    "sha": "a69117177bb067933189072b2b8799c63f388f32",
    "url": "https://api.github.com/repos/BRO3886/git-db-example/git/trees/a69117177bb067933189072b2b8799c63f388f32",
    "tree": [
        {
            "path": "README.md",
            "mode": "100644",
            "type": "blob",
            "sha": "bc7b1321063b4075c97bf16e6f8130b6f9fa6537",
            "size": 54,
            "url": "https://api.github.com/repos/BRO3886/git-db-example/git/blobs/bc7b1321063b4075c97bf16e6f8130b6f9fa6537"
        },
        {
            "path": "helloworld",
            "mode": "040000",
            "type": "tree",
            "sha": "82a82f6788b44fe93774597ff2e76ac66ae1e657",
            "url": "https://api.github.com/repos/BRO3886/git-db-example/git/trees/82a82f6788b44fe93774597ff2e76ac66ae1e657"
        }
    ],
    "truncated": false
}

Here we need response.sha for the next step.

Add a commit

This step is pretty simple. We just need the response.sha from the previous step. We make a request to the following endpoint:

POST https://api.github.com/repos/{user}/{repo}/git/commits

Along with the body:

{
    "tree":"a69117177bb067933189072b2b8799c63f388f32",
    "message":"some commit msg",
    "parents": ["3c408bafa55eda6b1c51de5df0fc36304f37414c"]
}

parents: The SHAs of the commits that were the parents of this commit. If omitted or empty, the commit will be written as a root commit. For a single parent, an array of one SHA should be provided; for a merge commit, an array of more than one should be provided.

In our case, for the parent, since we want to add another commit on a particular branch, we need to get the SHA for it.

For that, we need to make a request to the following endpoint:

GET https://api.github.com/repos/BRO3886/git-db-example/git/refs/heads/{branch}

It returns a JSON with details about the branch:

{
    "ref": "refs/heads/main",
    "node_id": "REF_kwDOG87gc69yZWZzL2hlYWRzL21haW4",
    "url": "https://api.github.com/repos/BRO3886/git-db-example/git/refs/heads/main",
    "object": {
        "sha": "3c408bafa55eda6b1c51de5df0fc36304f37414c",
        "type": "commit",
        "url": "https://api.github.com/repos/BRO3886/git-db-example/git/commits/3c408bafa55eda6b1c51de5df0fc36304f37414c"
    }
}

For the parent SHA, we need response.object.sha.

In the response of create commit API, we’ll get another SHA:

{
    "sha": "544aa83c4d4a784c4c8490d6548c248b0e57d0ac",
    "node_id": "C_kwDOG87gc9oAKDU0NGFhODNjNGQ0YTc4NGM0Yzg0OTBkNjU0OGMyNDhiMGU1N2QwYWM",
    "url": "https://api.github.com/repos/BRO3886/git-db-example/git/commits/544aa83c4d4a784c4c8490d6548c248b0e57d0ac",
    "html_url": "https://github.com/BRO3886/git-db-example/commit/544aa83c4d4a784c4c8490d6548c248b0e57d0ac",
...
}

We’ll need this SHA value for the last step.

Updating ref

This step is for updating the ref from where we pulled the SHA under parents (create commit step). For my case, since the ref was main, I'll update it. Make a request to the following endpoint:

PATCH https://api.github.com/repos/{user}/{repo}/git/refs/heads/{branch}

With the following body:

{
    "sha":"544aa83c4d4a784c4c8490d6548c248b0e57d0ac"
}

If in the response the object.sha is same as the one sent as a part of the request, along with status code 200, that means your changes will be reflected on GitHub.

I hope you got some idea. Here is the GitHub link of the sample repo you can refer: github.com/BRO3886/git-db-example

You can always reach out to me via https://sidv.dev/contact

Please do 👏 if the blog was helpful.