3 Ways to Back Up and Restore Repositories in Bitbucket Cloud
Losing any amount of data can disrupt business in significant ways: it can erase history, lead to regulation violations, and even bankrupt a business. Wholly apart from server outages and natural disasters, it’s not uncommon for users to make mistakes while handling data. They could accidentally delete information from a repository, rogue employees could destroy data, accounts might get deleted, not to mention malicious hijackers could compromise your data.
With so many unforeseeable scenarios that could lead to data loss, this is why backups are a necessity. Repositories store information crucial to modern applications, such as code, documentation, and work history. Whichever repository platform you use, it’s vital to maintain regular backups.
Unfortunately, in the case of Bitbucket, your data is not easily accessible for restore. Although Bitbucket does an automatic backup of all repositories hosted, it does not provide restore access to users. Hence the need to back up repositories on your own without having to depend on Bitbucket support. Backing up your Bitbucket repository on your own gives you the flexibility to back up and restore your data at any time and in any way you want.
This guide will introduce you to three different tools useful for the backup and restore of your repositories in Bitbucket Cloud. You'll learn how to use them, as well as the pros and cons of each method.
Method 1: Using
git clone Command-line Script
Bitbucket does not provide a built-in backup feature. Currently, the simplest way to back up a repository in Bitbucket is to download it manually using the
git clone command. Every clone of a repository is a backup.
git clone command is a Git command primarily used to point to an existing repository, in this case your Bitbucket repository. It clones or makes a copy of your repository and downloads it to your computer. You will then have an exact local copy of your repository just like it is on the Bitbucket site.
To use this command, you must have Git installed on your terminal:
git clone <repository_url> <directory>
In the sample above,
<repository_url> represents the link to your Bitbucket repository. The
<directory> command is an optional argument that points to the path of the folder you want the repository downloaded to.
<directory> defaults to your current folder.
The advantage of using this backup method is that it’s simple to use and can be run from any computer with a terminal. There's also the flexibility of backing up on several computers and other cloud options.
Its major drawback is that it’s a very manual process. The repo owner is responsible for making sure that clones are made periodically so as to have up-to-date backups. Having to back up manually undermines best practices of backing up data, sometimes to the point of defeating the purpose. Also, the backups do not include the bells and whistles of a Bitbucket repository, such as pull requests, comments, and issues.
To learn more about the
git clone command, check out the git clone documentation.
bbbackup is a free open-source library used as a backup solution for cloud-to-local repository cloning. It’s written in Python, which means to use it you need to have Python installed on your machine.
To start using
bbbackup, you need to first clone the tool's repository to your local computer by running:
git clone firstname.lastname@example.org:OpenReplyDE/bbbackup.git
bbbackup requires the use of a config file called
bbbackup.cfg. This file will contain your Bitbucket credentials and other configurations.
The repository comes with a sample config file called
bbbackup_sample.cfg. Rename it to
bbbackup.cfg. Open the file and edit the authorization config as
auth_method = uidpwd, since you are going to be using the password method to authenticate your Bitbucket account.
[bitbucket_account] section, fill the
userid. This should be the normal Bitbucket username of the account containing your repositories. For the
password variable, you might need to create an app password in Bitbucket, if you don't already have one. To create an app password, check the Bitbucket documentation on creating app passwords. The
teamname variable refers to the user/organization name being backed up.
[backup] section, the
filepath variable represents the name of the folder you want your Bitbucket backups to be stored in. Make sure you include a full path to the folder.
To start using
bbbackup, create a virtual environment with Python 3:
python3 -m venv BitBucketBackup
Next, activate the virtual environment, upgrade
pip to its latest version, and install the Python libraries required:
pip install --upgrade pip
pip install -r requirements.txt
Now, test that the backup script is working by running:
python bbbackup.py --help
bbbackup script is going to back up the repositories by using SSH, make sure that you have set up an SSH key for your Bitbucket account before moving on.
To start the backup process, run:
python3 bbbackup.py --configuration bbbackup.cfg --notify --backup
The above command will back up all of your Bitbucket repositories to the folder you specified in the
filepath section of the config file. The backed-up repositories are stored in the backup folders in a plain state, so in this case restoring the repository to Bitbucket is as simple as running a
git push command.
bbbackup is a great tool for backing up your Bitbucket repository, a step up from simply using
git clone manually for every single repository in your Bitbucket account. With
bbbackup, you can back up all your repositories in one fell swoop.
Note that it supports automation only on Unix machines. This is because it includes a
bbbackup.sh script, which is prepared to be called by a cron job or macOS daemon. It also includes support for reporting backup progress via Slack.
A major con of using
bbbackup is that it does not provide an option to exclude certain repositories from the backups. Another disadvantage is that the backups do not include pull requests, comments, and issues.
Method 3: SCM Backup
SCM Backup is a free open-source backup tool used offline for backing up repositories. It supports backing up from several online hosting platforms, like Bitbucket and GitHub. It uses the hosting platform's API to list and fetch your repositories, then uses the version control system installed on your machine to clone the latest data of every repository into a local backup folder. It also supports hosting and backing up multiple users/teams per hosting platform.
SCM Backup is a framework dependent tool, requiring the use of .NET Core. It’s been written and tested only on the Windows platform for that reason. To demonstrate the use of SCM Backup, this guide will use Docker to avoid cross-platform errors.
SCM Backup requires a
settings.yml file that contains relevant information like your hosting platform, repository name, workspace ID, and authentication details. Create a folder called
scm_backup and in it create a
mkdir scm_backup && cd scm_backup
Add the following code to the
- title: backup_from_bitbucket
localFolder value should represent the name of a directory where you want all the backed-up repositories to be stored. But since we are using Docker, the backups will be stored inside a folder,
/scm-backup-out, in the Docker container.
title represents the title of your backup. It must be a unique name in the file because you can have several
sources with their own titles.
hoster represents the hosting platform name, in this case Bitbucket.
type refers to whether you are backing up on the account of a single user or an org. The values could be
name refers to the Bitbucket user or organization name being backed up.
authname refers to the normal Bitbucket username of the Bitbucket account. For the
password, you might need to create an app password in Bitbucket. To create an app password, check the Bitbucket documentation on creating an app password.
SCM Backup supports excluding repositories from the backup. To use that feature, add the following to your
repo2 represent your repository name. You can list as many repositories as you want for exclusion from backup.
Create a file named
Dockerfile and add the following code:
# install .NET core and SCM Backup
RUN apt-get update && apt-get -y install software-properties-common && \
add-apt-repository universe && apt-get update && \
apt-get -y install sudo wget unzip apt-transport-https git && \
wget -q https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb && \
dpkg -i packages-microsoft-prod.deb && rm packages-microsoft-prod.deb && \
apt-get update && \
apt-get -y install dotnet-sdk-3.1 && \
wget https://github.com/christianspecht/scm-backup/releases/download/1.4.0/scm-backup-1.4.0.ae7edc3.zip && \
# copy settings.yml from local filesystem to the container
COPY settings.yml .
# run the backup
ENTRYPOINT dotnet ScmBackup.dll
In the Dockerfile, an Ubuntu environment is created, the working directory is changed to
scm_backup, and .NET Core and SCM Backup tools are downloaded. Next, the
settings.yml config file is copied into the container directory, and the SCM Backup tool is run.
Make sure Docker is running, then build the Docker image using the
docker build command:
docker build -t scm-backup .
Before running the container, we need to create a folder outside the container where Docker will transfer the Bitbucket backups. You can name it
Now run the container:
docker run -t --mount src=`pwd`/bitbucket-backups,target=/scm-backup-out,type=bind scm-backup
bitbucket-backups folder created above, and you should see the
_config folder. It contains a backup of your current settings and the
backup_from_bitbucket folder, which contains all the backed-up repositories.
Restoring a Repository with SCM Backup
To restore a repository backup into a working directory, navigate into
bitbucket-backups and from there navigate into the directory of one of your backed-up repositories. In it you should see folder
repo and/or a folder
wiki, if the repository has a wiki.
Restore by running the code below, where
<repo_name> refers to the local path and name you want the repository restored to on your machine:
git clone repo </path/repo_name>
Now you should see your restored repository as a working directory in /
SCM Backup is a great option for backing up your Bitbucket repositories because it supports other hosting platforms and version control systems, not just Bitbucket and GitHub. It also supports excluding repositories you don’t want to back up. It is simple to use and easy to restore backed-up repositories.
A major con of using it, however, is that it’s another manual process and doesn’t support automatic backup. It also does not back up parts of a repository like pull requests, comments, and issues.
Since Bitbucket itself has not introduced a way to back up repositories, it’s important to be able to back up and restore your repository data on your own. In this guide, you learned three different methods for backing up a Bitbucket repository that don’t depend on the hosting platform.
All of these methods require that you write your own command-line script or manually run them. They’re not fully automated or monitored for daily backups. None of them back up to third-party locations by default (e.g. Amazon S3), so you'll have to write a script to move files. They also don’t pull added data that accompanies repositories, such as pull requests, issues, and comments.