How to Backup MongoDB Database to DigitalOcean Spaces with Bash

Introduction

Setting up database backup is one of the most essential things for any system or database administrator. If you have your own dedicated server then you’ll need to set up backups yourself. Regularly backing up a database is a very hard and time-consuming process. In this case, you will need to think about a headache-free backup strategy or a backup tool that allows you to automatic backups from their control panel. This is where SnapShooter comes into the picture. SnapShooter provides a web-based backup solution that provides a user-friendly control panel to manage and automate the MongoDB database via the web browser. SnapShooter works with a MongoDB server hosted on your own server as well as MongoDB managed in a DigitalOcean Managed databases.

This post will show you how to backup MongoDB databases to DigitalOcean Spaces manually and using SnapShooter.

Create Your DigitalOcean Spaces

Before starting, you will need to create a new space on the DigitalOcean. Follow the below steps to create a new space on the DigitalOcean:

Step 1 - Sign in to your DigitalOcean account and click on the Spaces in the left pane.

Step 2 - Click on the Create a Space button. You should see the following screen:

Step 3 - Select your region, Restrict File Listing, Provide your unique space name and click on Create a Space button. Once the Space is created, you should see the following screen:

Create Your DigitalOcean Credentials

In order to create backups with DigitalOcean spaces, you will need the correct access key and secret key to authenticate with it. Follow the below steps to create a DigitalOcean Credentials:

Step 1 - Log in to your DigitalOcean Spaces account or go directly to https://cloud.digitalocean.com/spaces

Step 2 - Click on the Manage Keys button on the right-hand side of the Spaces screen.

Step 3 - Click on the Generate New Key next to Spaces access keys. You should see the following screen:

Step 4 - Provide a name for your key and click the checkmark. A new Key and Secret will be generated as shown below:

Backup MongoDB Database to DigitalOcean Space Manually

In this section, we will show you how to back up a MongoDB database from your server to the DigitalOcean Space using the S3cmd tool.

Install S3cmd

Before starting, you will need to install the S3cmd tool on the server from where you want to back up your MongoDB database.

S3cmd is a free and open-source command-line tool that allows you to upload, download and manage data from your server to DigitalOcean Space and other cloud storage service providers. Follow the below steps to install the S3cmd to your server.

First, install the Python and other dependencies by running the following command:

apt-get install python3 python3-setuptools curl -y

Next, download the latest version of S3cmd using the following command:

curl -LO https://github.com/s3tools/s3cmd/releases/download/v2.2.0/s3cmd-2.2.0.tar.gz

Next, extract the downloaded file with the following command:

tar -xvzf s3cmd-2.2.0.tar.gz

Next, navigate to the extracted directory and install it using the following command:

cd s3cmd-2.2.0
python3 setup.py install

Once the S3cmd is installed, verify the S3cmd version using the following command:

s3cmd --version

You will get the following output:

s3cmd version 2.2.0

Configure S3cmd

Next, you will need to configure the S3cmd using the DigitalOcean Access key and Secret key. You can configure it using the following command:

s3cmd --configure

You will be asked to provide your DigitalOcean Access Key, Secret Key, Region, and Endpoint as shown below:

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.
 
Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key: Your-Access-Key
Secret Key: Your-Secret-Key
Default Region [US]: Frankfurt 1
 
Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.
S3 Endpoint [s3.amazonaws.com]: fra1.digitaloceanspaces.com
 
Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used
if the target S3 system supports dns based buckets.
DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: %(bucket)s.fra1.digitaloceanspaces.com
 
Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password: jethva
Path to GPG program:
 
When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [Yes]:
 
On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't connect to S3 directly
HTTP Proxy server name:
 
New settings:
  Access Key: AQL6SGGWNDUTTLFW2B6I
  Secret Key: zy15cA7jTET5cHCFr4BnT72N5MtcXAeo/BpoM77OjNg
  Default Region: Frankfurt 1
  S3 Endpoint: fra1.digitaloceanspaces.com
  DNS-style bucket+hostname:port template for accessing a bucket: %(bucket)s.fra1.digitaloceanspaces.com
  Encryption password: jethva
  Path to GPG program: None
  Use HTTPS protocol: True
  HTTP Proxy server name:
  HTTP Proxy server port: 0
 
Test access with supplied credentials? [Y/n] Y
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)
 
Now verifying that encryption works...
Not configured. Never mind.
 
Save settings? [y/N] y
Configuration saved to '/root/.s3cfg'

Once the S3cmd is configured, you can verify your bucket information using the following command:

s3cmd info s3://mongodb-dump/

You will get the following output:

s3://mongodb-dump/ (bucket):
   Location:  fra1
   Payer:     none
   Expiration Rule: none
   Policy:    none
   CORS:      none
   ACL:       6945323: FULL_CONTROL

Use S3cmd to Backup MongoDB Database to DigitalOcean Spaces

First, log in to your server and connect to the MongoDB shell with the following command:

mongo

Once you are log in, list all available databases using the following command:

show dbs

You will get the following output:

admin   0.000GB
config  0.000GB
local   0.000GB
testdb  0.000GB

Next, backup the testdb database inside the /opt directory using the following command:

mongodump --db testdb --out /opt/

You will get the following output:

2022-03-03T05:46:37.287+0000    writing testdb.user to
2022-03-03T05:46:37.288+0000    done dumping testdb.user (1 document)

To backup all MongoDB databases, run the following command:

mongodump --out /opt

You will get the following output:

2022-03-03T05:46:51.920+0000    writing admin.system.version to
2022-03-03T05:46:51.921+0000    done dumping admin.system.version (1 document)
2022-03-03T05:46:51.921+0000    writing testdb.user to
2022-03-03T05:46:51.922+0000    done dumping testdb.user (1 document)

Next, verify all backup databases using the following command:

ls /opt

You will get the following output:

admin  testdb

Next, run the S3cmd command to copy the testdb database directory to the DigitalOcean Spaces:

tar -czvf - /opt/testdb | s3cmd put - s3://mongodb-dump/testdb.tar.gz

Next, run the S3cmd command to copy the admin directory for all databases to the DigitalOcean Spaces:

tar -czvf - /opt/admin | s3cmd put - s3://mongodb-dump/admin.tar.gz

You can now verify all database backups on the DigitalOcean Spaces using the following command:

s3cmd ls s3://mongodb-dump

You will get the following output:

2022-03-03 05:51          339  s3://mongodb-dump/admin.tar.gz
2022-03-03 05:50          347  s3://mongodb-dump/testdb.tar.gz

You can also log in to the DigitalOcean Spaces and verify your backup files as shown below: