How to Backup MongoDB Database to Backblaze with bash
Backblaze is an online cloud-based object storage service that allows you to store your data in a safe and secure location. It offers simple, affordable, and predictable pricing. Storage is $0.005/GB/Month, with download costs of $0.01/GB.
Using SnapShooter, you can automate MongoDB database backup to Backblaze Bucket easily. SnapShooter provides a simple and user-friendly control panel that helps beginners to control and manage their backups. SnapShooter will work with MongoDB hosted on your own droplets as well as MongoDB-managed databases.
This post will show you how to backup MongoDB to Backblaze Bucket manually and using SnapShooter.
Create your Backblaze Bucket
First, you will need to create a new bucket on the Backblaze website. Follow the below steps to create a new bucket on the Backblaze:
Step 1 - Sign in to your Backblaze account and click on the Bucket in the left pane to create a new Bucket.
Step 2 - Click on the Create a Bucket button. You should see the following page:
Step 3 - Provide all required information and click on the Create a Bucket button. Once the Bucket is created, you should see the following page:
Note down the Backblaze Bucket URL from the above image. You will need it later in this article.
Create Your Backblaze Credentials
In order to create backups with Backblaze, you will need to create the credentials to access it.
Step 1 - On the Backblaze dashboard, click on the "App Keys" in the left pane as shown below:
Step 2 - Now, click on "Add a new Application Key" button. You should see the following page:
Step 3 - Provide all required details and click on the Create New Key button. You should see the following page:
Note down the keyID, and applicationKey from the above image. You will need both to integrate Backblaze with the application.
Backup MongoDB to Backblaze Bucket Manually
In this section, we will show you how to back up a MongoDB database from your server to the Backblaze Bucket using the S3cmd tool.
Before starting, you will need to install the S3cmd tool on the server from where you want to back up your MongoDB database.
S3cmd is a free and open-source command-line tool that allows you to upload, download and manage data in Backblaze Bucket and other cloud storage service providers. Follow the below steps to install the S3cmd to your server.
First, install the Python and other dependencies by running the following command:
apt-get install python3 python3-setuptools curl -y
Next, download the latest version of S3cmd using the following command:
curl -LO https://github.com/s3tools/s3cmd/releases/download/v2.2.0/s3cmd-2.2.0.tar.gz
Next, extract the downloaded file with the following command:
tar -xvzf s3cmd-2.2.0.tar.gz
Next, navigate to the extracted directory and install it using the following command:
cd s3cmd-2.2.0python3 setup.py install
Once the S3cmd is installed, verify the S3cmd version using the following command:
You will get the following output:
s3cmd version 2.2.0
Configure S3cmd to Connect Backblaze Bucket
Next, you will need to configure the S3cmd using the Backblaze Access key and Secret key. You can configure it using the following command:
You will be asked to provide your Backblaze Access Key, Secret Key, Region, and Endpoint as shown below:
Enter new values or accept defaults in brackets with Enter.Refer to user manual for detailed description of all options.Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.Access Key: 004d55130756c060000000003Secret Key: K0042u1+4azolEAs5UuvLJN9dBSF1XEDefault Region [US]: us-west-004Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.S3 Endpoint [s3.amazonaws.com]: s3.us-west-004.backblazeb2.comUse "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be usedif the target S3 system supports dns based buckets.DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: %(bucket)s.s3.us-west-004.backblazeb2.comEncryption password is used to protect your files from readingby unauthorized persons while in transfer to S3Encryption password: asdPath to GPG program [/usr/bin/gpg]:When using secure HTTPS protocol all communication with Amazon S3servers is protected from 3rd party eavesdropping. This method isslower than plain HTTP, and can only be proxied with Python 2.7 or newerUse HTTPS protocol [Yes]:On some networks all internet access must go through a HTTP proxy.Try setting it here if you can't connect to S3 directlyHTTP Proxy server name:New settings:Access Key: 004d55130756c060000000003Secret Key: K0042u1+4azolEAs5UuvLJN9dBSF1XEDefault Region: us-west-004S3 Endpoint: s3.us-west-004.backblazeb2.comDNS-style bucket+hostname:port template for accessing a bucket: %(bucket)s.s3.us-west-004.backblazeb2.comEncryption password: asdPath to GPG program: /usr/bin/gpgUse HTTPS protocol: TrueHTTP Proxy server name:HTTP Proxy server port: 0Test access with supplied credentials? [Y/n] YPlease wait, attempting to list all buckets...Success. Your access key and secret key worked fine :-)Now verifying that encryption works...Success. Encryption and decryption worked fine :-)Save settings? [y/N] YConfiguration saved to '/root/.s3cfg'
Once the S3cmd is configured, you can verify your bucket information using the following command:
s3cmd info s3://snapshooter-mongo-backup/
You will get the following output:
s3://snapshooter-mongo-backup/ (bucket):Location: us-west-004Payer: noneExpiration Rule: nonePolicy: noneCORS: noneACL: d55130756c06: FULL_CONTROL
Use S3cmd to Backup MongoDB to Backblaze Bucket
At this point, S3cmd is installed and configured to manage the Backblaze.
First, go to the server from where you want to back up your MongoDB database, then log in to the MongoDB using the following command:
Once you are logged in, list all MongoDB databases using the following command:
> show dbs
You will get all databases in the following output:
admin 0.000GBconfig 0.000GBlocal 0.000GBtestdb 0.000GB
Now, exit from the MongoDB shell using the following command:
Now, backup all MongoDB databases to the /mnt directory using the following command:
mongodump --out /mnt/
2022-03-28T16:00:29.185+0000 writing admin.system.version to /mnt/admin/system.version.bson2022-03-28T16:00:29.186+0000 done dumping admin.system.version (1 document)2022-03-28T16:00:29.187+0000 writing admin.movie to /mnt/admin/movie.bson2022-03-28T16:00:29.189+0000 done dumping admin.movie (2 documents)2022-03-28T16:00:29.189+0000 writing testdb.movie to /mnt/testdb/movie.bson2022-03-28T16:00:29.189+0000 writing testdb.inventory to /mnt/testdb/inventory.bson2022-03-28T16:00:29.190+0000 done dumping testdb.inventory (1 document)2022-03-28T16:00:29.195+0000 done dumping testdb.movie (2 documents)
Next, run the S3cmd command to copy the testdb database directory to the Backblaze Bucket:
tar -czvf - /mnt/testdb | s3cmd put - s3://snapshooter-mongo-backup/testdb.tar.gz
/mnt/testdb//mnt/testdb/movie.bson/mnt/testdb/inventory.metadata.json/mnt/testdb/movie.metadata.json/mnt/testdb/inventory.bsonupload: '<stdin>' -> 's3://snapshooter-mongo-backup/testdb.tar.gz' [part 1 of -, 542B] [1 of 1]542 of 542 100% in 0s 4.44 KB/s done
Next, run the S3cmd command to copy the admin database directory to the Backblaze Bucket:
tar -czvf - /mnt/admin | s3cmd put - s3://snapshooter-mongo-backup/admin.tar.gz
/mnt/admin//mnt/admin/system.version.bson/mnt/admin/movie.bson/mnt/admin/movie.metadata.json/mnt/admin/system.version.metadata.jsonupload: '<stdin>' -> 's3://snapshooter-mongo-backup/admin.tar.gz' [part 1 of -, 506B] [1 of 1]506 of 506 100% in 0s 9.57 KB/s done
You can now verify all database backups on the Backblaze Bucket using the following command:
s3cmd ls s3://snapshooter-mongo-backup/
You will get the following output:
2022-03-28 16:14 506 s3://snapshooter-mongo-backup/admin.tar.gz2022-03-28 16:14 542 s3://snapshooter-mongo-backup/testdb.tar.gz
You can also log in to the Backblaze Bucket and verify your backup files as shown below:
Thank you for helping us improve!