How to Backup File to DigitalOcean Spaces
DigitalOcean Spaces is an S3-compatible object storage service that allows you to store and serve a large amount of data. It has a built-in content delivery network (CDN) that minimizes page load times, improves performance, and reduces bandwidth and infrastructure costs. The base rate of a DigitalOcean Spaces subscription is $5.00 per month and gives you the ability to create multiple Spaces.
SnapShooter provides a cost-effective backup solution that makes it easy to schedule frequent backups for your servers, websites, and databases. SnapShooter’s advanced retention policies allow you to keep as many daily, weekly or monthly backups as needed. Only pay for what you store.
This post will show you how to backup files to DigitalOcean Spaces manually and using SnapShooter.
Create DigitalOcean Spaces
Before starting, you must have a bucket on the DigitalOcean spaces. If you don't have any bucket, follow the below steps to create a new space on the DigitalOcean:
Step 1 - Sign in to your DigitalOcean account and click on the Spaces in the left pane.
Step 2 - Click on the Create => Spaces from the right side of the DigitalOcean control panel.
Step 3 - Select your region, Restrict File Listing, Provide your unique space name and click on Create a Space button. Once the Space is created, you should see the following screen:
Create DigitalOcean Credentials
After creating DigitalOcean spaces, you will require the correct access key and secret key to authenticate with DigitalOcean spaces. Follow the below steps to create a DigitalOcean Credentials:
Step 1 - Log in to your DigitalOcean Spaces account.
Step 2 - Click on the Manage Keys button on the right-hand side of the Spaces screen.
Step 3 - Click on the Generate New Key next to Spaces access keys. You should see the following screen:
Step 4 - Provide a name for your key and click the checkmark. A new Key and Secret will be generated as shown below:
Backup File to DigitalOcean Spaces Manually
In this section, we will show you how to back up files from your server to the DigitalOcean Spaces using the S3cmd tool.
S3cmd is a free and open-source command-line tool that allows you to upload, download and manage data in DigitalOcean Spaces and other cloud storage service providers.
You will need to install the S3cmd tool on the server from where you want to back up your files. Follow the below steps to install the S3cmd to your server.
First, install the Python and other dependencies by running the following command:
apt-get install python3 python3-setuptools curl -y
Next, download the latest version of S3cmd using the following command:
curl -LO https://github.com/s3tools/s3cmd/releases/download/v2.2.0/s3cmd-2.2.0.tar.gz
Next, extract the downloaded file with the following command:
tar -xvzf s3cmd-2.2.0.tar.gz
Next, navigate to the extracted directory and install it using the following command:
cd s3cmd-2.2.0python3 setup.py install
Once the S3cmd is installed, verify the S3cmd version using the following command:
You will get the following output:
s3cmd version 2.2.0
Next, you will need to configure the S3cmd using the DigitalOcean Access key and Secret key. You can configure it using the following command:
You will be asked to provide your DigitalOcean Access Key, Secret Key, Region, and Endpoint as shown below:
Enter new values or accept defaults in brackets with Enter.Refer to user manual for detailed description of all options.Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.Access Key: 7T4U6JSUYPGNT2M5ILYSSecret Key: VoYYWhxmpTzDNDxLd7GbQ6SOGceCjSYoUdUHRYuQ/9UDefault Region [US]: Frankfurt 1Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.S3 Endpoint [s3.amazonaws.com]: fra1.digitaloceanspaces.comUse "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be usedif the target S3 system supports dns based buckets.DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: %(bucket)s.fra1.digitaloceanspaces.comEncryption password is used to protect your files from readingby unauthorized persons while in transfer to S3Encryption password: jethvaPath to GPG program:When using secure HTTPS protocol all communication with Amazon S3servers is protected from 3rd party eavesdropping. This method isslower than plain HTTP, and can only be proxied with Python 2.7 or newerUse HTTPS protocol [Yes]:On some networks all internet access must go through a HTTP proxy.Try setting it here if you can't connect to S3 directlyHTTP Proxy server name:New settings:Access Key: 7T4U6JSUYPGNT2M5ILYSSecret Key: VoYYWhxmpTzDNDxLd7GbQ6SOGceCjSYoUdUHRYuQ/9UDefault Region: Frankfurt 1S3 Endpoint: fra1.digitaloceanspaces.comDNS-style bucket+hostname:port template for accessing a bucket: %(bucket)s.fra1.digitaloceanspaces.comEncryption password: jethvaPath to GPG program: NoneUse HTTPS protocol: TrueHTTP Proxy server name:HTTP Proxy server port: 0Test access with supplied credentials? [Y/n] YPlease wait, attempting to list all buckets...Success. Your access key and secret key worked fine :-)Now verifying that encryption works...Not configured. Never mind.Save settings? [y/N] yConfiguration saved to '/root/.s3cfg'
Once the S3cmd is configured, you can verify your bucket information using the following command:
s3cmd info s3://snapshooter-file-backup/
You will get the following output:
s3://snapshooter-file-backup/ (bucket):Location: fra1Payer: noneExpiration Rule: nonePolicy: noneCORS: noneACL: 6945323: FULL_CONTROL
Backup Files to DigitalOcean Spaces Using S3cmd
After installing and configuring S3cmd, you can able to copy files or directories using S3cmd.
To copy a file named file.img to the DigitalOcean Spaces, run the following command:
s3cmd put file.img s3://snapshooter-file-backup/
To copy the directory named /var/log, compress it and copy it to the DigitalOcean Spaces, run the following command:
tar -czvf - /var/log | s3cmd put - s3://snapshooter-file-backup/log_backup.tar.gz
Next, verify the file and directory that you have copied using the following command:
s3cmd ls s3://snapshooter-file-backup/
You should see them in the following output:
2022-01-19 14:13 104857600 s3://snapshooter-file-backup/file.img2022-01-19 14:13 242781 s3://snapshooter-file-backup/log_backup.tar.gz
Backup File to DigitalOcean Spaces Using SnapShooter
Backing up a file and directory manually to DigitalOcean Spaces is very difficult for any beginner user. It is also difficult to manage and control all backups. In this case, SnapShooter provides an easier way to create a backup, compress it and schedule it automatically for you. SnapShooter also provides a web-based admin panel to manage all your backup jobs from the web browser.
Follow the below steps to Backup File to DigitalOcean Spaces:
Step 1 - First, go to the SnapShooter website and register for their 14-day free trial.
Step 2 - On the SnapShooter dashboard, click on the Backup Jobs in the left sidebar. You should see the following screen:
Step 3 - Click on the Create New Job. You should see the various database option on the following screen:
Step 4 - Click on the Server File. You should see the following screen:
Step 5 - Define your backup name and click on the Continue button. You should see the Add Your Server screen:
Step 6 - Now, copy the code shown in the above screen and paste it to your server as shown below:
curl -sSL "https://ingestor.snapshooter.com/api/server/add?token=eyJpdiI6IlppcWdyRTd4RW9PdExmN3oxTG1DUEE9PSIsInZhbHVlIjoiM0E1R2N0WWFjV1JCQm9uSWJwakxnQT09IiwibWFjIjoiMzQ2ZDdlNzQwZTFkZTBhMzNkNWQzNWYxYzIxNTFmYzI5NDM2MjE1Yjk3YTkzNzQ1NzFmODc4MTE3NzYxNDU5NSIsInRhZyI6IiJ9&key=5895" | bash
You should see the following output:
Welcome to SnapShooter Server SetupEnvironment Detected: OS: linux, OS Type: amd64Installing SSH public keyCreating temp SSH key fileThis SSH key is already installed on this serverRemoving temp SSH key fileScanning SSH configChecking 22matched (22)SSH test confirmed, Server Added to SnapShooter ...
Step 7 - Once your server is added to the SnapShooter, you should see the following screen:
Step 8 - Click on the Next button. You should see the file backup configuration screen:
Step 9 - Define the directory path that you want to backup, compression level, and click on the Test button to test the connection. If everything is fine, you should see the following screen:
Step 10 - Click on the Save and Next button. You should see the Storage Selection screen:
Step 11 - Here, you will need to define your DigitalOcean Spaces to store the file. Click on the Add New Storage Provider. You should see the following screen:
Step 12 - Click on the Config S3 button. You should see the following screen:
Step 13 - Provide your friendly name, select your storage provider, define your region, provide your DigitalOcean Space name, Access Key, and Secret Key, and click the Test and Save button. Once your DigitalOcean Spaces is connected to the SnapShooter, you should see the following screen:
Step 14 - Click on the Set Storage button to set your DigitalOcean Spaces as default storage. You should see the Backup Schedule Setup screen:
Step 15 - Select and define all options as per your requirements and click on the Set Schedule and Finish button. You should see the following screen:
Step 16 - Click on the Backup Now button to run your first backup job. Once your backup job is executed successfully, you should see the following screen:
Now, go back to your DigitalOcean Spaces and verify your backup as shown below:
In this guide, you learned how to backup files to DigitalOcean Spaces using SnapShooter and manually. You can now manage and control all your backup from the centralized location.
Thank you for helping us improve!