Copy Data From AWS S3 Bucket to DigitalOcean Spaces

Introduction
Create Your DigitalOcean Spaces
Create Your DigitalOcean Credentials
Add a Server to Snapshooter
Attache Both AWS S3 and DigitalOcean Bucket to Snapshooter
Create a Backup Job to Copy the Content of AWS S3 Bucket to DigitalOcean Spaces
Conclusion

Introduction

Amazon S3 and DigitalOcean Spaces are both cloud-based object storage service provided that provides object storage through a web service interface and API. It allows you to store any amount of data at any time.

In some cases, you will need to move or copy data from one storage provider to another. This is the place where SnapShooter comes into the picture.

SnapShooter provides an S3 Sync tool that allows you to copy, move or sync the data from one storage provider to another. Simply using the S3 Sync tool, you can backup or copy data from the AWS S3 bucket to DigitalOcean Spaces or any other S3 compatible storage. SnapShooter provides an easier way to schedule an S3 Sync service to sync data between multiple buckets and keep different buckets updated at a specific interval of time.

In this post, we will show you how to copy data from the AWS S3 bucket to DigitalOcean Spaces using SnapShooter.

Create Your DigitalOcean Spaces

Before starting, you will need to create a new space on the DigitalOcean to save your data copying from the AWS S3 bucket. Follow the below steps to create a new space on the DigitalOcean:

Step 1 - Sign in to your DigitalOcean account and click on the Spaces in the left pane.

Step 2 - Click on the Create => Space button. You should see the following screen:

Step 3 - Select your region, Restrict File Listing, Provide your unique space name and click on Create a Space button. Once the Space is created, you should see the following screen:

Create Your DigitalOcean Credentials

Next, you will need the correct access key and secret key to authenticate with DigitalOcean Spaces. Follow the below steps to create a DigitalOcean Credentials:

Step 1 - Log in to your DigitalOcean Spaces account. You should see the following screen:

Step 2 - Click on the Manage Keys button on the right-hand side of the Spaces screen.

Step 3 - Click on the Generate New Key next to Spaces access keys. Then, provide a name for your key and click the checkmark. A new Key and Secret will be generated as shown below:

Add a Server to Snapshooter

You will also need to add a server to the SnapShooter. The S3 Sync service uses a server to copy data from the AWS S3 bucket to DigitalOcean Spaces.

Follow the below steps to add a server to Snapshooter:

Step 1 - First, go to the SnapShooter website and register for their 14-day free trial.

Step 2 - On the SnapShooter dashboard, click on the Servers in the left sidebar. You should see the following screen:

Step 3 - Click on the Connect Server. You should see the following screen:

Step 4 - Copy and paste the code shown in the above image to your server:

curl -sSL "https://ingestor.snapshooter.com/api/server/add?token=eyJpdiI6Imp6Y3diYi91eXIxalk1czRrQlBHSkE9PSIsInZhbHVlIjoieEFhY0RnMWlHOUdubTgwbWQxOGVzUT09IiwibWFjIjoiMDBhODY3MTkwZWJhYzFiMTJjNzdlYzczNmEyNTJmYjk2NzQ5NzBmZmUwODgxOGI5MWJjNmRlYmM2OTQyYTlhNiIsInRhZyI6IiJ9&key=5895" | bash

Once your server is connected to the Snapshooter, you will get the following output:

Welcome to SnapShooter Server Setup
Environment Detected: OS: linux, OS Type: amd64
Installing SSH public key
Creating temp SSH key file
grep: /root/.ssh/authorized_keys: No such file or directory
cp: cannot stat '/root/.ssh/authorized_keys': No such file or directory
Creating authorized_keys backup /root/.ssh/authorized_keys.bak-1653474994
Key Installed /root/.ssh/authorized_keys
Removing temp SSH key file
Scanning SSH config
Checking 22
matched (22)
SSH test confirmed, Server Added to SnapShooter ...

Step 5 - Go back to your web browser, you should see your added server on the following screen:

Attache Both AWS S3 and DigitalOcean Bucket to Snapshooter

Next, you will need to attach both the AWS S3 bucket and DigitalOcean Spaces to the SnapShooter.

First, click here to attach the AWS S3 bucket to the SnapShooter You should see the following screen:

Choose a Friendly Name, Select AWS S3, select region, type your access key and secret access key, type your AWS S3 bucket name and press the Attach External Storage button.

Again click here to attach the DigitalOcean S3 Spaces to the SnapShooter You should see the following screen:

Choose a Friendly Name, Select DigitalOcean Spaces, select region, type your access key and secret access key, type your DigitalOcean Spaces name and press the Attach External Storage button.

Create a Backup Job to Copy the Content of AWS S3 Bucket to DigitalOcean Spaces

Now, you will need to create a backup job to copy data from the AWS S3 bucket to DigitalOcean Spaces at a specific interval of time. Follow the below steps to create a backup job:

Step 1 - On the SnapShooter dashboard, click on the Backup Jobs in the left pane. You should see the following screen:

Step 2 - Click on the Create New Jobs. You should see the following screen:

Step 3 - Click on the S3 Sync, you should see the following screen:

Step 4 - Provide your backup job name and click on the Continue button. You should see the following screen:

Step 5 - Select your server and click on the Next button. You should see the following screen:

Step 6 - Define the directory path in the destination bucket, select your source AWS S3 bucket and click on the Test button. You should see the following screen:

Step 7 - Click on the Save and Next button. You should see the following screen:

Step 8 - Select your destination DigitalOcean Spaces and click on the Set Storage button. You should see the following screen:

Step 9 - Define your backup schedule and click on the Set Schedule and Finish button. You should see the following screen: - Define your backup schedule and click on the

Step 10 - Click on the Backup Now button to start copying data from AWS S3 bucket to DigitalOcean Spaces. Once the backup process is completed successfully, you should see the following screen:

You can also verify your backup by accessing the DigitalOcean Spaces. You should see your all data in the following screen:

Conclusion

In this guide, you learned how to copy data from AWS S3 bucket to DigitalOcean Spaces using SnapShooter. SnapShooter offers a simple and user-friendly web interface to manage all backups and restore them easily in the event of data lose.

SnapShooter support many S3 provider on the marketplace, including AWS S3, DigitalOcean Spaces, Backblaze, Upcloud, Scaleway, Minio, and custom endpoints. If you have object storage missing let us know we are happy to add any. We believe everyone should keep a copy of there data offsite and secure.