How to Install and Use Rclone on Ubuntu 20.04

Rclone is a command-line software tool to copy, sync, manage and migrate files and directories to a remote server, cloud or other storage. You can install it on Linux, Windows, macOS, Debian, OpenBSD, and several other operating systems. Currently, it supports over 40+ cloud storage, including AWS, Google Cloud, Google Drive, Dropbox, pCloud, Zoho, Mega and more. It also supports SSH and SFTP protocols to sync files between Linux servers.

Rclone employs server-side transfers to minimize local bandwidth usage and transfers from one provider to another without using the local disk. In addition, it allows you to mount any local or cloud filesystem as a disk on Linux and Windows.

This guide will show you how to copy and sync files and directories to a remote Linux server using Rclone over the SSH protocol.

Prerequisites
Install Rclone
Configure Rclone
How to Use Rclone
Conclusion

Prerequisites

Two servers running Ubuntu 20.04 server. one for Rclone and one for remote SSH.
A root password is configured on both servers.

Install Rclone

By default, Rclone is included in the Ubuntu default repository. You can install it using the following command:

apt install rclone -y

After installing Rclone, verify the Rclone version with the following command:

rclone --version

Sample output:

rclone v1.50.2 - os/arch: linux/amd64 - go version: go1.13.6

Run the following command to see all options available with Rclone:

rclone --help

Sample output:

Available Commands: about Get quota information from the remote. authorize Remote authorization. cachestats Print cache stats for a remote cat Concatenates any files and sends them to stdout. check Checks the files in the source and destination match. cleanup Clean up the remote if possible config Enter an interactive configuration session. copy Copy files from source to dest, skipping already copied copyto Copy files from source to dest, skipping already copied copyurl Copy url content to dest. cryptcheck Cryptcheck checks the integrity of a crypted remote. cryptdecode Cryptdecode returns unencrypted file names. dbhashsum Produces a Dropbox hash file for all the objects in the path. dedupe Interactively find duplicate files and delete/rename them. delete Remove the contents of path. deletefile Remove a single file from remote. genautocomplete Output completion script for a given shell. gendocs Output markdown docs for rclone to the directory supplied. hashsum Produces an hashsum file for all the objects in the path. help Show help for rclone commands, flags and backends. link Generate public link to file/folder. listremotes List all the remotes in the config file. ls List the objects in the path with size and path. lsd List all directories/containers/buckets in the path. lsf List directories and objects in remote:path formatted for parsing lsjson List directories and objects in the path in JSON format. lsl List the objects in path with modification time, size and path. md5sum Produces an md5sum file for all the objects in the path. mkdir Make the path if it doesn't already exist. mount Mount the remote as file system on a mountpoint. move Move files from source to dest. moveto Move file or directory from source to dest. ncdu Explore a remote with a text based user interface. obscure Obscure password for use in the rclone.conf purge Remove the path and all of its contents. rc Run a command against a running rclone. rcat Copies standard input to file on remote. rcd Run rclone listening to remote control commands only. rmdir Remove the path if empty. rmdirs Remove empty directories under the path. serve Serve a remote over a protocol. settier Changes storage class/tier of objects in remote. sha1sum Produces an sha1sum file for all the objects in the path. size Prints the total size and number of objects in remote:path. sync Make source and dest identical, modifying destination only. touch Create new file or change file modification time. tree List the contents of the remote in a tree like fashion. version Show the version number.

Configure Rclone

First, you will need to configure Rclone to define a storage provider or remote SSH server. In this section, we will configure Rclone to use a remote SSH server as a backup server.

Run the following command to start the rclone configuration.

rclone config

You should see the following options:

No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n

Type n and hit Enter to create a new remote server.

name> remote-ssh

Type your remote backup server name and hit Enter. You will be asked to choose from the list of storage providers:

Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value 1 / 1Fichier \ "fichier" 2 / Alias for an existing remote \ "alias" 3 / Amazon Drive \ "amazon cloud drive" 4 / Amazon S3 Compliant Storage Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, etc) \ "s3" 5 / Backblaze B2 \ "b2" 6 / Box \ "box" 7 / Cache a remote \ "cache" 8 / Citrix Sharefile \ "sharefile" 9 / Dropbox \ "dropbox" 10 / Encrypt/Decrypt a remote \ "crypt" 11 / FTP Connection \ "ftp" 12 / Google Cloud Storage (this is not Google Drive) \ "google cloud storage" 13 / Google Drive \ "drive" 14 / Google Photos \ "google photos" 15 / Hubic \ "hubic" 16 / JottaCloud \ "jottacloud" 17 / Koofr \ "koofr" 18 / Local Disk \ "local" 19 / Mail.ru Cloud \ "mailru" 20 / Microsoft Azure Blob Storage \ "azureblob" 21 / Microsoft OneDrive \ "onedrive" 22 / OpenDrive \ "opendrive" 23 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH) \ "swift" 24 / Pcloud \ "pcloud" 25 / Put.io \ "putio" 26 / SSH/SFTP Connection \ "sftp" 27 / Transparently chunk/split large files \ "chunker" 28 / Union merges the contents of several remotes \ "union" 29 / Webdav \ "webdav" 30 / Yandex Disk \ "yandex" 31 / http Connection \ "http" 32 / premiumize.me \ "premiumizeme" Storage> 26

Type 26 and hit Enter for the remote SSH server. You should see the following output:

SSH host to connect to Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value 1 / Connect to example.com \ "example.com" host> remote-ssh-server-ip SSH username, leave blank for current username, root Enter a string value. Press Enter for the default (""). user> root SSH port, leave blank to use default (22) Enter a string value. Press Enter for the default (""). port> 22 SSH password, leave blank to use ssh-agent. y) Yes type in my own password g) Generate random password n) No leave this optional password blank y/g/n> y Enter the password: password: Confirm the password: password:

Type the IP address, port number, username and password of the remote SSH server. You should see the following options:

Path to PEM-encoded private key file, leave blank or set key-use-agent to use ssh-agent. Enter a string value. Press Enter for the default (""). key_file>

Just press Enter to the default:

The passphrase to decrypt the PEM-encoded private key file. Only PEM encrypted key files (old OpenSSH format) are supported. Encrypted keys in the new OpenSSH format can't be used. y) Yes type in my own password g) Generate random password n) No leave this optional password blank y/g/n> y

Type y and hit Enter. You will be asked to set the SSH passphrase.

Enter the password: password: Confirm the password: password:

Define your password and hit Enter:

When set forces the usage of the ssh-agent. When key-file is also set, the ".pub" file of the specified key-file is read and only the associated key is requested from the ssh-agent. This allows to avoid `Too many authentication failures for *username*` errors when the ssh-agent contains many keys. Enter a boolean value (true or false). Press Enter for the default ("false"). key_use_agent>

Press Enter for the default:

Enable the use of insecure ciphers and key exchange methods. Those algorithms are insecure and may allow plaintext data to be recovered by an attacker. Enter a boolean value (true or false). Press Enter for the default ("false"). Choose a number from below, or type in your own value 1 / Use default Cipher list. \ "false" 2 / Enables the use of the aes128-cbc cipher and diffie-hellman-group-exchange-sha256, diffie-hellman-group-exchange-sha1 key exchange. \ "true" use_insecure_cipher> 1

Type 1 and hit Enter:

Disable the execution of SSH commands to determine if remote file hashing is available. Leave blank or set to false to enable hashing (recommended), set to true to disable hashing. Enter a boolean value (true or false). Press Enter for the default ("false"). disable_hashcheck> Edit advanced config? (y/n) y) Yes n) No y/n> n

Type n and hit Enter:

Remote config -------------------- [remote-ssh] host = 45.58.42.69 user = root port = 22 pass = *** ENCRYPTED *** key_file_pass = *** ENCRYPTED *** use_insecure_cipher = false -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y

If everything is fine, type y and hit Enter to finish the setup:

Current remotes: Name Type ==== ==== remote-ssh sftp e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config

Press q to exit from the configuration wizard.

At this point, Rclone is configured to sync and copy files over the SSH to a remote server with the name remote-ssh.

How to Use Rclone

First, check all the directories on the remote filesystem with the following command:

rclone lsd remote-ssh:/

Sample output:

-1 2021-07-08 07:23:38 -1 bin -1 2020-05-14 03:35:45 -1 boot -1 2021-07-08 07:23:34 -1 dev -1 2021-07-08 07:23:18 -1 etc -1 2020-05-14 03:38:40 -1 home -1 2020-05-15 20:07:11 -1 lib -1 2020-04-23 07:34:14 -1 lib32 -1 2020-05-14 03:29:00 -1 lib64 -1 2020-04-23 07:34:14 -1 libx32 -1 2020-05-14 03:28:53 -1 lost+found -1 2020-05-14 03:29:01 -1 media -1 2020-04-23 07:34:19 -1 mnt -1 2020-04-23 07:34:19 -1 opt -1 2021-07-08 07:23:25 -1 proc -1 2020-05-15 20:12:47 -1 root -1 2021-07-08 07:43:21 -1 run -1 2020-05-14 03:36:23 -1 sbin -1 2020-04-23 07:34:19 -1 srv -1 2021-07-08 07:23:26 -1 sys -1 2021-07-08 07:23:38 -1 tmp -1 2020-05-14 03:29:00 -1 usr -1 2020-05-14 03:29:00 -1 var

To check the size of the specific remote directory, run:

rclone size remote-ssh:/etc

Sample output:

Total objects: 759 Total size: 26.654 MBytes (27948477 Bytes)

To list files and directories on the remote server in a tree format, run:

rclone tree remote-ssh:/etc/default/

Sample output:

/ ├── acpid ├── bsdmainutils ├── console-setup ├── cron ├── dbus ├── grub ├── grub.d │ └── init-select.cfg ├── keyboard ├── locale ├── motd-news ├── networkd-dispatcher ├── nss ├── ntp ├── ssh └── useradd

To create a backup directory on the remote system, run:

rclone mkdir remote-ssh:/backup

To copy all files and directories from the /boot/grub directory on the local system to the backup directory on the remote system, run:

rclone copy /boot/grub remote-ssh:/backup/

To copy all files and directories from the backup directory on the remote system to the mnt directory on the local system, run:

rclone copy remote-ssh:/backup/ /mnt/

To remove a single file on a remote ssh server, run:

rclone deletefile remote-ssh:/backup/grub.cfg

To remove all the content of the remote directory, run:

rclone delete remote-ssh:/backup/i386-pc

To remove an empty directory on the remote system, run:

rclone remove remote-ssh:/backup/directory

To move files and directories from the local system to the remote system, run:

rclone move /mnt remote-ssh:/backup/

To display the content of the remote file, run:

rclone cat remote-ssh:/etc/fstab

Sample output:

# /etc/fstab: static file system information. # Use 'blkid' to print the universally unique identifier for a # device; this may be used with UUID= as a more robust way to name devices # that works even if disks are added and removed. See fstab(5). # <file system> <mount point> <type> <options> <dump> <pass> # / was on /dev/sda1 during installation UUID=29a0b164-1ba1-45a7-b23a-cdb98f23edbc / ext4 noatime,errors=remount-ro 0 1 /swapfile none swap sw 0 0

To synchronize the local directory with the remote directory, run:

rclone sync /mnt remote-ssh:/backup

To synchronize the remote directory with the local directory, run:

rclone sync remote-ssh:/backup /mnt

Conclusion

The above guide shows you how to copy, move and synchronize files and directories to the remote server with Rclone over the SSH protocol. You can now use Rclone to sync your local files and directories with Google Drive, Amazon S3 or other storage providers.

What is the difference between rsync and rclone?

Rsync only has one thread. Rclone has several threads and allows for a user-defined number of concurrent transfers. Rclone can sometimes pipe data without requiring local download between two entirely remote places. One side of an Rsync transfer needs to be a local drive.

Does rclone use SSH?

Rclone will attempt to contact an ssh-agent if pass, key file, key pem, or ask password is not specified. To require the use of an ssh-agent, you can also specify a key use agent. In this situation, a key file or key pem can also be given to requiring the ssh-agent to use a particular key.