SnapShooter Learning Center

How to Install and Use Rclone on Ubuntu 20.04

How to Install and Use Rclone on Ubuntu 20.04

Simon Bennett]
Simon Bennett
Last Updated: Aug 10, 2021
Table of Contents

Rclone is a command-line software tool used to copy, sync, manage and migrate files and directories to a remote server, cloud or other storage. It can be installed on Linux, Windows, macOS, Debian, OpenBSD, and several other operating systems. Currently, it supports over 40+ cloud storage including, AWS, Google Cloud, Google Drive, Dropbox, pCloud, Zoho, Mega and more. It also supports SSH and SFTP protocol to sync files between Linux servers.

Rclone employs server-side transfers to minimize local bandwidth usage and transfers from one provider to another without using the local disk. It allows you to mount any local or cloud filesystem as a disk on Linux and Windows.

In this guide, we will show you how to copy and sync files and directories to a remote Linux server using Rclone over the SSH protocol.


# Prerequisites

  • Two servers running Ubuntu 20.04 server. one for Rclone and one for remote SSH.
  • A root password is configured on both servers.

# Install Rclone

By default, Rclone is included in the Ubuntu default repository. You can install it using the following command:

apt install rclone -y

After installing Rclone, verify the Rclone version with the following command:

rclone --version

Sample output:

rclone v1.50.2
- os/arch: linux/amd64
- go version: go1.13.6

Run the following command to see all options available with Rclone:

rclone --help

Sample output:

Available Commands:
 about           Get quota information from the remote.
 authorize       Remote authorization.
 cachestats      Print cache stats for a remote
 cat             Concatenates any files and sends them to stdout.
 check           Checks the files in the source and destination match.
 cleanup         Clean up the remote if possible
 config          Enter an interactive configuration session.
 copy            Copy files from source to dest, skipping already copied
 copyto          Copy files from source to dest, skipping already copied
 copyurl         Copy url content to dest.
 cryptcheck      Cryptcheck checks the integrity of a crypted remote.
 cryptdecode     Cryptdecode returns unencrypted file names.
 dbhashsum       Produces a Dropbox hash file for all the objects in the path.
 dedupe          Interactively find duplicate files and delete/rename them.
 delete          Remove the contents of path.
 deletefile      Remove a single file from remote.
 genautocomplete Output completion script for a given shell.
 gendocs         Output markdown docs for rclone to the directory supplied.
 hashsum         Produces an hashsum file for all the objects in the path.
 help            Show help for rclone commands, flags and backends.
 link            Generate public link to file/folder.
 listremotes     List all the remotes in the config file.
 ls              List the objects in the path with size and path.
 lsd             List all directories/containers/buckets in the path.
 lsf             List directories and objects in remote:path formatted for parsing
 lsjson          List directories and objects in the path in JSON format.
 lsl             List the objects in path with modification time, size and path.
 md5sum          Produces an md5sum file for all the objects in the path.
 mkdir           Make the path if it doesn't already exist.
 mount           Mount the remote as file system on a mountpoint.
 move            Move files from source to dest.
 moveto          Move file or directory from source to dest.
 ncdu            Explore a remote with a text based user interface.
 obscure         Obscure password for use in the rclone.conf
 purge           Remove the path and all of its contents.
 rc              Run a command against a running rclone.
 rcat            Copies standard input to file on remote.
 rcd             Run rclone listening to remote control commands only.
 rmdir           Remove the path if empty.
 rmdirs          Remove empty directories under the path.
 serve           Serve a remote over a protocol.
 settier         Changes storage class/tier of objects in remote.
 sha1sum         Produces an sha1sum file for all the objects in the path.
 size            Prints the total size and number of objects in remote:path.
 sync            Make source and dest identical, modifying destination only.
 touch           Create new file or change file modification time.
 tree            List the contents of the remote in a tree like fashion.
 version         Show the version number.

# Configure Rclone

First, you will need to configure Rclone to define a storage provider or remote SSH server. In this section, we will configure Rclone to use a remote SSH server as a backup server.

Run the following command to start the rclone configuration.

rclone config

You should see the following options:

No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n

Type n and hit Enter to create a new remote server.

name> remote-ssh

Type your remote backup server name and hit Enter. You will be asked to choose from the list of storage providers:

Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / 1Fichier
   \ "fichier"
 2 / Alias for an existing remote
   \ "alias"
 3 / Amazon Drive
   \ "amazon cloud drive"
 4 / Amazon S3 Compliant Storage Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, etc)
   \ "s3"
 5 / Backblaze B2
   \ "b2"
 6 / Box
   \ "box"
 7 / Cache a remote
   \ "cache"
 8 / Citrix Sharefile
   \ "sharefile"
 9 / Dropbox
   \ "dropbox"
10 / Encrypt/Decrypt a remote
   \ "crypt"
11 / FTP Connection
   \ "ftp"
12 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
13 / Google Drive
   \ "drive"
14 / Google Photos
   \ "google photos"
15 / Hubic
   \ "hubic"
16 / JottaCloud
   \ "jottacloud"
17 / Koofr
   \ "koofr"
18 / Local Disk
   \ "local"
19 / Mail.ru Cloud
   \ "mailru"
20 / Microsoft Azure Blob Storage
   \ "azureblob"
21 / Microsoft OneDrive
   \ "onedrive"
22 / OpenDrive
   \ "opendrive"
23 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ "swift"
24 / Pcloud
   \ "pcloud"
25 / Put.io
   \ "putio"
26 / SSH/SFTP Connection
   \ "sftp"
27 / Transparently chunk/split large files
   \ "chunker"
28 / Union merges the contents of several remotes
   \ "union"
29 / Webdav
   \ "webdav"
30 / Yandex Disk
   \ "yandex"
31 / http Connection
   \ "http"
32 / premiumize.me
   \ "premiumizeme"
 Storage> 26

Type 26 and hit Enter for the remote SSH server. You should see the following output:

SSH host to connect to
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / Connect to example.com
   \ "example.com"
host> remote-ssh-server-ip
SSH username, leave blank for current username, root
Enter a string value. Press Enter for the default ("").
user> root
SSH port, leave blank to use default (22)
Enter a string value. Press Enter for the default ("").
port> 22
SSH password, leave blank to use ssh-agent.
y) Yes type in my own password
g) Generate random password
n) No leave this optional password blank
y/g/n> y
Enter the password:
password:
Confirm the password:
password:

Type the IP address, port number, username and password of the remote SSH server. You should see the following options:

Path to PEM-encoded private key file, leave blank or set key-use-agent to use ssh-agent.
Enter a string value. Press Enter for the default ("").
key_file> 

Just press Enter to the default:

The passphrase to decrypt the PEM-encoded private key file.

Only PEM encrypted key files (old OpenSSH format) are supported. Encrypted keys
in the new OpenSSH format can't be used.
y) Yes type in my own password
g) Generate random password
n) No leave this optional password blank
y/g/n> y

Type y and hit Enter. You will be asked to set the SSH passphrase.

Enter the password:
password:
Confirm the password:
password:

Define your password and hit Enter:

When set forces the usage of the ssh-agent.

When key-file is also set, the ".pub" file of the specified key-file is read and only the associated key is
requested from the ssh-agent. This allows to avoid `Too many authentication failures for *username*` errors
when the ssh-agent contains many keys.
Enter a boolean value (true or false). Press Enter for the default ("false").
key_use_agent> 

Press Enter for the default:

Enable the use of insecure ciphers and key exchange methods. 

Those algorithms are insecure and may allow plaintext data to be recovered by an attacker.
Enter a boolean value (true or false). Press Enter for the default ("false").
Choose a number from below, or type in your own value
 1 / Use default Cipher list.
   \ "false"
 2 / Enables the use of the aes128-cbc cipher and diffie-hellman-group-exchange-sha256, diffie-hellman-group-exchange-sha1 key exchange.
   \ "true"
 use_insecure_cipher> 1

Type 1 and hit Enter:

Disable the execution of SSH commands to determine if remote file hashing is available.
Leave blank or set to false to enable hashing (recommended), set to true to disable hashing.
Enter a boolean value (true or false). Press Enter for the default ("false").
disable_hashcheck> 
Edit advanced config? (y/n)
y) Yes
n) No
y/n> n

Type n and hit Enter:

    Remote config
 --------------------
[remote-ssh]
host = 45.58.42.69
user = root
port = 22
pass = *** ENCRYPTED ***
key_file_pass = *** ENCRYPTED ***
use_insecure_cipher = false
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y

If everything is fine, type y and hit Enter to finish the setup:

Current remotes:

Name                 Type
====                 ====
remote-ssh           sftp

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config

Press q to exit from the configuration wizard.

At this point, Rclone is configured to sync and copy files over the SSH to a remote server with the name remote-ssh.


# How to Use Rclone

First, check all the directories on the remote filesystem with the following command:

rclone lsd remote-ssh:/

Sample output:

      -1 2021-07-08 07:23:38        -1 bin
      -1 2020-05-14 03:35:45        -1 boot
      -1 2021-07-08 07:23:34        -1 dev
      -1 2021-07-08 07:23:18        -1 etc
      -1 2020-05-14 03:38:40        -1 home
      -1 2020-05-15 20:07:11        -1 lib
      -1 2020-04-23 07:34:14        -1 lib32
      -1 2020-05-14 03:29:00        -1 lib64
      -1 2020-04-23 07:34:14        -1 libx32
      -1 2020-05-14 03:28:53        -1 lost+found
      -1 2020-05-14 03:29:01        -1 media
      -1 2020-04-23 07:34:19        -1 mnt
      -1 2020-04-23 07:34:19        -1 opt
      -1 2021-07-08 07:23:25        -1 proc
      -1 2020-05-15 20:12:47        -1 root
      -1 2021-07-08 07:43:21        -1 run
      -1 2020-05-14 03:36:23        -1 sbin
      -1 2020-04-23 07:34:19        -1 srv
      -1 2021-07-08 07:23:26        -1 sys
      -1 2021-07-08 07:23:38        -1 tmp
      -1 2020-05-14 03:29:00        -1 usr
      -1 2020-05-14 03:29:00        -1 var

To check the size of the specific remote directory, run:

rclone size remote-ssh:/etc

Sample output:

Total objects: 759
Total size: 26.654 MBytes (27948477 Bytes)

To list files and directories on the remote server in a tree format, run:

rclone tree remote-ssh:/etc/default/

Sample output:

/
β”œβ”€β”€ acpid
β”œβ”€β”€ bsdmainutils
β”œβ”€β”€ console-setup
β”œβ”€β”€ cron
β”œβ”€β”€ dbus
β”œβ”€β”€ grub
β”œβ”€β”€ grub.d
β”‚   └── init-select.cfg
β”œβ”€β”€ keyboard
β”œβ”€β”€ locale
β”œβ”€β”€ motd-news
β”œβ”€β”€ networkd-dispatcher
β”œβ”€β”€ nss
β”œβ”€β”€ ntp
β”œβ”€β”€ ssh
└── useradd

To create a backup directory on the remote system, run:

rclone mkdir remote-ssh:/backup

To copy all files and directories from the /boot/grub directory on the local system to the backup directory on the remote system, run:

rclone copy /boot/grub remote-ssh:/backup/

To copy all files and directories from the backup directory on the remote system to the mnt directory on the local system, run:

rclone copy remote-ssh:/backup/ /mnt/

To remove a single file on remote ssh server, run:

rclone deletefile remote-ssh:/backup/grub.cfg

To remove all the content of the remote directory, run:

rclone delete remote-ssh:/backup/i386-pc

To remote an empty directory on the remote system, run:

rclone remove remote-ssh:/backup/directory

To move files and directories from the local system to the remote system, run:

rclone move /mnt remote-ssh:/backup/

To display the content of the remote file, run:

rclone cat remote-ssh:/etc/fstab

Sample output:

# /etc/fstab: static file system information.
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda1 during installation
UUID=29a0b164-1ba1-45a7-b23a-cdb98f23edbc /               ext4    noatime,errors=remount-ro 0       1
/swapfile                                 none            swap    sw              0       0

To synchronize the local directory with the remote directory, run:

rclone sync /mnt remote-ssh:/backup

To synchronize the remote directory with the local directory, run:

rclone sync remote-ssh:/backup /mnt

# Conclusion

In the above guide, we show you how to copy, move and synchronize files and directories to the remote server with Rclone over the SSH protocol. You can now use Rclone to sync your local files and directories with Google drive, Amazone S3 or other storage providers.


Did you find this article helpful?