rsync vs rclone for nightly offsite backups
We moved a 2TB nightly backup from a rented box to object storage. The obvious question was whether to keep rsync and mount the bucket, or switch to rclone. The answer turned out to depend entirely on what a restore looks like.
Where rsync still wins
rsync's delta algorithm only pays off when both ends can read the file. Against a real filesystem over ssh, changing 40MB inside a 3GB file transfers roughly 40MB. It also preserves hardlinks, xattrs, and sparse files, which matters more than people expect when the thing you are backing up is a VM image.
rsync -aHAX --delete --numeric-ids \
--partial --info=progress2 \
/srv/data/ backup@offsite:/pool/data/
Where it falls apart
Point rsync at a FUSE-mounted bucket and the delta algorithm becomes a liability: it reads the whole remote file to compute checksums, so you pay full egress to upload a small change. The nightly job went from 20 minutes to over four hours.
rclone, with the flags that matter
rclone speaks the storage API directly and compares by size and modtime instead of rolling checksums. The defaults are conservative; these three changed the runtime the most for us:
| Flag | Effect |
|---|---|
--transfers=16 | parallel uploads, the single biggest win |
--fast-list | one listing pass instead of per-directory calls |
--backup-dir | moves overwritten files aside instead of destroying them |
rclone sync /srv/data remote:backups/data \
--transfers=16 --checkers=32 --fast-list \
--backup-dir=remote:backups/history/$(date +%F) \
--log-file=/var/log/rclone/nightly.log --log-level INFO
--backup-dir is the important one. Plain sync is a mirror,
which means a corrupted source faithfully replicates the corruption and you have no
previous copy. With a dated backup dir you get cheap versioning without paying for
full snapshots.
The part everyone skips
Restore timing. Uploading 2TB overnight is fine; pulling it back during an incident at 40MB/s is nine hours you did not budget for. Test the restore path once a quarter and time it, or you do not have a backup, you have a receipt.