#Error when using aws sync to R2

4 messages · Page 1 of 1 (latest)

quasi jacinth
#

I use this command to sync my folder to R2. It used to work, but it now fails with x-amz-checksum-crc64nvme not implemented — How do I solve this?

aws s3 sync \
    /Volumes/G24/Folder \
    s3://mybucket/Folder \
    --endpoint-url https://xxxxxxx.r2.cloudflarestorage.com \
    --exclude "*/.DS_Store" \
    --exclude ".DS_Store" \
    --exclude "*/Temporary*" \
    --exclude "Temporary*" \
    --size-only 

upload failed: An error occurred (NotImplemented) when calling the UploadPart operation: unsupported trailed: x-amz-checksum-crc64nvme not implemented

candid rose
#

use rclone

#

?rclone

silk cragBOT
#

rclone is a command-line program which allows you to easily manage R2 buckets using the S3 API.

Setup

  1. Install rclone
  2. Configuration guide

Cheatsheet

  • rclone copy: upload new files without deleting non-existing files (will overwrite files with same name)
  • rclone sync: upload new files and delete unexpected files (will overwrite files with same name)
  • rclone purge: delete entire bucket (will remove all objects and then the bucket, use --checkers 100 for better performance)
  • rclone cleanup -o max-age=0s: delete all pending multipart uploads
  • -vP to show progress (strongly recommended)
  • --s3-chunk-size 100M --s3-upload-concurrency=20 --transfers 10 options to get better performance when copying to R2
  • --retries 10 if your connection isn't reliable
  • --dry-run to make sure you aren't going to accidentally wipe your entire bucket

Examples:

  • rclone copy -vP ./dist r2:<bucket>: copies your ./dist directory to the R2 bucket you specified
  • rclone sync -vP r2:<bucket1> r2:<bucket2>: move your objects from one bucket to another (will use server-side copies)
  • rclone sync -vP s3:<bucket> r2:<bucket>: migrate your objects from another provider to R2 (will proxy all traffic through your machine, make sure you got a very good connection)