S5cmd: Parallel S3 and local filesystem execution tool

Summary

Overview s5cmd is a very fast S3 and local filesystem execution tool. It comes with support for a multitude of operations including tab completion and wildcard support for files, which can be very handy for your object storage workflow while working with large number of files. There are already other utilities to work with S3 and similar object storage services, thus it is natural to wonder what s5cmd has to offer that others don't. In short, s5cmd offers a very fast speed. Thanks to Joshua Robinson for his study and experimentation on s5cmd; to quote his medium post: For uploads, s5cmd is 32x faster than s3cmd and 12x faster than aws-cli. For downloads, s5cmd can saturate a 40Gbps link (~4.3 GB/s), whereas s3cmd and aws-cli can only reach 85 MB/s and 375 MB/s respectively. If you would like to know more about performance of s5cmd and the reasons for its fast speed, refer to benchmarks section Features s5cmd supports wide range of object management tasks both for cloud storage services and local filesystems. List buckets and objects Upload, download or delete objects Move, copy or rename objects Set Server Side Encryption using AWS Key Management Service (KMS) Set Access Control List (ACL) for objects/files on the upload, copy, move. Print object contents to stdout Select JSON records from objects using SQL expressions Create or remove buckets Summarize objects sizes, grouping by storage class Wildcard support for all operations Multiple arguments support for delete operation Command file support to run commands in batches at very high execution speeds Dry run support S3 Transfer Acceleration support Google Cloud Storage (and any other S3 API compatible service) support Structured logging for querying command outputs Shell auto-completion S3 ListObjects API backward compatibility Installation Official Releases Binaries The Releases page provides pre-built binaries for Linux, macOS and Windows. Homebrew For macOS, a homebrew tap is provided: brew install peak/tap/s5c...