I decided to use AWS S3 more actively to backup files that I don't need to keep permanently on my laptop. And this is quite easy using the command line tool S3cmd.
Here are my quick notes getting started.
First, sign-up to Amazon AWS
Quoting from S3cmd's README file:
S3 is a paid service operated by Amazon. Before storing anything into S3 you must sign up for an "AWS" account... to obtain a pair of identifiers: Access Key and Secret Key.
Head to the Amazon AWS website and sign up for the Amazon S3 service, if not yet done. You will get your Access key ID and Secret access key, which are needed to sign requests that you make to an AWS service if you use the AWS SDKs or call the APIs. Check the documentation about AWS Security Credentials here for the details.
Install and configure the tool
S3cmd is a Python-based tool, so in the simplest case, you would install it using
$ pip install s3cmd
Or you can use the more manual way: Download the package from the Github repository or from PyPI, unzip it and then run the setup script.
$ python setup.py install
After the installation step, configure it:
$ s3cmd --configure
To complete the configuration process, you will be asked to provide the two keys that you obtain from your Amazon AWS account dashboard.
Your Amazon S3 buckets
To add or make an S3 bucket with S3cmd, use the
mb subcommand and provide a unique identifier for the bucket.
$ s3cmd mb s3://contentgardening.com-backups Bucket 'contentgardening.com-backups' created
You can list your buckets with
$ s3cmd ls 2016-07-12 13:01 s3://contentgardening.com-backups
Upload files to S3
I am now able to upload a bunch of image files to the bucket using the
$ s3cmd put --force ../Screenshots/*.png s3://contentgardening.com-backups/images/Screenshots/ ... upload: '../Screenshots/Screenshot 2015-07-03 14.40.17.png' -> 's3://contentgardening.com-backups/images/Screenshots/Screenshot 2015-07-03 14.40.17.png' [1 of 79] 142517 of 142517 100% in 9s 15.06 kB/s done upload: '../Screenshots/Screenshot 2015-07-03 14.40.18.png' -> 's3://contentgardening.com-backups/images/Screenshots/Screenshot 2015-07-03 14.40.18.png' [2 of 79] ...
It is worth nothing that Amazon S3 has a flat structure, with no hierarchy, to store objects. In the above example,
Screenshots/ parts in the "S3 destination" are key name prefixes used to group the objects.
So you could think of folders and subfolders, though technically, we do not have a directory structure.
There are more possibilities offered such as the
sync subcommand a bit like with the
on Unix/Linux. Also one could integrate this tool in automation scripts for specific needs, which I am going to
explore at some point.