Skip to main content

S3 Buckets

This guide provides a quick reference for interacting with AWS S3 buckets using the AWS CLI.

Setup​

Install AWS CLI​

If you haven't installed the AWS CLI yet, you can do so using Homebrew on macOS:

Install AWS CLI
brew install awscli

Configure AWS CLI​

Before you can use the CLI, you need to configure it with your AWS credentials:

Configure AWS CLI
aws configure

You will be prompted to enter:

  • AWS Access Key ID
  • AWS Secret Access Key
  • Default region name (e.g., us-east-1)
  • Default output format (e.g., json, text, or table)

List Buckets​

To list all the S3 buckets in your AWS account:

List S3 Buckets
aws s3 ls
Sample Output
2019-02-20 14:53:51 my-bucket
2019-02-20 14:53:51 my-other-bucket
note

To list buckets, you need to have the s3:ListAllMyBuckets permission.

List Contents of a Bucket​

To list the contents of a specific S3 bucket:

List Bucket Contents
aws s3 ls s3://your-bucket-name/
Sample Output
2019-02-20 14:53:51         1234 file1.txt
2019-02-20 14:53:51         5678 file2.txt
2019-02-20 14:53:51         9012 file3.txt

If your bucket contains "folders" (prefixes), you can navigate into them:

Navigate Prefixes
aws s3 ls s3://your-bucket-name/prefix/
Sample Output
2019-02-20 14:53:51         1234 file1.txt
2019-02-20 14:53:51         5678 file2.txt
2019-02-20 14:53:51         9012 file3.txt

Download a File from S3​

To download a file from an S3 bucket to your local machine, you have two options:

Option 1: Download to the current directory​

Download to Current Directory
aws s3 cp s3://your-bucket-name/path/to/file.ext .
Sample Output
download: s3://your-bucket-name/path/to/file.ext to ./file.ext

Option 2: Download to a specific directory​

Download to Specific Directory
aws s3 cp s3://your-bucket-name/path/to/file.ext /path/to/local/destination/
Sample Output
download: s3://your-bucket-name/path/to/file.ext to /path/to/local/destination/file.ext

Download All Files in a Prefix​

To download all files from a specific prefix (folder) in an S3 bucket, you have two options:

Option 1: Download to the current directory​

Download All to Current Directory
aws s3 cp s3://your-bucket-name/prefix/ . --recursive
Sample Output
download: s3://your-bucket-name/prefix/file1.txt to ./file1.txt
download: s3://your-bucket-name/prefix/file2.txt to ./file2.txt
download: s3://your-bucket-name/prefix/file3.txt to ./file3.txt

Option 2: Download to a specific directory​

Download All to Specific Directory
aws s3 cp s3://your-bucket-name/prefix/ /path/to/local/destination/ --recursive
Sample Output
download: s3://your-bucket-name/prefix/file1.txt to /path/to/local/destination/file1.txt
download: s3://your-bucket-name/prefix/file2.txt to /path/to/local/destination/file2.txt
download: s3://your-bucket-name/prefix/file3.txt to /path/to/local/destination/file3.txt

Upload a File to S3​

To upload a file from your local machine to an S3 bucket:

Upload File to S3
aws s3 cp /path/to/local/file.ext s3://your-bucket-name/path/in/bucket/
Sample Output
upload: /path/to/local/file.ext to s3://your-bucket-name/path/in/bucket/file.ext

Remove a File from S3​

To delete a file in an S3 bucket:

Remove File from S3
aws s3 rm s3://your-bucket-name/path/to/file.ext
Sample Output
delete: s3://your-bucket-name/path/to/file.ext

Additional Tips​

  • Use --dryrun to simulate actions without actually making changes.
  • Use --recursive to apply a command to all files in a directory.
  • Use --profile if you manage multiple AWS accounts.

This guide covers the basic commands needed to navigate, download, and manage files in S3 using the AWS CLI. For more advanced usage, refer to the AWS CLI documentation.