Ever needed to back up thousands of files to the cloud, but dreaded the thought of uploading everything again? That’s where AWS S3 sync comes to the rescue.
This powerful command is like having a smart assistant that knows exactly which files need updating—and skips the rest.
Think of it as the difference between moving your entire house versus just packing what’s changed since last time. Way more efficient, right?
In this guide, you’ll learn how the aws s3 sync command works, when to use it over other methods, and master essential options like the delete flag.
We’ll walk through real examples, from simple backups to syncing massive datasets between buckets. Plus, you’ll learn cost-saving tricks that could significantly reduce your AWS bill.
Let’s make cloud synchronization simple.
What is aws s3 sync Command?
The aws s3 sync command is your go-to tool for keeping files in perfect harmony between your computer and Amazon’s cloud storage (S3).
Think of it as a smart copying assistant that automatically figures out which files actually need to be transferred—and leaves the rest alone.
Here’s what makes it special: instead of blindly copying everything like a traditional file transfer, sync command compares the source and destination first. It checks each file’s size and last modification date.
Only files that are new, changed, or missing get transferred. This intelligent approach saves you time, bandwidth, and money, especially when dealing with large directories or frequent backups.
aws s3 sync
aws s3 sync
What You Can Sync:
- Local folder → S3 bucket (upload your files to the cloud)
- S3 bucket → Local folder (download files to your computer)
- S3 bucket → Another S3 bucket (copy between cloud locations)
The Key Benefit:
Here’s the game-changer: let’s say you have 10,000 files in a folder, but only 50 have changed since yesterday. Traditional copy commands would transfer all 10,000 files.
The sync command? It smartly transfers just those 50 changed files. That’s the difference between waiting hours versus minutes, and between paying for 10,000 transfers versus 50.
If you’re backing up your website, syncing project files, or managing data across multiple locations, aws s3 sync does the heavy lifting while keeping your costs down.
You can find the official aws s3 sync documentation here.
Core Concepts: Understanding aws s3 sync
Use this section to quickly understand how aws s3 sync works, why it saves time and costs, and when cp –recursive is a better fit.
aws s3 sync vs cp Command: Which One Should You Use?
Both commands move files to and from S3, but they work very differently. Choosing the wrong one could mean wasting time and money, so let’s clear up the confusion.
The Key Difference:
-
aws s3 cp --recursivecopies EVERYTHING from source to destination, even files that already exist. It’s like photocopying an entire filing cabinet, including documents you already have. -
aws s3 syncis smarter, it checks what’s already at the destination and only copies what’s new or changed. It’s like updating your filing cabinet with just the new or modified documents.
| Feature | aws s3 sync |
aws s3 cp --recursive |
|---|---|---|
| What gets copied | Only new and changed files | All files, every time |
| Checks the destination first | Yes, compares before copying | No, just copies everything |
| Can delete removed files | Yes (with --delete flag) |
No, never deletes anything |
| Best for | Regular backups, ongoing sync | Brand new uploads, one-time copies |
| Speed for updates | Fast, skips unchanged files | Slow, copies everything again |
| Cost efficiency | Lower, fewer transfers | Higher, transfers all files |
| Overwrites existing files | Only if the source is newer/different | Always overwrites |
When to Use aws s3 sync
Choose sync when:
- You’re Doing Regular Backups: Uploading your project files every night? Sync only transfers what changed today.
- Managing Large, Existing Directories: Got 50GB already in S3 and adding 100MB of updates? Sync saves you from re-uploading the entire 50GB.
- You Need Two-Way Maintenance: The
--deleteoption keeps your destination as an exact mirror of the source. - Cost Matters: Why pay to transfer files that haven’t changed?
Real example: Your website has 5,000 images. You update 10 product photos. Sync transfers just those 10. CP would transfer all 5,000.
When to Use aws s3 cp
Choose cp --recursive when:
- Starting Fresh: First-time upload of a new project or directory.
- Copying Individual Files: Just need to move one or two specific files.
- You Want to Force Replacement: Need to overwrite everything regardless of changes.
- Simple One-Time Transfers: Moving data you’ll never sync again.
Real example: Migrating a completely new application to S3 for the first time, nothing exists there yet, so sync’s comparison would be pointless.
aws s3 sync Command Options You’ll Actually Use
These are the flags that make sync work the way you need it to. Each one solves a specific problem, so let’s keep it simple.
--dryrun – See what would happen without actually doing it
This is your safety net. Run sync with --dryrun first to preview which files would transfer or delete. Nothing actually moves—you just get a list of what would happen.
aws s3 sync ./my-folder s3://my-bucket/ --delete --dryrun
Always use this before running commands with --delete. It’s saved countless people from accidental deletions.
--exclude and --include – Pick which files to sync
These filters let you be picky about what gets copied. Want to skip log files? Only upload images? These flags have you covered.
Important: Order matters. The filters run in the sequence you write them, and the last matching rule wins.
# Upload everything except .log files
aws s3 sync ./app s3://my-bucket/ --exclude "*.log"
# Upload only .jpg and .png files
aws s3 sync ./photos s3://my-bucket/ --exclude "*" --include "*.jpg" --include "*.png"
--no-follow-symlinks – Don’t follow symbolic links
By default, sync follows symbolic links (shortcuts) and copies the actual files they point to. Add this flag if you want to skip symlinks entirely.
aws s3 sync ./project s3://my-bucket/ --no-follow-symlinks
--storage-class – Choose where S3 stores your files
Not all files need expensive, instant-access storage. This flag lets you save money by picking cheaper storage tiers for things like backups or archives.
# Store as infrequent access (cheaper for rarely-accessed files)
aws s3 sync ./backups s3://my-bucket/ --storage-class STANDARD_IA
Options include STANDARD, STANDARD_IA, GLACIER, and others.
--sse – Encrypt your files automatically
Add encryption when files land in S3. This keeps your data secure without extra steps.
# Use S3's default encryption
aws s3 sync ./sensitive-data s3://my-bucket/ --sse AES256
--only-show-errors – Keep the Output Clean
Sync normally prints every file it processes. For automated scripts or CI/CD pipelines, that’s too much noise. This flag silences everything except errors.
aws s3 sync ./build s3://my-bucket/ --only-show-errors
Perfect for cron jobs or deployment scripts where you only care when something breaks.
Include/exclude Filters
Here’s where things get powerful. Filters decide which files participate in the sync. They’re evaluated against each file in the source directory, and the order you write them actually changes the outcome.
The golden rule: Filters are processed in order, and the last matching rule wins.
Think of it like this: each file walks through your filter list from top to bottom. Whatever the last rule says about that file is what happens.
Example 1: Upload only .html Files
Start by excluding everything, then include what you want.
aws s3 sync ./website s3://my-bucket/ --exclude "*" --include "*.html"
Every file hits --exclude "*" first (excluded). Then .html files hit --include "*.html" (now included). Everything else stays excluded.
Example 2: Skip Images
Want everything except images? Exclude the image types.
aws s3 sync ./files s3://my-bucket/ --exclude "*.jpg" --exclude "*.png" --exclude "*.gif"
Example 3: Keep a Folder from Being Deleted
Here’s a tricky one. When using --delete, you might want to protect certain folders from deletion. The key is excluding them before sync even looks at them.
aws s3 sync ./site s3://my-bucket/ --delete --exclude "archived/*"
This tells sync to ignore the archived/ folder entirely. Since it’s not part of the sync operation, those files won’t be deleted, even with --delete enabled.
Files excluded from sync are also excluded from deletion checks. If sync doesn’t see them, it can’t remove them.
Combining Filters for Complex Scenarios
Need to get fancy? Stack multiple rules.
# Sync everything except logs, but DO include error.log
aws s3 sync ./app s3://my-bucket/ --exclude "*.log" --include "error.log"
The error.log file matches both rules, but --include "error.log" comes last, so it wins and gets uploaded.
When debugging filter issues, add --dryrun to see exactly which files match your rules before anything actually syncs.
Wrapping Up
The aws s3 sync command takes the guesswork out of managing files between your computer and Amazon S3. It’s smart enough to skip files that haven’t changed, which saves time and money on every run.
Remember the essentials: always test with --dryrun before using --delete, understand that filter order matters, and let sync’s comparison logic do the heavy lifting.
Even if you’re backing up important data, deploying a website, or moving files between buckets, these techniques will keep your workflow smooth and your AWS bills reasonable.
The difference between a good sync and a great one often comes down to the right combination of flags. Start simple, add options as you need them, and keep those filters organized.
Ready to put this into practice? Grab one of the examples from this guide and try it with --dryrun first. You’ll be syncing like a pro in no time.