AWS S3 sync --delete, removed new files in local
Amazon Web-ServicesAmazon S3Aws CliAmazon Web-Services Problem Overview
aws s3 sync --delete
removed some new files.
For example:
There is a file in the bucket - S3://my-bucket/images/1.jpg
Then, I uploaded a file to the server: 2.jpg
There are 2 files in the server: 1.jpg
and 2.jpg
Start running the sync cronjob:
aws s3 sync s3://my-bucket/ ./ --delete
aws s3 sync ./ s3://my-bucket/ --delete
Why do we add --delete
- we want to delete the files in s3 and sync it to the server.
We will upload files to the server and remove the files in s3.
Is there any way to fix it?
Amazon Web-Services Solutions
Solution 1 - Amazon Web-Services
By default, the aws sync
command (see documentation) does not delete files. It simply copies new or modified files to the destination.
Using the --delete
option deletes files that exist in the destination but not in the source.
So, if your source contains: 1.jpg
and 2.jpg
and the destination contains 1.jpg
, 2.jpg
and 3.jpg
, then using the --delete
option will delete 3.jpg
from the destination.
I see that you are running the sync
command in both directions. Your first command (that syncs S3 to a local directory) will delete any local files that are not in S3.
If your goal is to copy all local files to S3 and all S3 files to the local directory, without deleting any files, then do not use the --delete
option.
Solution 2 - Amazon Web-Services
--delete option to remove files or objects from the target not present in the source.
Solution 3 - Amazon Web-Services
For the new s3cmd
you can use the option --delete-removed
to remove extra files in the destination, that are not present the source.