从AWS S3恢复已删除文件夹

Question

我有一个启用了版本控制的S3存储桶。可以恢复已删除的文件，但是如何恢复已删除的文件夹呢？

我知道，S3没有文件夹... 但是如何恢复常见前缀呢？是否有递归恢复文件的可能性？

- Roland Ettinger

Answer 1

我创建了这个简单的bash脚本来恢复我删除的S3文件夹中的所有文件：

#!/bin/bash
    
recoverfiles=$(aws s3api list-object-versions --bucket MyBucketName  --prefix TheDeletedFolder/ --query "DeleteMarkers[?IsLatest && starts_with(LastModified,'yyyy-mm-dd')].{Key:Key,VersionId:VersionId}")
for row in  $(echo "${recoverfiles}" | jq -c '.[]'); do
    key=$(echo "${row}" | jq -r '.Key'  )
    versionId=$(echo "${row}" | jq -r '.VersionId'  )
    echo aws s3api delete-object --bucket MyBucketName --key $key --version-id $versionId
done

yyyy-mm-dd = 清空文件夹的日期

- Noz

---

这个脚本会回显你需要运行的所有命令。你可以将这个脚本的输出重定向（>）到一个文本文件中，然后运行 cat myTextFile.txt | bash 来实际执行所有的恢复命令。 - sandbar

你是救世主 :) - ketankk

Answer 2

我在这里找到了一个令人满意的解决方案（链接），更多细节请参见这里。

总之，当前没有可用的工具来解决这个问题，但是可以使用一个简单的bash脚本包装AWS工具“s3api”来实现递归还原。

该解决方案适用于我。唯一的缺点是，Amazon似乎会在约30,000个文件后限制还原操作。

- Roland Ettinger

Answer 3

您无法恢复常规前缀。 您需要逐个恢复一个对象。当一个对象出现时，任何相关的文件夹也会重新出现。

恢复已删除的数据有两种方法：

• 删除删除标记以撤消删除，或者
• 将对象的先前版本复制到自身，这将使最新版本比删除标记更新，因此它将重新出现。(我希望你明白了！)

- John Rotenstein

Answer 4

如果一个文件夹及其内容被删除，您可以使用以下脚本进行恢复，该脚本受到以前的答案的启发。

该脚本适用于预先启用版本控制的S3存储桶。它使用删除标记标签来恢复S3前缀中的文件。

#!/bin/bash
# Inspired by https://www.dmuth.org/how-to-undelete-files-in-amazon-s3/

# This script can be used to undelete objects from an S3 bucket.
# When run, it will print out a list of AWS commands to undelete files, which you
# can then pipe into Bash.
#

#
# You will need the AWS CLI tool from https://aws.amazon.com/cli/ in order to run this script.
#
# Note that you must have the following permissions via IAM:
#
# Bucket permissions:
#
#   s3:ListBucket
#   s3:ListBucketVersions
#
# File permissions:
#
#   s3:PutObject
#   s3:GetObject
#   s3:DeleteObject
#   s3:DeleteObjectVersion
#
# If you want to do this in a "quick and dirty manner", you could just grant s3:* to
# the account, but I don't really recommend that.
#
                
# profile = company
# bucket = company-s3-bucket
# prefix = directory1/directory2/directory3/lastdirectory/
# pattern = (.*)

# USAGE
# bash undelete.sh  > recover_files.txt  | bash

read -p "Enter your aws  profile: " PROFILE
read -p "Enter your S3 bucket name: " BUCKET
read -p "Enter your S3 directory/prefix to be recovered from, leave empty for to recover all of the S3 bucket: " PREFIX
read -p "Enter the file pattern looking to recover, leave empty for all: " PATTERN

# Make sure Profile and Bucket are entered
[[ -z "$PROFILE" ]] && { echo "Profile is empty" ; exit 1; }
[[ -z "$BUCKET" ]] && { echo "Bucket is empty" ; exit 1; }

# Fill PATTERN to match all if empty
PATTERN=${PATTERN:-(.*)}

# Errors are fatal
set -e


if [ "$PREFIX" = "" ]; 

# To recover all of the S3 bucket
then

    aws --profile ${PROFILE} --output text s3api list-object-versions --bucket ${BUCKET} \
            | grep -i $PATTERN \
            | grep -E "^DELETEMARKERS" \
            | awk -v PROFILE=$PROFILE -v BUCKET=$BUCKET -v PREFIX=$PREFIX  \
            -F "[\t]+" '{ print "aws --profile " PROFILE " s3api delete-object --bucket " BUCKET "--key \""$3"\" --version-id "$5";"}' 


# To recover a directory
else

    aws --profile ${PROFILE} --output text s3api list-object-versions --bucket ${BUCKET} --prefix ${PREFIX} \
            | grep -E $PATTERN \
            | grep -E "^DELETEMARKERS" \
            | awk -v PROFILE=$PROFILE -v BUCKET=$BUCKET -v PREFIX=$PREFIX  \
            -F "[\t]+" '{ print "aws --profile " PROFILE " s3api delete-object --bucket " BUCKET "--key \""$3"\" --version-id "$5";"}' 
fi

- Natanan

Answer 5

如果您已启用存储桶的版本，请使用以下命令检索文件或文件夹。

对于具有相应文件夹的情况：

echo '#!/bin/bash' > undeleteScript.sh && aws --output text s3api list-object-versions --bucket bucketname --prefix path/to/retrieve| grep -E "^DELETEMARKERS" | awk '{FS = "[\t]+"; print "aws s3api delete-object --bucket buckername --key \42"$3"\42 --version-id "$5";"}' >> undeleteScript.sh && . undeleteScript.sh; rm -f undeleteScript.sh;

针对存储桶中的文件：

echo '#!/bin/bash' > undeleteScript.sh && aws --output text s3api list-object-versions --bucket bucketname --prefix | grep -E "^DELETEMARKERS" | awk '{FS = "[\t]+"; print "aws s3api delete-object --bucket buckername --key \42"$3"\42 --version-id "$5";"}' >> undeleteScript.sh && . undeleteScript.sh; rm -f undeleteScript.sh;

- Prabhakaran Ganesan

Answer 6

我写了一个优化版本，可以批量恢复1000个文件。

#!/bin/bash

if [[ "$#" -lt 3 ]]; then
    echo "Run: $0 <BUCKET> <PREFIX> <PROFILE>"
    exit 1
fi

BUCKET=$1
PREFIX=$2
PROFILE=$3

echo "Restore bucket PREFIX - ${BUCKET}/${PREFIX}"

while true; do
  result="$(aws --profile ${PROFILE} s3api list-object-versions --max-items 1000 --bucket ${BUCKET} --prefix ${PREFIX} --query '{Objects: DeleteMarkers[0:999].{Key:Key,VersionId:VersionId}}')"
  
  if [ "$(echo $result|jq '.Objects')" == "null" ]; then
    echo "No more files to undelete."
    break
  fi
  
  echo "Restoring:"
  aws --profile ${PROFILE} --no-cli-pager s3api delete-objects --bucket ${BUCKET} --delete "$result" |grep Key
done

- Or Kramer

---

根据目前的写法，你的回答不够清晰。请编辑以添加更多细节，帮助其他人理解这如何回答所提出的问题。你可以在帮助中心找到关于如何撰写好回答的更多信息。 - Community