我想在AWS上安排我的Python脚本，但我不希望实例一直运行。因此，我想自动化以下过程：
由于脚本需要更多RAM来执行并行处理，所以我不能将脚本直接运行为Lambda函数，因此选择了更大的AWS实例而不是编写为Lambda函数的方式。另外，由于费用昂贵，不希望该实例一直运行。
到目前为止，我按照Automatic starting and stopping of AWS EC2 instances with Lambda and CloudWatch · matoski.com 创建了一个Lambda函数，在特定时间启动和停止实例，但我找不到一种方法使Python脚本在实例启动后运行。
有人能指点我吗？
我的应用程序每天在13:39 UST运行一个实例，并在处理完成后自动关闭。它使用以下内容
import boto3
def lambda_handler(event, context):
ec2 = boto3.client('ec2', region_name='ap-south-1')
ec2.start_instances(InstanceIds=['i-xxxxxxx'])
print('started your instances: ' + str('i-xxxxxx'))
return触发一个拥有定时执行 Python 脚本的 cron 实例
@reboot python /home/Init.py
脚本完成后，Python 作业使用下面的片段自行关闭
import boto.ec2
import boto.utils
import logging
logger=logging.getLogger()
def stop_ec2():
conn = boto.ec2.connect_to_region("ap-south-1") # or your region
# Get the current instance's id
my_id = boto.utils.get_instance_metadata()['instance-id']
logger.info(' stopping EC2 :'+str(my_id))
conn.stop_instances(instance_ids=[my_id])针对未来前来查看这个问题的开发人员，现在有一种更新的方法：
AmazonEC2RoleforSSM 策略的角色 EC2 实例因此：
按照这里的步骤进行操作：https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html
使用以下 Lambda 函数模板:
import time
import boto3
REGION_NAME = 'us-east-1'
WORKING_DIRECTORY = '<YOUR WORKING DIRECTORY, IF ANY>'
COMMAND = """
echo "Hello, world!"
"""
INSTANCE_ID = '<YOUR INSTANCE ID>'
def start_ec2():
ec2 = boto3.client('ec2', region_name=REGION_NAME)
ec2.start_instances(InstanceIds=[INSTANCE_ID])
while True:
response = ec2.describe_instance_status(InstanceIds=[INSTANCE_ID], IncludeAllInstances=True)
state = response['InstanceStatuses'][0]['InstanceState']
print(f"Status: {state['Code']} - {state['Name']}")
# If status is 16 ('running'), then proceed, else, wait 5 seconds and try again
if state['Code'] == 16:
break
else:
time.sleep(5)
print('EC2 started')
def stop_ec2():
ec2 = boto3.client('ec2', region_name=REGION_NAME)
ec2.stop_instances(InstanceIds=[INSTANCE_ID])
while True:
response = ec2.describe_instance_status(InstanceIds=[INSTANCE_ID], IncludeAllInstances=True)
state = response['InstanceStatuses'][0]['InstanceState']
print(f"Status: {state['Code']} - {state['Name']}")
# If status is 80 ('stopped'), then proceed, else wait 5 seconds and try again
if state['Code'] == 80:
break
else:
time.sleep(5)
print('Instance stopped')
def run_command():
client = boto3.client('ssm', region_name=REGION_NAME)
time.sleep(10) # I had to wait 10 seconds to "send_command" find my instance
cmd_response = client.send_command(
InstanceIds=[INSTANCE_ID],
DocumentName='AWS-RunShellScript',
DocumentVersion="1",
TimeoutSeconds=300,
MaxConcurrency="1",
CloudWatchOutputConfig={'CloudWatchOutputEnabled': True},
Parameters={
'commands': [COMMAND],
'executionTimeout': ["300"],
'workingDirectory': [WORKING_DIRECTORY],
},
)
command_id = cmd_response['Command']['CommandId']
time.sleep(1) # Again, I had to wait 1s to get_command_invocation recognises my command_id
retcode = -1
while True:
output = client.get_command_invocation(
CommandId=command_id,
InstanceId=INSTANCE_ID,
)
# If the ResponseCode is -1, the command is still running, so wait 5 seconds and try again
retcode = output['ResponseCode']
if retcode != -1:
print('Status: ', output['Status'])
print('StdOut: ', output['StandardOutputContent'])
print('StdErr: ', output['StandardErrorContent'])
break
print('Status: ', retcode)
time.sleep(5)
print('Command finished successfully') # Actually, 0 means success, anything else means a fail, but it didn't matter to me
return retcode
def lambda_handler(event, context):
retcode = -1
try:
start_ec2()
retcode = run_command()
finally: # Independently of what happens, try to shutdown the EC2
stop_ec2()
return retcode
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:*:*:*"
},
{
"Effect": "Allow",
"Action": [
"ec2:Start*",
"ec2:Stop*"
],
"Resource": "*"
}
]
}
转到https://console.aws.amazon.com/iam/home#/home，在左侧选择角色。确保您选择Lambda作为AWS服务，并附加在步骤1中创建的策略。
然后进入Lambda控制台，点击创建函数。选择Python 3.7，然后单击权限旁边的下拉菜单并使用现有角色，附加您在步骤2中创建的IAM角色。
将以下代码用作您的代码：
import boto3
region = 'us-west-1' # Dont use the specific, like instead of us-east-1d just write us-east-1
instances = ['i-xxxxxxxxxxxx']
ec2 = boto3.client('ec2', region_name=region)
def lambda_handler(event, context):
ec2.start_instances(InstanceIds=instances)
print('started your instances: ' + str(instances))
which python以查找Python路径，并将其写下来。然后，键入crontab -e以编辑您的CRON作业。不要使用sudo...因为有时候当您没有使用它来运行Python文件时，sudo会搞砸一些东西。在我的实例中，我有一个存储密码的pgpass文件，但是删除sudo就可以解决问题！@reboot /path/to/python /path/to/file.py。例如，对于我来说，这是@reboot /home/init/python /home/init/Notebooks/mypredictor.pyimport boto3
region = 'us-west-1' # Dont use the specific, like instead of us-east-1d just write us-east-1
instances = ['i-xxxxxxxxxxxx']
ec2 = boto3.client('ec2', region_name=region)
ec2.stop_instances(InstanceIds=instances)
ec2.stop_instances这一行（或编辑你的crontab），保存并等待机器重新启动，然后再启动机器。可能的改进是：在调用你的Python脚本之前，你运行git pull来拉取任何代码更改，然后再运行你的Python脚本。 - Corey Levinson
/etc/rc.local调用脚本。 - OneCricketeer