boto3在logrotate中执行时找不到配置文件myprofile

问题描述

我想将一些日志从实例上传到S3。日志旋转配置如下。 Logrotate调用包装程序外壳，该外壳程序调用python（2.7）脚本以使用boto3执行上传。我尝试了各种设置AWS_CONfig_FILE

的方法

1. 作为操作系统全局变量/etc/profile.d/my_aws_config.sh
2. 使用python设置全局变量

/var/log/secure
{
    rotate 1
    hourly
    missingok
    compress
    sharedscripts
    copytruncate
    postrotate
        /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
    endscript
    dateext
    dateformat -%Y-%m-%d-%s
    lastaction
        /bin/sh -xv /opt/upload_to_s3.sh 
    endscript
}

这是一个包装器脚本，依次将参数传递给python脚本 带有代码：

import sys
import os
import logging
import boto3
from botocore.exceptions import ClientError

os.environ["AWS_CONfig_FILE"] = "/root/.aws/config"

archive_session = boto3.session.Session(profile_name='dev')
s3_client = archive_session.client('s3')

def upload_file(file_name,bucket,object_name=None):

    # If S3 object_name was not specified,use file_name
    if object_name is None:
        object_name = file_name

    try:
        response = s3_client.upload_file(file_name,object_name)
    except ClientError as e:
        logging.error(e)
        return False
    return True

folder_path=sys.argv[5]+"/"+sys.argv[4]+"/{}"
print folder_path
upload_file(sys.argv[1],sys.argv[2],folder_path.format(sys.argv[3]))

错误

Traceback (most recent call last):
  File "/opt/techarch-scripts/python.py",line 29,in <module>
    archive_session = boto3.session.Session(profile_name='dev')
  File "/usr/lib/python2.7/site-packages/boto3/session.py",line 80,in __init__
    self._setup_loader()
  File "/usr/lib/python2.7/site-packages/boto3/session.py",line 120,in _setup_loader
    self._loader = self._session.get_component('data_loader')
  File "/usr/lib/python2.7/site-packages/botocore/session.py",line 685,in get_component
    return self._components.get_component(name)
  File "/usr/lib/python2.7/site-packages/botocore/session.py",line 924,in get_component
    self._components[name] = factory()
  File "/usr/lib/python2.7/site-packages/botocore/session.py",line 158,in <lambda>
    lambda:  create_loader(self.get_config_variable('data_path')))
  File "/usr/lib/python2.7/site-packages/botocore/session.py",line 241,in get_config_variable
    logical_name)
  File "/usr/lib/python2.7/site-packages/botocore/configprovider.py",line 301,in get_config_variable
    return provider.provide()
  File "/usr/lib/python2.7/site-packages/botocore/configprovider.py",line 398,in provide
    value = provider.provide()
  File "/usr/lib/python2.7/site-packages/botocore/configprovider.py",line 459,in provide
    scoped_config = self._session.get_scoped_config()
  File "/usr/lib/python2.7/site-packages/botocore/session.py",line 340,in get_scoped_config
    raise ProfileNotFound(profile=profile_name)
botocore.exceptions.ProfileNotFound: The config profile (dev) Could not be found

打印python的变量，看起来确实是在设置变量：

{'MAILTO': 'root','LANG': 'en_US.UTF-8','SHELL': '/bin/bash','XDG_RUNTIME_DIR': '/run/user/0','SHLVL': '5','PWD': '/root','LOGNAME': 'root','USER': 'root','AWS_CONfig_FILE': '/root/.aws/config','HOME': '/root','PATH': '/sbin:/bin:/usr/sbin:/usr/bin','XDG_SESSION_ID': '871','_': '/usr/bin/python'}

我的个人资料肯定存在：

[profile dev]
role_arn = arn:aws:iam::XXXXXXXX:role/ec2-s3-role-for-dev
credential_source = Ec2InstanceMetadata

解决方法

Named profiles在AWS中使用以下命名模式命名为user1的配置文件

[user1]
aws_access_key_id=AKIAI44QH8DHBEXAMPLE
aws_secret_access_key=je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY

您只需将dev放在方括号中，而不是profile dev：

[dev]
role_arn = arn:aws:iam::XXXXXXXX:role/ec2-s3-role-for-dev
credential_source = Ec2InstanceMetadata