Using AWS Config to ensure EC2 Cost Optimizer is enabled


Travers Annan

5 minute read

Travers Annan

5 minute read

Wherever possible, organizations of all sizes are moving to remote work these days, which can lead to a spike in IT costs. To help mitigate this, we’re developing a suite of tests and checks to help our customers optimize their accounts.

One handy cost-cutting AWS service is the Compute Optimizer. When enabled, this feature automatically checks your EC2 instances for incorrectly provisioned machines. Once enough data has been collected, the service will make recommendations that should 1) help lower costs (over-provisioned) or 2) improve customer experience (under-provisioned).

We’ve developed a solution that creates a Config rule and a Lambda function to automatically check if the Compute Optimizer is enabled in an account. Here’s how it works:

  1. An AWS Lambda permission is created to allow Config to trigger Lambda functions.
  2. A Config rule is created to trigger a Lambda function daily.
  3. A Lambda function that checks the status of Compute Optimizer is attached to the Config rule.

This solution uses the Serverless Framework to handle deployment, so let’s walk through the code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
# The name of our service.
service: computeOptCheck

# No need to include the readme in the deployment package.
package:
  individually: true
  exclude:
    - README.md

provider:
  name: aws
  runtime: python3.7
  region: ${opt:region, '<your region>'}
  stage: ${opt:stage,'<your ci stage>'}
  owner: ${opt:owner,'<your name>'}
  ownerEmail: ${opt:owner,'<your email>'}
  app: ${opt:app,'computeOptCheck'}
  # This function required less than 100Mb of ram to run in testing, so we use the minimum memory setting.
  memorySize: 128
  # IAM role statements go here. These permissions will work , but you should lock them down to only the resources that require permissions.
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - compute-optimizer:GetEnrollmentStatus
        - config:PutEvaluations
      Resource:
        - '*'

functions:
  computeOptCheck:
    description: Check if compute optimizer is enabled.
    # This project is structured such that there is a folder in the project root directory called src containing the python file computeOptCheck.py
    handler: src/computeOptCheck.lambda_handler

resources:
  Resources:
    configPermissionToCallComputeOptCheckRule:
      Type: AWS::Lambda::Permission
      Properties:
        # When getting an attribute from a function, there is a standard naming convention:
        # the function name must be capitalized and you must append 'LambdaFunction' to it.
        FunctionName: !GetAtt ComputeOptCheckLambdaFunction.Arn
        Action: "lambda:InvokeFunction"
        Principal: "config.amazonaws.com"

    computeOptCheckRule:
      Type: AWS::Config::ConfigRule
      Properties:
        ConfigRuleName: ComputeOptimizerEnabledCheck
        Description: 'Check if the compute optimizer is enabled.'
        Source:
          Owner: "CUSTOM_LAMBDA"
          SourceDetails:
            - EventSource: 'aws.config'
              MessageType: 'ScheduledNotification'
          SourceIdentifier: !GetAtt ComputeOptCheckLambdaFunction.Arn
      DependsOn: configPermissionToCallComputeOptCheckRule

With the Serverless Framework handling the heavy lifting, we can focus on the essential parts of our application. The file above contains the configuration information for our serverless application, such as the name, memory size, and IAM permissions.

It’s also where we point to the lambda_handler method in our python file and define the resources we want to create, i.e. the Config rule and Lambda permission. The Lambda permission allows our Config rule to invoke our Lambda function, and the config rule is scheduled to call our function once per day by default.

Speaking of Lambda functions, let’s take a look at the business end of our application:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
import boto3
import json
import datetime

def evaluate_compliance():
    # Creates a compute-optimizer client object and gets the enrollment status of the account.
    compute_client = boto3.client("compute-optimizer")
    response = compute_client.get_enrollment_status()
    status = response['status']


    # If the enrollment status in the account IS NOT ‘Active’, set the compliance type to ‘NON_COMPLIANT’.
    if status != 'Active':
        compliance_type = 'NON_COMPLIANT'
    else:
        compliance_type = 'COMPLIANT'
    return compliance_type


def lambda_handler(event, context):
    # Start a config client so that we can interact with config rules.
    config_client = boto3.client("config")

    # Set the variables necessary for a put_evaluations() call to default values, then try to get the actual values out of the event passed to lambda_handler and print them for debugging.
    account_id = "No ID found."
    if "accountId" in event:
        account_id = event["accountId"]
    print('account id: ', account_id)

    invokingEvent = "No invoking event found."
    if "invokingEvent" in event:
        invoking_event = json.loads(event["invokingEvent"])
    print('invoking event: ', invoking_event)
    try:
        timestamp = invoking_event['notificationCreationTime']
    except:
        timestamp = datetime.datetime.now()

    rule_parameters = "No rule parameters found."
    if "ruleParameters" in event:
        rule_parameters = json.loads(event["ruleParameters"])
    print('rule params: ', rule_parameters)

    result_token = "No token found."
    if "resultToken" in event:
        result_token = event["resultToken"]
    print('Result token: ', result_token)


    # This sends the result of the compliance evaluation to the config rule, along with the metadata extracted from the event.
    config_client.put_evaluations(
        Evaluations=[
            {
                'ComplianceResourceType': 'AWS::::Account',
                'ComplianceResourceId': account_id,
                'ComplianceType': evaluate_compliance(),
                'OrderingTimestamp': timestamp
            },
        ],
        ResultToken=result_token
    )

There are two methods in this python file, lambda_handler() and evaluate_compliance(), so let’s tackle them individually.

lambda_handler(event, context):

This method serves as the entry point into our program while pulling double-duty as our main method. We first start a config client, then parse out all of the information required to run a successful put_evaluations() API call. The put_evaluations() method will push the results of our evaluation to the Config rule we set up earlier in the serverless.yml file. Inside of that API call, we call our evaluate_compliance() method to return the compliance status of the account.

evaluate_compliance():

This method determines whether the account this function is run in has Compute Optimizer enabled, and returns either ‘COMPLIANT’ or ‘NON_COMPLIANT’. It starts by creating a compute-optimizer client, then it runs a get_enrollment_status() API call and parses out the status value. If that status value is active, the method returns ‘COMPLIANT’, otherwise it returns ‘NON_COMLPIANT’.

And there you have it, a serverless application that sets up a Config rule to check if Compute Optimizer is enabled. According to the documentation, Compute Optimizer can reduce EC2 costs by up to 25%, which can add up to a significant chunk of change. Considering that there are no additional charges for this service, that’s something that’s worth checking on all of your accounts.


Orbit

Like what you read? Why not subscribe to the weekly Orbit newsletter and get content before everyone else?