Wherever possible, organizations of all sizes are moving to remote work these days, which can lead to a spike in
IT costs. To help mitigate this, we’re developing a suite of tests and checks to help our customers optimize their
accounts.
One handy cost-cutting AWS service is the Compute Optimizer. When enabled, this feature automatically checks your EC2
instances for incorrectly provisioned machines. Once enough data has been collected, the service will make recommendations
that should 1) help lower costs (over-provisioned) or 2) improve customer experience (under-provisioned).
We’ve developed a solution that creates a Config rule and a Lambda function to automatically check if the Compute
Optimizer is enabled in an account. Here’s how it works:
- An AWS Lambda permission is created to allow Config to trigger Lambda functions.
- A Config rule is created to trigger a Lambda function daily.
- A Lambda function that checks the status of Compute Optimizer is attached to the Config rule.
This solution uses the Serverless Framework to handle deployment, so let’s walk through the code:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
|
# The name of our service.
service: computeOptCheck
# No need to include the readme in the deployment package.
package:
individually: true
exclude:
- README.md
provider:
name: aws
runtime: python3.7
region: ${opt:region, '<your region>'}
stage: ${opt:stage,'<your ci stage>'}
owner: ${opt:owner,'<your name>'}
ownerEmail: ${opt:owner,'<your email>'}
app: ${opt:app,'computeOptCheck'}
# This function required less than 100Mb of ram to run in testing, so we use the minimum memory setting.
memorySize: 128
# IAM role statements go here. These permissions will work , but you should lock them down to only the resources that require permissions.
iamRoleStatements:
- Effect: "Allow"
Action:
- compute-optimizer:GetEnrollmentStatus
- config:PutEvaluations
Resource:
- '*'
functions:
computeOptCheck:
description: Check if compute optimizer is enabled.
# This project is structured such that there is a folder in the project root directory called src containing the python file computeOptCheck.py
handler: src/computeOptCheck.lambda_handler
resources:
Resources:
configPermissionToCallComputeOptCheckRule:
Type: AWS::Lambda::Permission
Properties:
# When getting an attribute from a function, there is a standard naming convention:
# the function name must be capitalized and you must append 'LambdaFunction' to it.
FunctionName: !GetAtt ComputeOptCheckLambdaFunction.Arn
Action: "lambda:InvokeFunction"
Principal: "config.amazonaws.com"
computeOptCheckRule:
Type: AWS::Config::ConfigRule
Properties:
ConfigRuleName: ComputeOptimizerEnabledCheck
Description: 'Check if the compute optimizer is enabled.'
Source:
Owner: "CUSTOM_LAMBDA"
SourceDetails:
- EventSource: 'aws.config'
MessageType: 'ScheduledNotification'
SourceIdentifier: !GetAtt ComputeOptCheckLambdaFunction.Arn
DependsOn: configPermissionToCallComputeOptCheckRule
|
With the Serverless Framework handling the heavy lifting, we can focus on the essential parts of our application.
The file above contains the configuration information for our serverless application, such as the name, memory size,
and IAM permissions.
It’s also where we point to the lambda_handler method in our python file and define the resources we want to
create, i.e. the Config rule and Lambda permission. The Lambda permission allows our Config rule to invoke our
Lambda function, and the config rule is scheduled to call our function once per day by default.
Speaking of Lambda functions, let’s take a look at the business end of our application:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
|
import boto3
import json
import datetime
def evaluate_compliance():
# Creates a compute-optimizer client object and gets the enrollment status of the account.
compute_client = boto3.client("compute-optimizer")
response = compute_client.get_enrollment_status()
status = response['status']
# If the enrollment status in the account IS NOT ‘Active’, set the compliance type to ‘NON_COMPLIANT’.
if status != 'Active':
compliance_type = 'NON_COMPLIANT'
else:
compliance_type = 'COMPLIANT'
return compliance_type
def lambda_handler(event, context):
# Start a config client so that we can interact with config rules.
config_client = boto3.client("config")
# Set the variables necessary for a put_evaluations() call to default values, then try to get the actual values out of the event passed to lambda_handler and print them for debugging.
account_id = "No ID found."
if "accountId" in event:
account_id = event["accountId"]
print('account id: ', account_id)
invokingEvent = "No invoking event found."
if "invokingEvent" in event:
invoking_event = json.loads(event["invokingEvent"])
print('invoking event: ', invoking_event)
try:
timestamp = invoking_event['notificationCreationTime']
except:
timestamp = datetime.datetime.now()
rule_parameters = "No rule parameters found."
if "ruleParameters" in event:
rule_parameters = json.loads(event["ruleParameters"])
print('rule params: ', rule_parameters)
result_token = "No token found."
if "resultToken" in event:
result_token = event["resultToken"]
print('Result token: ', result_token)
# This sends the result of the compliance evaluation to the config rule, along with the metadata extracted from the event.
config_client.put_evaluations(
Evaluations=[
{
'ComplianceResourceType': 'AWS::::Account',
'ComplianceResourceId': account_id,
'ComplianceType': evaluate_compliance(),
'OrderingTimestamp': timestamp
},
],
ResultToken=result_token
)
|
There are two methods in this python file, lambda_handler() and evaluate_compliance(), so let’s tackle them individually.
lambda_handler(event, context):
This method serves as the entry point into our program while pulling double-duty as our main method. We first start a
config client, then parse out all of the information required to run a successful put_evaluations() API call.
The put_evaluations() method will push the results of our evaluation to the Config rule we set up earlier in the
serverless.yml file. Inside of that API call, we call our evaluate_compliance() method to return the compliance status
of the account.
evaluate_compliance():
This method determines whether the account this function is run in has Compute Optimizer enabled, and returns
either ‘COMPLIANT’ or ‘NON_COMPLIANT’. It starts by creating a compute-optimizer client, then it runs a
get_enrollment_status() API call and parses out the status value. If that status value is active, the method returns
‘COMPLIANT’, otherwise it returns ‘NON_COMLPIANT’.
And there you have it, a serverless application that sets up a Config rule to check if Compute Optimizer is enabled.
According to the documentation, Compute Optimizer can reduce EC2 costs by up to 25%, which can add up to a significant
chunk of change. Considering that there are no additional charges for this service, that’s something that’s worth
checking on all of your accounts.
