If you’re not familiar with CloudWatch Logs its a feature of the CloudWatch service which allows us to persist logs from applications and operating systems on the AWS platform.
Once you’ve installed - and configured - the Unified CloudWatch Log agent, you’re able to gather logs from both EC2 and Server instances (on-premise).
By default, CloudWatch Logs stores log data indefinitely. This is fantastic, but we need to remember that we pay for log storage. While the costs are not high, this is one of those services that can quietly sneak up on you and end up costing a fair amount every month.
Before looking at how to configure log retention, let’s talk about some CloudWatch Logs terms.
Retention periods are set on each Log Group, and as always, can be done via the AWS Management Console, the CLI, or by using one of the AWS-provided SDKs.
Here’s an example of how you could set the retention period of a newly created CloudWatch Log Group using AWS CloudFormation.
NOTE: I’ve included KMS here - which is new(ish) to CloudFormation.
If you have existing CloudWatch Log Groups and need to update them my recommendation would be to script this. One idea may be to create a Lambda function which loops through all the Log Groups in a region and sets a standard retention period.
If you have just a few Log Groups, you can do this easily through the Management Console.
In the end, it’s up to you to decide on how to set retention periods on your Log Groups. My suggestion would be to establish a standard log retention policy and apply it universally across all your Log Groups.
Like what you read? Why not subscribe to the weekly Orbit newsletter and get content before everyone else?