Blog
Troubleshooting access to S3 buckets in AWS can be frustrating at times, especially given the generic nature of AWS Access
Denied errors. These errors often stem from misconfigured policies on the S3 bucket or IAM principal. However, I have
encountered a few more exotic cases in the field.
On the rare occasion that the issue is not policy-related, there are three places that I like to check:
- Does the bucket have server-side encryption (SSE) configured with a customer-managed KMS Key (CMK)?
- Does the traffic to the S3 bucket pass through any VPC S3 endpoints?
- Are you trying to configure a Public S3 bucket when all public access is blocked at the account level?
Server-Side Encryption
If an S3 bucket is configured with SSE and is using a CMK, make sure that the principals who require access to the bucket have been granted the ability to use the following actions in the KMS key policy:
Mar 11, 2024
Blog
I encountered this issue while attempting to reset the password of a user on a Windows server 2016 managed node using
AWS Systems Manager. In the console, I could start the workflow and enter a replacement password. However, I was not
prompted to confirm the change.
Unfortunately, that meant I could no longer access the EC2 Instance. On top of that, I couldn’t RDP to the target
instance with the new password. Thankfully, I could work around this console bug using the AWS CLI.
Jan 3, 2023
Blog
This bug is specific to deploying CloudFormation templates through a CI/CD pipeline, in this case Gitlab. If you are using
the intrinsic function Fn::Import statements in your templates, you may have run into this error message
"Template error: the attribute in Fn::ImportValue must not depend on any resources, imported values, or Fn::GetAZs"
If so, the likely culprit is a reference typo within your import statement. However, if you’ve given the template a
once-over and haven’t found any errors, consider checking the deployment script. When I make use of the Import function
in CloudFormation, I often pass a stack name from the CI script to the template as a parameter to allow the name of the
import target to be dynamically constructed. This allows for more flexibility in using the template, but introduces more
opportunities for typos. If there is an error in that parameter, CloudFormation will be unable to resolve the Import, and
will throw the above error.
Nov 20, 2022
Blog
In the same vein as Bug Byte 1, here is another encryption permission issue you may encounter while working with AWS Key
Management Service (KMS). Have you seen an error like the one below in CloudFormation while creating a KMS-encrypted AWS resource?
"The specified KMS key does not exist or is not allowed to be used"
If so, there’s a chance that the key policy assigned to your Customer Managed Key (CMK) could be the culprit. It is possible
that the service you are trying to encrypt does not have access to the KMS key in question.
Nov 5, 2022
Blog
Have you ever encountered an obscure error like the one below in the AWS Autoscaling Console?
Client.InternalError: Client error on launch.
Often vague errors like these can be traced back to encryption-related permissions issues. AWS intentionally obfuscates
Key Management Service (KMS) errors to prevent malicious actors from gaining information from error messages, which can
sometimes make debugging a frustrating experience.
If the Amazon Machine Image (AMI) you are using for your Auto Scaling Group is encrypted with a KMS key, and you are
seeing this issue in your Autoscaling Group, check the policy on your KMS key. It should allow access to the KMS key from
the AutoScaling Service Role, like so:
Oct 26, 2022
