How to Tag New AWS Resources Automatically and Deploy the solution with Terraform
A step-by-step guide for building an auto tag solution for new AWS resources & deploy with Terraform
Tagging cloud resources is considered a best practice and a crucial component of any organization’s cloud governance strategy. The benefits range from improved cost allocation or optimization strategy to ease of finding resources regardless of location within your infrastructure or classifying resources based on the security level.
Scenario
You’ve been tasked with implementing a solution to automatically tag new AWS resources with the identity of the user or role that provisioned the resource. For example, apply a tag name such as CreatedByUser and a tag value such as Fisher Price. The tags from the user or role such as the contact email, team, etc should also be applied to new resources. Finally, the solution should be deployed using Terraform.
Solution
The solution adopted here consists of these 3 services; AWS Cloudtrail, AWS EventBridge, and AWS Lambda. When a new AWS resource is created a corresponding event will be logged by AWS Cloudtrail. This event will be captured by Eventbridge and trigger a rule set to monitor resource creation events. The triggered rule will send the event to a Lambda function which will tag each new resource with the name of the IAM user or role that created the resource and any associated tags applied to that identity.
As a use case, we will focus on tagging newly created AWS Simple Notification Service (SNS) Topics but the same principles can be applied to other AWS resource types.
Let’s get started 🍿
Pre-requisites
This guide assumes you have a basic knowledge of Terraform, AWS services, and python. For this tutorial, you will need:
- Terraform CLI (1.0+) installed
- An AWS account and an IAM user with admin permissions
- AWS CLI (2.0+) installed and configured with access keys
Note: Destroy the infrastructure at the end of this tutorial to avoid unnecessary charges.
Creating the Infrastructure with Terraform
The terraform config files and source code used in this post are available on my GitHub repository.
1. Declare required providers and region of deployment
Let’s declare three providers (& their required versions) and terraform CLI version to be used. We will also apply a default set of tags to all resources deployed in this project and set the region to deploy our resources.
2. Define a set of variables for our terraform configuration
This solution requires creating a Cloudtrail trail that logs management events. If your account already has a trail configured, set the create_trail variable value to false otherwise leave it at the default value of true.
3. Create a Cloudtrail trail if none exists in your account
The configuration below is used to create a trail. You will also need to create an S3 bucket with the required policy that allows Cloudtrail to write log files. Your bucket name must be globally unique. Here, we will use the random_pet provider previously declared to generate a unique name for the bucket.
4. Create a policy for the Cloudtrail S3 bucket
For Cloudtrail to write to the bucket, at a minimum it needs permissions to carry out a PutObject and GetBucketAcl API action. You can also implement finer restrictions for the API call to be made from the Cloudtrail created in the previous step.
5. Create Eventbridge Rule
The solution requires Eventbridge to monitor desired resource creation events. Hence, create the rule with the specified pattern below to monitor newly created SNS Topics. If for example, you plan to auto-tag new S3 buckets specify the CreateBucket API action as the event name and s3.amazonaws.com as the event source. The event will be sent to a lambda function, as a result, the target for the rule is a lambda function as shown. Eventbridge will also require permission to invoke the lambda function.
6. Create IAM Role and policy for the Lambda function
As mentioned earlier, our objective is to auto-tag SNS Topics and add tags from the user or role that created the resource. Hence, we will allow these three API actions SNS:TagResource, iam:ListRoleTags and iam:ListUserTags to be made by our lambda function. We will also add permissions required for the Lambda function to write Cloudwatch logs.
7. Create lambda function and Log group
Let's create the lambda function resource, declare python to be used as the runtime and create the log group for the Lambda function to write logs to.
8. Create an S3 bucket to store the Lambda deployment package
The Lambda function defined in the previous step gets the source code from an S3 bucket, so we will need to create the bucket and an S3 object (which is a zipped file containing our code.) Bucket names again must be unique and we will maintain uploaded revisions of our source code hence the use of bucket versioning.
One of the many reasons I love terraform is the availability of various providers for several purposes. In this case, we will use the archive_file provider declared in step 1 to zip the source code file. The consequence of the configuration below is, each time our source code is updated and terraform applyis run, a new S3 object will be uploaded and the lambda function code will be updated. Isn’t that cool? 🌟
9. Finally onto the heart 💪 of our solution, write the Lambda function
Once the lambda_handler() function receives the event, it will carry out 4 main tasks per our objective
- Get the IAM user or role name that created the new resource
- Get the tags applied to that IAM user or role
- Get the arn of the new resource. In this case, an SNS Topic arn
- Create and apply the tags to the resource using its arn
The event received contains a lot of useful information and we use it in the get_resource_tag() function to also get the date & time the resource was created and apply that as a tag to the new resource.
Depending on your requirement, in our case, the user or role tags will be obtained using the get_iam_identity_tags() function and applied to the new resource as well.
Deploy the Solution
Now to the fun part, let's deploy our solution 😄
Before you can deploy, you must provide the credentials for the AWS provider you’ve declared in step 1. One option is to export your IAM user’s access keys as environment variables as shown below or set up a profile after installing the AWS CLI.
$ export AWS_ACCESS_KEY_ID="anaccesskey"
$ export AWS_SECRET_ACCESS_KEY="asecretkey"Run the commands below to clone my GitHub repository, initialize the provider plugins (and backend), view the deployment plan, and deploy our solution.
git clone https://github.com/yemisprojects/aws-autotag.git
cd aws-autotag
terraform init
terraform plan
terraform apply --auto-approveYou should see 15 resources added if you elected to create a Cloudtrail in step 3.
I love seeing that green status message when the terraform apply completes successfully, it almost feels therapeutic.
Test the Solution
To test what we have deployed, let’s run these commands.
- Tag an IAM user with these four tags
aws iam tag-user --user-name admin \ --tags Key=ContactEmail,Value=”adminuser@example.com” \ Key=Team,Value=Marketing- Create a dummy SNS Topic using the same IAM user
aws sns create-topic --name dummy_topic| cat- Confirm what tags are applied after the resource is created
aws sns list-tags-for-resource \
--resource-arn "arn:aws:sns:us-west-1:<account_id>:dummy_topic"|catThe expected result is the SNS Topic should have these four tags: a ContactEmail, Team, CreatedByUser, and DateCreated tag as shown below.
Clean up your infrastructure
Once you are done reviewing the deployed infrastructure and testing, remember to clean up the infrastructure by running the command below.
terraform destroy --auto-approveCongratulations on creating and deploying a solution to auto-tag new AWS resources 🎉👏. I hope you learned something new.
Feedback and Future Posts
This was my first published post, so I’d appreciate some feedback for improvement. In my next post, we will build on this solution and create a custom solution using AWS config to track resource compliance, auto-remediate non-compliant resources, and notify the resource creator via email.
More content at PlainEnglish.io. Sign up for our free weekly newsletter. Follow us on Twitter, LinkedIn, YouTube, and Discord. Interested in Growth Hacking? Check out Circuit.
