Why your CDK deploys might be slow - tags propagate to children

I noticed quite some time ago that my CDK deployments had gotten slow, and that a lot of the churn was SQS. I did not really have time to dig into it, and I half-convinced myself it was just a feeling - that it had not actually slowed down. Eventually it nagged at me enough that I sat down with an LLM agent to investigate. The culprit turned out to be a one-liner buried in a construct I use everywhere.

Disclaimer

This article was put together by an LLM agent, not written by me word for word. The idea, the opinions and all the technical content are mine - I just did not have the time to write it up, so I asked an agent to assemble it from my earlier posts. The agent did not come up with any of this. As with every generated article on this blog, I have proofread and edited it myself before publishing.

The symptom

I have a construct that tags all my Lambda functions with the current git revision. It is a nice little habit - when something misbehaves in production you look at the function's revision tag and you instantly know which commit is running. Simplified, the helper behind it looks like this:

export function tagFunctionsWithRevision(stack: Stack): void {
  const revision = getRevision(); // `git rev-parse --short HEAD`
  Aspects.of(stack).add(new TagFunctionsWithRevisionAspect({ revision }));
}

class TagFunctionsWithRevisionAspect implements IAspect {
  constructor(private readonly props: { readonly revision: string }) {}

  public visit(node: IConstruct): void {
    if (node instanceof Function) {
      Tags.of(node).add("revision", this.props.revision);
    }
  }
}

Looks innocent, right? It is an Aspect that walks the construct tree, and for every Lambda Function it adds a revision tag. The name says it all - it tags functions. Boy was I naive again.

I had a hunch going in that the tagging was behind it, so I pointed the agent straight at that code and gave it read-only access to my staging AWS account. The churn showed up identically in staging and production, so staging was a safe place to let it dig. It pulled the deployed template and counted which resources actually carried the tag. This is what came back:

 10 / 10   AWS::SQS::Queue
 11 / 11   AWS::SNS::Topic
 11 / 11   AWS::CloudWatch::Alarm
 12 / 13   AWS::IAM::Role
  5 /  5   AWS::Lambda::EventSourceMapping
 12 / 13   AWS::Lambda::Function

The revision tag was on everything. Not just the functions. And because the revision string changes on every commit, CloudFormation saw a changed Tags property on ~60 resources on every deploy and dutifully issued an in-place UPDATE for each one. That was the SQS churn I had been squinting at. It was real, not a feeling.

Why it happens

So my hunch about the tagging was right - but what the tagging was actually doing caught me off guard. Here is the part I had wrong in my head. I thought Tags.of(node).add(...) tags that node. It does - and then it propagates the tag down to every taggable child construct underneath it. I did not know CDK cascaded tags like that.

That matters because of where the queues live in the construct tree. My SQS handler construct creates the input queue, the DLQ, the DLQ alarm and the error-alarm topic as children of the Lambda function. If you look at the aws:cdk:path of a queue in the synthesized template, it reads something like:

MyStack/Api/Tasks/MainHandler/Input/Queue/Resource
                   ^^^^^^^^^^^ the Lambda Function construct
                               ^^^^^^^^^^^^ the queue is a CHILD of it

So even though the aspect only ever calls .add() on Function nodes, CDK happily cascades that tag onto the queue, the DLQ, the topic and the alarm nested beneath each function. Tag propagation flows down the construct tree. The if (node instanceof Function) guard filtered which nodes it calls add on - it did nothing to limit where the tag ends up.

This is the same runtime-vs-deployment-time trap I wrote about before, just wearing a different hat. The abstraction is so comfortable that you forget there is a tree underneath, and the tree has rules of its own.

The fix

CDK's Tags.add takes a TagProps with includeResourceTypes and excludeResourceTypes. If you only want the tag on the function resource and nowhere else, whitelist the CloudFormation resource type:

public visit(node: IConstruct): void {
  if (node instanceof Function) {
    Tags.of(node).add("revision", this.props.revision, {
      includeResourceTypes: ["AWS::Lambda::Function"],
    });
  }
}

Now the tag lands only on AWS::Lambda::Function resources. The queues, DLQs, topics, alarms and roles stop churning. The functions still get a changing revision tag on every commit - which is fine, their code asset already changes when the code changes, so there is no extra churn beyond what you would expect anyway.

To prove it rather than trust it, the agent added a test that nests a queue under a tagged function and asserts the queue does not inherit the tag:

test("does not propagate the revision tag to child constructs", () => {
  const stack = new Stack(new App(), "TestStack");
  const fn = new Nodejs24Function(stack, "Fn", {
    entry: `${prebuiltDir}/`,
    handler: "index.handler",
  });
  new Queue(fn, "InputQueue"); // child of the function on purpose
  tagFunctionsWithRevision(stack);

  const template = Template.fromStack(stack);
  const queue = Object.values(template.findResources("AWS::SQS::Queue"))[0];
  const tags = queue?.Properties?.Tags ?? [];
  expect(tags.some((tag) => tag.Key === "revision")).toBe(false);
});

A tag or an env var?

I will be honest: while debugging this I had no grand thoughts about tagging strategy, I just wanted the churn gone. It was the agent that floated the alternatives. One of them was to set the revision as an environment variable on the function instead:

fn.addEnvironment("REVISION", getRevision());

Env vars do not propagate - they live on the function and nowhere else - and you can read the value at runtime (process.env.REVISION) which is arguably more useful than a tag you can only see in the console. The trade-off is that an env-var change updates the function on every commit even when the code is byte-for-byte identical.

The more interesting tension is coupling. The moment the revision becomes an env var on my Nodejs24Function construct, every consumer of that construct inherits a dependency on git: either the deploy hard-fails when there is no git history, or it quietly ships a placeholder. That is a real decision to force on people - it is a single config flag (enforceGit: fail loudly on "no git", or carry on safely with a fallback string), but it is still a flag they now have to think about. The tag-via-aspect approach has no such reach: it is applied at the stack level, touches nothing in the construct's public surface, and nobody downstream has to care. For a pure traceability marker, the non-coupling option wins, so I left the env var out of the construct. I might add it later as an opt-in - and if you want the revision in your logs, do both - but that is a feature with a flag, not a bug fix. The bug here was never "tag vs env var", it was propagation.

One mistake rarely travels alone

With the tag churn fixed, I remembered this was not the only thing about deploying these projects that had quietly annoyed me. So within the same debug session I pointed the agent at the next problem: every alarm created its own SNS topic with its own email subscription to the same address. Eleven alarms meant eleven topics, eleven subscriptions, and eleven confirmation emails you have to click. Collapsing that to a single shared topic + subscription cut a pile of duplicate resources and the confirmation-email spam. Worth a look if you have an alarm-per-handler construct too - just remember that moving the topic changes its construct path, so the old ones get replaced on the next deploy and the new subscription needs confirming.

Both of these live in our @beesolve/cdk-constructs package, so if you use it you get the fixes for free.

Takeaway

If CloudFormation keeps updating resources you never touched, look at your tags. A single Tags.of(someConstruct).add(...) can quietly cascade a value that changes on every build onto half your stack, and every one of those resources becomes an UPDATE on every deploy. Scope your tags with includeResourceTypes / excludeResourceTypes, and when in doubt, synth the template and grep for the tag - the template never lies.

I kept the code above simplified to make the point. The real, fixed helper - includeResourceTypes and all - lives in @beesolve/cdk-constructs, together with the Nodejs24Function construct and the shared-alarm-topic setup from the previous section. The package is public, so if any of this sounds like a wheel you were about to reinvent: grab it, read it, use it. You do not have to make the same mistake first.