Create an Overlay

The shop wants checkout to publish onto the order queue. That fact is about two models, and it sits comfortably on neither: put it on the server and servers know about queues, put it on the queue and queues know about IAM roles. An overlay is the node for facts like that.

The clearest way to learn one is to read a working one, so this page walks through AwsCloudFrontLambdaOriginOverlay from @quadnix/octo-aws-cdk. It is the fact that a CloudFront distribution serves a Lambda function: the distribution needs the function's URL as an origin, and the function has to let the distribution call it. Two modules build the two models, and the overlay is what joins them.

octo create-overlay -n shop-queue-service --overlay shop-queue-access -t service --package @shop

-n is the module the overlay lives inside; --overlay is the overlay's own name. Overlays belong to a module because a module is what builds them. This one lives in the distribution's module, AwsCloudFrontServiceModule.

The schema

export class AwsCloudFrontLambdaOriginOverlaySchema extends BaseOverlaySchema {
  override properties = Schema<{ originId: string }>();
}

The properties are the relationship and nothing else. No model, no resource, no class from either side: an overlay is serialized and read back like any other node, so what it holds has to be plain data. Here that is one string, the distribution's origin the function is served from. Which function it is, the overlay already knows from the anchor it joins.

The class

@Overlay('@octo', 'aws-cloudfront-lambda-origin-overlay', AwsCloudFrontLambdaOriginOverlaySchema)
export class AwsCloudFrontLambdaOriginOverlay extends AOverlay<
  AwsCloudFrontLambdaOriginOverlaySchema,
  AwsCloudFrontLambdaOriginOverlay
> {
  declare anchors: [AwsCloudFrontAnchor, MatchingAnchor<AwsLambdaFunctionAnchorSchema>];
  declare properties: AwsCloudFrontLambdaOriginOverlaySchema['properties'];

  constructor (
    overlayId: string,
    properties: AwsCloudFrontLambdaOriginOverlaySchema['properties'],
    anchors: [AwsCloudFrontAnchor, MatchingAnchor<AwsLambdaFunctionAnchorSchema>],
  ) {
    super(overlayId, properties, anchors);
  }

  override async diffAnchors (): Promise<Diff[]> {
    return [];
  }
}

AOverlay's constructor is protected, so the subclass declares a public one that does nothing but widen it. Without it the new below does not typecheck.

The anchors parameter is the interesting line. This overlay joins exactly two, in a fixed order, and typing them as a tuple means nobody can build one with the anchors swapped or missing. Declaring the same tuple on the class lets the action read them back typed.

One is a class, AwsCloudFrontAnchor, because the distribution's module owns both the overlay and that anchor. The other is a MatchingAnchor<...Schema>, because it came from somewhere else and was found by matching a shape. That is the difference in a nutshell: use the class when you built it, and the schema when somebody else did.

The resources an overlay's action creates sit where the models your own module created sit, and their resource IDs carry a hash of that place. Models the overlay borrows from other modules, like the function here, are what it joins and not where it is, so joining one more, such as granting a function a table in another region, changes no ID. An overlay that joins only borrowed models sits where all of them sit.

Building it

In the module, once both anchors are in hand:

const [matchingLambdaFunctionAnchor] = await origin.source.getAnchorsMatchingSchema(
  AwsLambdaFunctionAnchorSchema,
  [],
  { searchBoundaryMembers: false },
);

overlays.push(
  new AwsCloudFrontLambdaOriginOverlay(
    `aws-cloudfront-lambda-origin-overlay-${inputs.distributionName}-${origin.originId}`,
    { originId: origin.originId },
    [cloudFrontAnchor, matchingLambdaFunctionAnchor],
  ),
);

Then return it from onInit along with everything else the module built.

The overlay ID has to be the same on every run for the same relationship, so it is built from the two things that define this one: the distribution's name and the origin's ID, which the user chose and which is unique within the distribution. Serve the same origin from the same distribution next week and the ID comes out identical. The function's name would not do: two regions may each run a function of one name, and one distribution may serve both.

Adding the anchors also creates the dependency edges, so the overlay depends on both parent models and disappears when either does. An overlay cannot hold an anchor of its own; Octo throws on a node pointing at itself.

The action

An overlay is diffed like any other node, and its action says what the relationship costs:

@Action(AwsCloudFrontLambdaOriginOverlay)
export class AddAwsCloudFrontLambdaOriginOverlayAction implements IModelAction<AwsCloudFrontServiceModule> {
  filter (diff: Diff): boolean {
    return (
      diff.action === DiffAction.ADD &&
      diff.node instanceof AwsCloudFrontLambdaOriginOverlay &&
      hasNodeName(diff.node, 'aws-cloudfront-lambda-origin-overlay') &&
      diff.field === 'overlayId'
    );
  }

  async handle (diff, actionInputs, actionOutputs): Promise<ActionOutputs> {
    const { originId } = diff.node.properties;
    const { distributionName } = actionInputs.inputs;
    const [, matchingLambdaFunctionAnchor] = diff.node.anchors;
    const { functionName } = matchingLambdaFunctionAnchor.getSchemaInstance().properties;

    const cloudFrontDistribution =
      actionInputs.resources[`cloudfront-${distributionName}`] as CloudFrontDistribution;

    const [matchingLambdaFunction] = await matchingLambdaFunctionAnchor
      .getActual()
      .getParent()
      .getResourcesMatchingSchema(LambdaFunctionSchema, [{ key: 'functionName', value: functionName }], [], {
        searchBoundaryMembers: false,
      });
    if (!matchingLambdaFunction) {
      throw new OverlayActionExceptionTransactionError(
        `No Lambda function found for "${functionName}"!`, diff, this.constructor.name,
      );
    }

    const lambdaFunction = matchingLambdaFunction.getActual() as LambdaFunction;
    lambdaFunction.properties.allowCloudFrontOriginAccess = true;
    cloudFrontDistribution.addLambdaOrigin(originId, matchingLambdaFunction);

    actionOutputs[lambdaFunction.resourceId] = lambdaFunction;
    actionOutputs[cloudFrontDistribution.resourceId] = cloudFrontDistribution;
    return actionOutputs;
  }
}

Note the two ways it reaches a resource. The distribution came from the same module, so it is in actionInputs.resources under the ID the module's own model action gave it. The function came from a different module, so the overlay goes through the function's anchor to the node that anchor belongs to, and asks that node, and only that node, for a resource matching LambdaFunctionSchema. That second path is how one module changes another module's resource without importing its classes. Searching the whole boundary by name instead would find the wrong function as soon as another region runs one of the same name.

Both resources go into actionOutputs. That is what puts the permission in the function's folder and the origin in the distribution's folder, from one relationship.

Which side you may write on

This overlay writes both halves because one library owns both: LambdaFunction carries allowCloudFrontOriginAccess because the same author wrote the distribution that needs it. A resource decides which grants it is willing to carry, and that decision belongs to whoever wrote it. So an overlay joining your model to somebody else's can only ask the other side for what its resource already offers.

The shop's queue is the case. Nothing on an IAM role says "may publish to a queue", so a queue access overlay cannot write the role half. It writes the grant on the queue instead, as a queue policy naming the role's ARN, which needs nothing from the role beyond an ARN it already publishes. That is the same shape as the function half above, a grant on the resource being reached that names who may reach it, and for the queue it is the half your own module owns.

Reaching for a grant the other resource does not offer is a sign you are writing the wrong module. Either the resource's author adds it, or the thing you need is a resource of your own.

Modifying rather than creating

An overlay action often does what this one does: reach existing resources and change them, rather than declare new ones. That is normal, and it is why an overlay's action is usually the shortest file in a module and the one that carries the most meaning.

Note also the failure: OverlayActionExceptionTransactionError when the function cannot be found. Throwing carries the diff into the error, so the message names the node and field, rather than producing an origin that points at nothing.

When not to write one

Only for facts that genuinely span two models. A property of one model with extra configuration is a property, and an overlay with one anchor is a node with no reason to exist.

The test worth applying: would putting this on either model teach that model something it should not know? If yes, it is an overlay. If no, it is a field.

The schema​

The class​

Building it​

The action​

Which side you may write on​

Modifying rather than creating​

When not to write one​