Skip to main content

AWS Module Catalog

@quadnix/octo-aws-cdk is our own implementation layer for AWS. It is a usable library, and it is a worked example of how to build one: every module in it is written with the same pieces described in Anatomy of a Module, and nothing in it is privileged by the framework. A region module you write yourself sits alongside these on equal terms.

That matters when reading this page. What follows is a set of answers one team gave to questions like "what is a region made of", not the answers. Where a module does not fit your account layout, the move is to write your own and keep the rest.

Each entry lists what a caller passes and what appears in AWS. The generated API documentation has every input in full.

Apps and accounts​

An app produces no cloud resources. An account decides which credentials to use and which provider block is rendered, and produces none either. Those credentials are also what Octo authenticates with where it calls AWS itself rather than through Terraform, which today is the image publishing below.

ModuleInputsCreates
SimpleAppModulename
AwsIniAccountModuleaccountId, app, iniProfile?, endpoints?
AwsCredentialsAccountModuleaccountId, app, credentials, endpoints?
AwsLocalstackAccountModuleapp, endpoints?
AwsMotoAccountModuleapp, endpoints?

AwsIniAccountModule reads a named profile from ~/.aws/credentials, which is what the templates use. AwsCredentialsAccountModule takes credentials directly, for CI. The Localstack and Moto modules point the provider at a local emulator, and exist for testing.

imports:
- className: AwsIniAccountModule
importPath: "@quadnix/octo-aws-cdk/modules/account/aws-ini-account"

Every import path follows that shape: the package, then modules/<model-type>/<module-name>.

Regions and subnets​

ModuleInputsCreates
AwsSingleAzRegionModuleaccount, name, regionId, vpcCidrBlockVPC, internet gateway
AwsMultiAzRegionModuleaccount, name, regionIds, vpcCidrBlockVPC, internet gateway
AwsServerlessRegionModuleaccount, name, regionId
AwsSimpleSubnetModuleregion, subnetName, subnetCidrBlock, subnetAvailabilityZone, subnetOptions?, subnetSiblings?, localFilesystems?subnet, route table, network ACL, security group, NAT gateway, EFS mount target
AwsServerlessSubnetModuleregion

Both region modules define a region as one VPC with an internet gateway. They differ in what the region publishes: a single-AZ region advertises one availability zone, a multi-AZ region advertises several, and that is what a subnet module reads when deciding where it can place itself. Their region IDs are Octo's own, aws-eu-west-1a, distinct from the AWS region.

AwsServerlessRegionModule is a region with no network: it creates nothing, and says only which AWS region its services are created in, as aws-us-east-1. It publishes a different shape from the other two, so a subnet or anything else that needs a VPC refuses it when the modules are composed. The static website sits in one, and a storage bucket, a DynamoDB table or an image takes a region of any of the three kinds. A serverless region holds one subnet, AwsServerlessSubnetModule, which also creates nothing: it is always named default, covers the whole region, and is there because an execution is always placed in a subnet.

The subnet module is where the network shape actually lives. subnetOptions carries createNatGateway, disableSubnetIntraNetwork, and subnetType; subnetSiblings is how a private subnet asks to route through a public one's NAT gateway.

A NAT gateway costs money by the hour

createNatGateway: true is the one input in this catalog with an ongoing bill attached, whether or not anything is running behind it.

Servers, deployments, and executions​

ModuleInputsCreates
AwsServerModuleaccount, serverKey, s3?, dynamodb?, lambda?, iamStatements?, securityGroupRules?
AwsDeploymentModuleserver, deploymentTag, image? and imageTag? or imageUri?, command?, ports?, cpu?, memory?, timeoutInSeconds?
AwsEcsEnvironmentModuleregion, environmentName, environmentVariables?ECS cluster
AwsServerlessEnvironmentModuleregion, environmentName, environmentVariables?
AwsEcsExecutionModuledeployments, environment, subnet, executionId, desiredCount, environmentVariables?, filesystems?, logRetentionInDays?, securityGroupRules?IAM role, log group, ECS service, task definition, security group
AwsLambdaExecutionModuledeployment, environment, subnet, executionId, environmentVariables?, invokeMode?, logRetentionInDays?, reservedConcurrency?IAM role, log group, Lambda function, function URL

These are one story told in parts. A server is the logical microservice: what its code may reach, and its network rules. A deployment is one version of its code: the container image, and what the code needs to run. An environment is a stage inside a region. An execution is a deployment running in an environment in a subnet, and it is the only one of them that creates anything: an ECS service, or a Lambda function, and the IAM role the code runs as, holding what the server may reach. The server and the deployment know nothing about where the code runs, so the same two run on ECS or on Lambda, and a server run in two environments gets a role in each. The role is named after executionId, and IAM role names are shared by every region of an account, so an executionId must be unique within its account: running the same execution in two regions takes two names, such as checkout-v1-staging-us and checkout-v1-staging-eu.

A new version of the code is a new deployment, run by a new execution with an executionId of its own, such as checkout-v2-staging. The old execution is not changed: both versions run side by side, each with its own role, until you take the old execution out of your definition file. That is what lets a release be gradual: move traffic to the new version, and back again if it misbehaves.

AwsServerModule's s3 input is the shop's receipts grant from Shop Stack: a list of storage services and, per service, the access prefixes the server may access with READ, WRITE, or READ_WRITE. dynamodb is the same for tables, each a DynamoDB service and one of those three, where read covers the table's indexes. Each grant is an overlay joining the server to the bucket or table, and creates nothing on its own. The execution running the server turns it into a policy on its role, and for a bucket into a statement on the bucket. The role is created in the execution's account, so every bucket and table a server is granted must be in that account too.

lambda lets the server's code invoke functions, each named by its execution from AwsLambdaExecutionModule, directly or through its URL, and read that URL. The function may be in another account, such as one an extends state or a store holds. The execution running the server writes one policy on its role for all of them, and admits the role to each function in the function's account, so the definition needs a Terraform provider for that account too.

iamStatements grants what Octo does not manage, such as an SES identity made by hand. Each statement is a list of actions and a list of resources, by ARN, and is allowed as written: Octo checks neither, and the resources may be in any account. The execution writes all of a server's statements into one policy on its role:

  - moduleId: checkout-server-module
moduleClass: AwsServerModule
moduleInputs:
account: "${{account-module.model.account}}"
serverKey: checkout
iamStatements:
- actions: [ses:SendEmail, ses:SendRawEmail]
resources: [arn:aws:ses:eu-west-1:123456789012:identity/shop.example.com]

A deployment names its image one of two ways. An image Octo publishes is named by image, from AwsEcrImageModule, and imageTag, from AwsEcrImageTagModule, which orders the push before anything that runs it. Any other image is named by imageUri, and only ECS runs one:

  - moduleId: checkout-deployment-v1-module
moduleClass: AwsDeploymentModule
moduleInputs:
server: "${{checkout-server-module.model.server}}"
deploymentTag: 0-0-1
image: "${{checkout-image-module.model.image}}"
imageTag: 0.0.1
command: [node, main.js]
ports:
- containerPort: 80
protocol: tcp
cpu: 256 # ECS: 256, 512, 1024, 2048, 4096, 8192, or 16384
memory: 512 # MiB
timeoutInSeconds: 30

Every field but the image is optional on the deployment, and the execution checks the ones its runtime needs, failing before anything is created: ECS needs cpu, memory, and a command, and Lambda needs memory from 128 to 10240, a timeoutInSeconds up to 900, and an image Octo publishes. The function runs on the processor that image is built for. deploymentTag is two to thirty-two characters of letters, numbers, hyphens, and underscores. A dot is not one of them, so a version reads as 0-0-1. command is a list with one argument per element, as a Dockerfile's CMD [...] is, so an argument may hold a space.

An ECS execution names one main container and any sidecars, each pairing a deployment with the container properties that deployment runs under. Every container in the task runs as one role, the one holding the main deployment's server grants, so a sidecar's deployment must run on a server that grants nothing:

  - moduleId: checkout-v1-staging-execution-module
moduleClass: AwsEcsExecutionModule
moduleInputs:
executionId: checkout-v1-staging
desiredCount: 1
environment: "${{staging-environment-module.model.environment}}"
subnet: "${{public-subnet-module.model.subnet}}"
deployments:
main:
containerProperties:
image:
command: [/bin/sh]
essential: true
name: checkout-v1
ports:
- containerPort: 80
protocol: tcp
deployment: "${{checkout-deployment-v1-module.model.deployment}}"
sidecars: [ ]

Every container writes its logs to the execution's log group, /ecs/<executionId>, under a stream prefix of its container name. The log group keeps them for logRetentionInDays, 30 days unless set.

Each execution's tasks run in one security group of its own, holding its securityGroupRules together with those of the server of every deployment it runs, main and sidecars. securityGroupRules is optional, and leaving it out gives the execution one egress rule allowing all outbound traffic, which is what AWS gives a security group you create by hand. A Fargate task has to reach ECR to pull the image it runs, so an execution that could reach nothing could never start. Declare an egress rule of your own and that default is not added, because you have said what this execution may reach:

      securityGroupRules:
- CidrBlock: 10.0.0.0/16
Egress: true
FromPort: 443
IpProtocol: tcp
ToPort: 443

A Lambda execution runs one deployment in a serverless region, as a function named executionId, so every version gets a function and a URL of its own. The image must be published in the function's own region, and a web server in it answers Lambda's requests through the Lambda Web Adapter, which the image carries. The adapter forwards each request to the deployment's port, so the deployment declares at most one, over TCP, and without one the adapter uses 8080. Every function gets a URL that answers only requests signed with AWS credentials; a server's lambda grant admits its execution's role to call it. The execution itself names only what it runs:

  - moduleId: checkout-v1-staging-execution-module
moduleClass: AwsLambdaExecutionModule
moduleInputs:
executionId: checkout-v1-staging
deployment: "${{checkout-deployment-v1-module.model.deployment}}"
environment: "${{staging-environment-module.model.environment}}"
subnet: "${{serverless-subnet-module.model.subnet}}"

The URL takes a request body of at most 6 MB, about 4.5 MB of binary once base64-encoded, and returns each response whole, up to 6 MB. Set invokeMode: RESPONSE_STREAM to stream the response as the server writes it, which lifts the limit to 200 MB; Octo then tells the adapter to stream as well, through AWS_LWA_INVOKE_MODE.

reservedConcurrency caps how many instances of the function run at once, and reserves them out of the account's concurrency so no other function can take them. Left out, nothing is reserved and nothing is capped. AWS keeps 10 of an account's concurrency unreserved, so an account whose limit is still 10, as a new account's often is, cannot reserve any: raise the limit first.

The function writes its logs to a log group of its own, /aws/lambda/<executionId>, which keeps them for logRetentionInDays, 30 days unless set. Its role may write to that log group and create none, so Lambda never makes one outside Octo.

executionId is at most fifty characters, because it names the role too. A function runs outside any VPC, so it ignores the server's securityGroupRules, which only an ECS execution applies. One server can run on both.

A Lambda execution needs the hashicorp/aws provider 6.28.0 or later, so set minVersion under terraform.requiredProviders to at least that. An older provider fails the plan with An argument named "invoked_via_function_url" is not expected here, because the permissions on a function's URL use it.

A new role can take minutes to become assumable, so a first apply may fail with "role cannot be assumed". Finish the failed run the way Octo prints, and run again before looking further.

Services​

ModuleInputsCreates
AwsS3StorageServiceModuleregion, bucketName, accessPrefixes?, blockPublicAccess?, corsRules?, directoryPath?, filter?, lifecycleRules?, transform?, subDirectoryOrFilePath?S3 bucket, public access block, bucket policy, CORS configuration with corsRules, lifecycle configuration with lifecycleRules, and with directoryPath one object per file
AwsS3StaticWebsiteServiceModuleregion, bucketName, directoryPath, filter?, transform?, subDirectoryOrFilePath?S3 bucket, public access block, bucket policy, website configuration, one object per file
AwsDynamoDBServiceModuleregion, TableName, KeySchema, AttributeDefinitions, billingMode, GlobalSecondaryIndexes?, LocalSecondaryIndexes?, StreamSpecification?, timeToLiveAttribute?, DeletionProtectionEnabled?, PointInTimeRecoveryEnabled?DynamoDB table
AwsDynamoDBGlobalServiceModuledynamoDBService, replicasDynamoDB global table replicas
AwsEcsAlbServiceModuleregion, albName, listeners, subnets, targets?application load balancer, listener, target group, security group
AwsCloudFrontServiceModuleaccount, distributionName, origins, routes, aliases?, certificateArn?, defaultRootObject?, errorResponses?, functions?, priceClass?, cachePolicies?, originRequestPolicies?, responseHeadersPolicies?CloudFront distribution, an origin access control per bucket or function it serves, a statement in each of those buckets' policies, a permission on each of those functions, a CloudFront Function per functions entry, and the custom policies declared
AwsCloudFrontInvalidationServiceModulecloudFront, sourcean invalidation of the distribution every time the bucket's files change

filter and transform are functions, which octo.yaml takes as an export of your own: a transform returning `app/${filePath}` uploads the directory under app/.

AwsS3StaticWebsiteServiceModule makes a public bucket that answers on its S3 website endpoint, over plain HTTP. A site served over HTTPS from a closed bucket is two modules instead: AwsS3StorageServiceModule holds the files, and AwsCloudFrontServiceModule serves them. Which module you pick is what decides whether the bucket is public, so removing a distribution can never reopen a bucket behind it.

AwsCloudFrontServiceModule sits under the account, because a distribution is not in any one region. origins names what it serves, each one an originId and a source: a bucket from AwsS3StorageServiceModule, or an execution from AwsLambdaExecutionModule, served through its URL. routes sends requests to an origin by its originId. A new version of a function is a second execution with a URL of its own, so moving traffic to it means changing the source of the origin to the new execution. The route with no pathPattern is the default, and the others are matched in the order they are given. For every bucket it serves, the module adds a statement to that bucket's policy naming the distribution, so the bucket stays closed to everyone else.

CloudFront signs every request it forwards to a function, and the function admits the distributions of its own account. A route to a function defaults to every HTTP method, no caching, and every header the viewer sent except Host. A client sending a body, a POST or a PUT, must send the body's SHA-256 in hex as the x-amz-content-sha256 header, because the signature covers the body; without it the function answers 403. CloudFront waits 60 seconds for a function to answer, so a function whose timeoutInSeconds is longer is rejected before anything is created.

  - moduleId: website-storage-module
moduleClass: AwsS3StorageServiceModule
moduleInputs:
bucketName: my-website-bucket
directoryPath: website
region: "${{region-module.model.region}}"

- moduleId: website-cloudfront-module
moduleClass: AwsCloudFrontServiceModule
moduleInputs:
account: "${{account-module.model.account}}"
defaultRootObject: index.html
distributionName: my-website
errorResponses:
- errorCode: 403
responseCode: 404
responsePagePath: /error.html
origins:
- originId: website
source: "${{website-storage-module.model.service}}"
routes:
- originId: website

- moduleId: website-invalidation-module
moduleClass: AwsCloudFrontInvalidationServiceModule
moduleInputs:
cloudFront: "${{website-cloudfront-module.model.service}}"
source: "${{website-storage-module.model.service}}"

A bucket served this way cannot have a dot in its name. The distribution reaches it at <bucket>.s3.<region>.amazonaws.com over HTTPS, and the certificate AWS serves there covers a single label, so a dotted name is refused when the modules are composed rather than answering 502 to every visitor later.

An origin can also be an ALB from AwsEcsAlbServiceModule. CloudFront reaches it over HTTP on port 80, the only listener the ALB has, and the ALB still answers anyone who calls its DNS name directly: putting CloudFront in front of it does not close it.

Set aliases to answer on your own domains and certificateArn to a certificate covering them; CloudFront reads certificates from us-east-1 alone, wherever the bucket lives. The DNS records pointing at the distribution are yours to create. priceClass decides which edge locations serve the site.

errorResponses decides what a missing path returns, and it applies to the whole distribution. A closed bucket reports a missing file as 403, so a content site maps both 403 and 404 to a real 404 carrying its error page, as above.

functions creates CloudFront Functions, JavaScript that runs at the edge on a request or a response, and a route runs one by its name. Each is a file, at a path from the project root. A single-page app served under a path rewrites every request that names no file to its index.html, on its own route only, so an API's real 404s stay 404s:

functions/my-app-deep-links.js
function handler(event) {
var request = event.request;
if (!request.uri.split('/').pop().includes('.')) {
request.uri = '/app/index.html';
}
return request;
}
      functions:
- name: my-app-deep-links
filePath: functions/my-app-deep-links.js
routes:
- originId: app
pathPattern: /app/*
functionAssociations:
- { eventType: viewer-request, functionName: my-app-deep-links, kind: cloudfront-function }

A function's name is unique in its account, and its file is at most 10 KB. It runs only on viewer-request or viewer-response. A route runs only the functions its distribution creates.

AwsCloudFrontInvalidationServiceModule takes a distribution and one bucket it serves, and drops what the edge locations hold for that bucket every time its files change. Without it CloudFront keeps serving the previous version of a file for as long as its cache policy says, which is a day: the upload succeeds and the site does not change. It runs after the files are uploaded, because it waits for both the distribution and the bucket.

Two things about the DynamoDB module are worth knowing before you use it, and its generated definition file says both: a global secondary index can be added to or removed from a live table, and a local secondary index cannot, so changing one recreates the table. DeletionProtectionEnabled changes in a run of its own: AWS takes it in a request of its own, so Octo refuses a run that changes anything else about the table with it, except PointInTimeRecoveryEnabled, which AWS changes through a request of its own as well.

AwsDynamoDBGlobalServiceModule takes the table's own service and a replica per further region, each one a region node and optional tags. A replica cannot name the region the table sits in. Every replica takes the table's DeletionProtectionEnabled and PointInTimeRecoveryEnabled. Regions that share an AWS region share one copy of the table, so they must give it the same tags, and a region in the table's own AWS region is served by the table itself:

  - moduleId: accounts-global-module
moduleClass: AwsDynamoDBGlobalServiceModule
moduleInputs:
dynamoDBService: "${{dynamodb-service-module.model.service}}"
replicas:
- region: "${{region-west-module.model.region}}"
tags:
Tier: data

AwsEcsAlbServiceModule has the deepest inputs in this catalog. listeners is a port and the actions taken on it; targets is one entry per container the listener forwards to, each naming an execution and how it is health checked. A target's Name is what a listener's targetGroupName refers to:

  - moduleId: qa-api-alb-module
moduleClass: AwsEcsAlbServiceModule
moduleInputs:
albName: qa-api-alb
region: "${{region-module.model.region}}"
subnets:
- "${{public-subnet-module.model.subnet}}"
- "${{public-backup-subnet-module.model.subnet}}"
listeners:
- Port: 80
rules: [ ]
DefaultActions:
- actionType: forward
action:
TargetGroups:
- targetGroupName: backend-v1-80
Weight: 100
targets:
- Name: backend-v1-80
containerName: backend-v1
containerPort: 80
execution: "${{backend-v1-qa-execution-module.model.execution}}"
healthCheck:
HealthCheckIntervalSeconds: 30
HealthCheckPath: /
HealthCheckPort: 80
HealthCheckProtocol: HTTP
HealthCheckTimeoutSeconds: 5
HealthyThresholdCount: 2
Matcher:
HttpCode: 200
UnhealthyThresholdCount: 2

A load balancer bills by the hour from the moment it exists, whether or not anything reaches it.

Filesystems and images​

ModuleInputsCreates
AwsEfsFilesystemModuleregion, filesystemNameEFS filesystem
AwsEcrImageModuleimageFamily, imageName, regionsECR repository, one per AWS region
AwsEcrImageTagModuleimage, imageTag, architecture, dockerfileDirectory, dockerfileName?, buildArguments?, buildSecrets?one published image per repository

An EFS filesystem belongs to a region; mounting it into a subnet is the subnet module's localFilesystems input, which is what produces the mount target.

The two image modules are a repository and the versions inside it, declared separately. AwsEcrImageModule creates the repositories and nothing else, one in each AWS region its regions resolve to. Every region must be in one account, because an image is used only inside its own account; a second account declares an image module of its own. Adding a region to regions or removing one leaves every other repository, and every version inside it, as it is. AwsEcrImageTagModule takes that module's image as its image input, builds the Dockerfile at dockerfileDirectory for architecture, arm64 or x86_64, and pushes the result into every repository the image has, under imageTag. A Lambda function or an ECS task running the image runs on that processor, and an ECS task whose images were built for different processors is rejected. dockerfileDirectory is relative to the project root, because it is part of what Octo commits and an absolute path would differ on every machine. One repository is pointed at by as many tag modules as you have versions, and adding or removing one leaves the others alone.

imageTag is the only input that decides what is published. A repository's tags are immutable, so editing architecture, dockerfileDirectory, dockerfileName, buildArguments, or buildSecrets publishes nothing: new content is a new tag, which is a new module. Changing the tag publishes the new one and deletes the old.

buildArguments are passed to the build as --build-arg, for the Dockerfile's ARG instructions. They are kept in Octo's state, so a secret goes in buildSecrets instead, which maps each secret's id, as the Dockerfile's RUN --mount=type=secret,id=... names it, to the environment variable that holds it. Octo keeps only the variable's name and passes the secret as --secret, and an apply where the variable is not set fails before anything is published:

  - moduleId: api-image-v1-module
moduleClass: AwsEcrImageTagModule
moduleInputs:
image: "${{api-image-module.model.image}}"
imageTag: 0.0.1
architecture: arm64
dockerfileDirectory: api
buildArguments:
NODE_ENV: production
buildSecrets:
NPM_TOKEN: NPM_TOKEN

This is one of the library's two external resources, the other being the CloudFront invalidation. Terraform has no resource that puts an image into a registry, so Octo runs the local docker CLI and the ECR API itself, which means the machine running the apply needs Docker, and builds the image for its own processor. Octo never runs docker login: each Docker command gets the registry's credentials on its own, so tags pushed in parallel never collide, and your Docker credential store is never written. An execution whose deployment names the image and tag is ordered behind the push, so the same run can publish a version and start something that runs it.

Anchors​

These are the shapes the modules above publish and consume, and they are what a module of your own needs to match to interoperate. Import a schema from @quadnix/octo-aws-cdk/anchors/<name>/schema, where <name> is the anchor's own name rather than the directory it sits in: anchors/aws-serverless-environment/schema, not anchors/aws-environment/schema.

AnchorOnPublishes
AwsAccountAnchorSchemaaccountawsAccountId
AwsRegionAnchorSchemaregionawsRegionId, awsRegionAZs, regionId, vpcCidrBlock
AwsServerlessRegionAnchorSchemaregionawsRegionId, regionId, serverless
AwsSubnetAnchorSchemasubnetsubnetName, CidrBlock, AvailabilityZone, awsAccountId, awsRegionId
AwsServerlessSubnetAnchorSchemasubnetsubnetName, serverless
AwsSubnetLocalFilesystemMountAnchorSchemaoverlaysubnetName, filesystemName, awsAccountId, awsRegionId
AwsServerAnchorSchemaserverserverKey, grants
AwsSecurityGroupAnchorSchemaserver, executionsecurityGroupName, rules
AwsDeploymentAnchorSchemadeploymentimage, command, ports, cpu, memory, timeoutInSeconds
AwsS3StorageServiceAnchorSchemaservicebucketName, awsAccountId, awsRegionId, contentDigest
AwsS3StorageServiceAccessPrefixAnchorSchemaserviceaccessPrefix, bucketName
AwsEcsClusterAnchorSchemaenvironmentclusterName, environmentVariables
AwsServerlessEnvironmentAnchorSchemaenvironmentenvironmentName, environmentVariables, serverless
AwsEcsExecutionAnchorSchemaexecutiondesiredCount, environmentVariables
AwsEcsServiceAnchorSchemaoverlayexecutionId
AwsLambdaExecutionAnchorSchemaexecutionfunctionName, environmentVariables
AwsLambdaFunctionAnchorSchemaoverlayfunctionName, timeoutInSeconds
AwsEcrImageAnchorSchemaimageimageId
AwsEcsAlbAnchorSchemaservicealbName
AwsCloudFrontAnchorSchemaserviceawsAccountId, distributionName
AwsDynamoDBAnchorSchemaserviceTableName, awsAccountId, awsRegionId
AwsDynamoDBGlobalAnchorSchemaserviceTableName
AwsEfsAnchorSchemafilesystemfilesystemName

Three of them sit on an overlay rather than on a model, because what they describe is a relationship: a filesystem mounted into a subnet, an execution's ECS service, and an execution's Lambda function.

The one you will meet first is AwsRegionAnchorSchema. Almost every module in this catalog takes a region, and what it actually requires is any node named region that publishes that shape, which is why replacing the region module changes nothing downstream.