Skip to main content

Composing Modules

A Terraform module's contract with its caller is a list of variables, and almost all of them are strings. Pass a subnet ID where a VPC ID was expected and nothing objects; both are text, and the mistake surfaces when the provider rejects it, partway through an apply. So the safe way to use somebody else's module is to read its source, which is not much better than not using it.

Octo modules are wired by handing over nodes. The shop's storage module is not given a region ID, it is given the region, and its schema states what it will accept. This page is what that looks like in practice: connecting modules, running one twice, replacing one, and what happens when a connection is wrong.

Handing over a node​

  - moduleId: region-module
    moduleClass: AwsSingleAzRegionModule
    moduleInputs:
      account: "${{account-module.model.account}}"
      name: eu-west-1
      regionId: aws-eu-west-1a
      vpcCidrBlock: 10.0.0.0/16

  - moduleId: receipts-module
    moduleClass: AwsS3StorageServiceModule
    moduleInputs:
      region: "${{region-module.model.region}}"
      bucketName: "${env.RECEIPTS_BUCKET_NAME}"

${{region-module.model.region}} resolves to the region node itself, and that is what the storage module needs. From the region it walks up to the account and the app to learn which account it is deploying into, and asks the region itself which AWS region that is. Handed the string eu-west-1 it could do none of that.

So the rule when writing a definition file is to pass the node. ${{region-module.model.region}} gives a module something it can ask questions of; ${{region-module.model.region.regionId}} gives it a string it can only trust.

The same module, twice​

A moduleId is per instance, so loading a class again gives an independent one. This is how the shop gets a public and a private subnet out of one subnet module:

  - moduleId: public-subnet-module
    moduleClass: AwsSimpleSubnetModule
    moduleInputs:
      region: "${{region-module.model.region}}"
      subnetAvailabilityZone: eu-west-1a
      subnetCidrBlock: 10.0.0.0/24
      subnetName: public-subnet
      subnetOptions:
        createNatGateway: true
        disableSubnetIntraNetwork: false
        subnetType: public

  - moduleId: private-subnet-module
    moduleClass: AwsSimpleSubnetModule
    moduleInputs:
      region: "${{region-module.model.region}}"
      subnetAvailabilityZone: eu-west-1a
      subnetCidrBlock: 10.0.2.0/24
      subnetName: private-subnet
      subnetOptions:
        createNatGateway: false
        disableSubnetIntraNetwork: true
        subnetType: private
      subnetSiblings:
        - attachToNatGateway: true
          subnet: "${{public-subnet-module.model.subnet}}"

Two folders, public-subnet-module/ and private-subnet-module/, and the second is handed the first. attachToNatGateway: true is the private subnet asking to route through the public one's NAT gateway, and the module works out what that costs in route tables.

Order​

Modules run in the order they are listed, and a module can only see what an earlier one registered. Move the receipts module above the region module and the run stops before anything is written:

Error: Input "receipts-module.input.region" refers to "${{region-module.model.region}}", which does not exist!
    at InputService.refuseMissingReference (.../octo/dist/services/input/input.service.js:340:15)
    at InputService.resolve (.../octo/dist/services/input/input.service.js:242:18)
    at .../octo/dist/modules/module.container.js:54:62
inputKey: receipts-module.input.region

The first line is the schema check passing, so the file itself is fine. The failure is one step later, while the modules are being run. It names the input that failed and what it pointed at, which is either listed too late or misspelled.

Modules of one class run together, at the place where the first of them is listed. With the public subnet listed first, a filesystem, and then a private subnet that mounts the filesystem, both subnets run before the filesystem exists, and the private subnet fails as above. List the filesystem above the first subnet, not only above the subnet that needs it.

This ordering has nothing to do with the order things are created in AWS. That comes from the dependency edges between nodes, which is why the shop's checkout folder applies before the receipts folder even though the receipts module is listed first.

Replacing a module​

The shop uses AwsSingleAzRegionModule. Our AWS library also has AwsMultiAzRegionModule, which takes a list of availability zones instead of one:

imports:
  - className: AwsMultiAzRegionModule
    importPath: "@quadnix/octo-aws-cdk/modules/region/aws-multi-az-region"

modules:
  - moduleId: region-module
    moduleClass: AwsMultiAzRegionModule
    moduleInputs:
      account: "${{account-module.model.account}}"
      name: eu-west-1
      regionIds:
        - aws-eu-west-1a
        - aws-eu-west-1b
      vpcCidrBlock: 10.0.0.0/16

Nothing downstream changes. The receipts module is still handed ${{region-module.model.region}}, and the checkout server module still resolves the same way, because neither was ever given an ID. They were handed a region, and both region modules publish the same AwsRegionAnchor.

What actually differs after the swap is worth being precise about. The generated region-module/ folder is the same VPC and internet gateway, because both modules define a region as one VPC. The change is in what the region advertises: its anchor now lists two availability zones instead of one, and that is what a subnet module reads when it decides where it can place itself.

The honest limit is that on live infrastructure this is still a migration. Octo will tell you exactly what gets created, replaced, and destroyed, and it does not sequence that transition for you.

Mixing libraries​

Nothing in the composition above depends on the modules coming from the same package. A module declares what it needs as an anchor schema, and any module publishing that shape satisfies it. So a region module you wrote in-house works with our storage module, provided it publishes an AwsRegionAnchor.

That is also the constraint. Composition works when the two sides agree on an anchor schema, and there is no automatic bridge when they do not. If your region publishes AcmeRegionAnchor and a module wants AwsRegionAnchor, you either publish both anchors from your region, or register a schema translation between them.

When a connection is wrong​

Point the storage module at an account instead of a region, and:

Error: Property "region" in schema could not be validated!
    at getSchemaInstance (.../octo/dist/functions/schema/schema.js:22:19)
    at ModuleContainer.apply (.../octo/dist/modules/module.container.js:59:46)
    at async Octo.compose (.../octo/dist/main.js:120:16)
schemaClassName: AwsS3StorageServiceModuleSchema

The two failures mean different things. "Does not exist" is a reference to something that is not there, listed too late or misspelled. "Could not be validated" is a real node of the wrong shape.

Both happen while the graph is being assembled, before a line of Terraform is written and long before anything is created. That is the trade the whole design is making: more errors, all of them early.