AWS Elastic Container Registry

In this workshop, you will create a Docker Image based on the project application. This Docker Image will be uploaded/pushed to a Docker Registry. Docker Registry is a centrally managed catalog of images that can be deployed to target environments and infrastructures. AWS provides the AWS Elastic Container Registry (ECR) service to enable developers to create private and public image registries. In this section you will create a new ECR Public Repository, using Terraform to host your project Docker image.

Pickup from previous module: DevSecOps

Using what you learned about Infrastructure as Code and Terraform in previous sections, you will build a new job that creates a AWS Public ECR into our pipeline.

To complete this module, your config.yml must be identical to the one at the end of the last module. If yours is different, that’s ok! Just go ahead and copy this snippet and paste it into the file:

version: 2.1
orbs:
  node: circleci/node@4.2.0
  snyk: snyk/snyk@1.2.3
  aws-cli: circleci/aws-cli@2.0.2
  terraform: circleci/terraform@2.0.0
jobs:
  run_tests:
    docker:
      - image: cimg/node:14.16.0
    steps:
      - checkout
      - node/install-packages:
          override-ci-command: npm install
          cache-path: ~/project/node_modules
      - run:
          name: Run Unit Tests
          command: |
            ./node_modules/mocha/bin/mocha test/ --reporter mocha-junit-reporter --reporter-options mochaFile=./test/test-results.xml
            ./node_modules/mocha/bin/mocha test/ --reporter mochawesome --reporter-options reportDir=test-results,reportFilename=test-results
      - store_test_results:
          path: test/
      - store_artifacts:
          path: test-results          
  scan_app:
    docker:
      - image: cimg/node:14.16.0
    steps:
      - checkout
      - run:
          name: Snyk Scan Application files 
          command: npm install 
      - snyk/scan:
          fail-on-issues: false
          monitor-on-build: false

Build a create ECR job

The project application for this workshop will be distributed in a Docker image, but before you can create the image, you must have a container registry available to host it. In this section, you will learn how to create this registry using Terraform from within a CI/CD job.

Now that your config.yml is ready, you can build the create ECR job. The following code snippet demonstrates how to define and provision a job that creates a new AWS Public ECR within your CI/Cd pipeline.

Copy this snippet and append it to the bottom of your config.yml file:

  create_ecr_repo:
    docker:
      - image: cimg/node:14.16.0
    steps:
      - checkout
      - run:
          name: Create .terraformrc file locally
          command: echo "credentials \"app.terraform.io\" {token = \"$TERRAFORM_TOKEN\"}" > $HOME/.terraformrc
      - terraform/install:
          terraform_version: "0.14.10"
          arch: "amd64"
          os: "linux"
      - run:
          name: Create ECR Repo
          command: echo 'Create AWS ECR Repo with Terraform'
      - terraform/init:
          path: ./terraform/ecr
      - terraform/apply:
          path: ./terraform/ecr
      - run: 
          name: "Retrieve ECR URIs"
          command: |
            cd ./terraform/ecr
            mkdir -p /tmp/ecr/
            terraform init
            echo 'export ECR_NAME='$(terraform output ECR_NAME) >> /tmp/ecr/ecr_envars
            export ECR_PUBLIC_URI=$(terraform output ECR_URI)
            echo 'export ECR_PUBLIC_URI='$ECR_PUBLIC_URI >> /tmp/ecr/ecr_envars
            echo 'export ECR_URL='$(echo ${ECR_PUBLIC_URI:1:-1} | cut -d"/" -f1,2) >> /tmp/ecr/ecr_envars
      - persist_to_workspace:
          root: /tmp/ecr/
          paths:
            - "*"

You should already be familiar with the docker:, step:s and checkout job elements so we’ll focus on the remaining - run: elements in this job.

command: echo “credentials \“app.terraform.io\” {token = \“$TERRAFORM_TOKEN\“}” > $HOME/.terraformrc creates a required file, that is used by the Terraform CLI to authenticate your Terraform Cloud credentials and grant access to interact with the service. Notice that the $TERRAFORM_TOKEN environment variable, that you created earlier, specified and represents a protected way of referencing sensitive data in the config.yml file. Using environment variables in this manner protects against exposing sensitive data in pipeline configurations.

The - terraform/install: block uses the Terraform orb to install the appropriate Terraform CLI binary into the executor. The CLI will be required to execute Terraform code in the pipeline.

The - terraform/init: block uses the Terraform orb to perform a terraform init command, which initializes the project in the terraform/ecr/ directory.

The - terraform/apply: block uses the Terraform orb to perform a terraform apply command, which executes the code in the terraform/ecr/ directory.

The - run: name: “Retrieve ECR URIs” block has multiple commands listed in the command: block. I will address them individually:

  • cd ./terraform/ecr
    • Change directory into the ./terraform/ecr directory
  • mkdir -p /tmp/ecr/
    • Create a new directory that will hold valuable files that will be shared between jobs
  • terraform init
    • initialize the Terraform project
  • echo ‘export ECR_NAME=‘$(terraform output ECR_NAME) >> /tmp/ecr/ecr_envars
    • Use the $(terraform output ECR_NAME) command to get the ECR name assigned using Terraform and save it a file as an environment variable to be used between jobs
  • export ECR_PUBLIC_URI=$(terraform output ECR_URI)
    • Use the $(terraform output ECR_URI) command to create a local variable and assign the ECR URI value
  • echo ‘export ECR_PUBLIC_URI=‘$ECR_PUBLIC_URI >> /tmp/ecr/ecr_envars
    • Saves the assigned $ECR_PUBLIC_URI value to the /tmp/ecr/ecr_envars file
  • echo ‘export ECR_URL=‘$(echo ${ECR_PUBLIC_URI:1:-1} | cut -d”/” -f1,2) >> /tmp/ecr/ecr_envars
    • Saves a parsed version of the $ECR_PUBLIC_URI to the /tmp/ecr/ecr_envars file

These commands essentially save data to a file. That data will be used in subsequent pipeline jobs.

- persist_to_workspace: is a special key that represents a CircleCI Workspace. When a workspace is declared in a job, files and directories can be added to it. Each addition creates a new layer in the workspace filesystem. Downstream jobs can then use this workspace for their own needs or add more layers on top. Workspaces are not shared between pipeline runs. The only time a workspace can be accessed after the pipeline has run is when a workflow is rerun within the 15 day limit.

Update Terraform with your Terraform Cloud Organization

Your Terraform Cloud organization needs to be specified in the project’s Terraform code for both the ECR and APP Runner resources.

Update the ECR Terraform Code

Open the main.tf file in terraform/ecr/ directory and enter the Terraform Organization name you created earlier then save the file.

terraform {
  backend "remote" {
    organization = "<Enter the Terraform Cloud Organization Name here>" # Enter the Terraform Cloud Organization Name here

    workspaces {
      name = "ecr-aws-circleci"
    }
  }
}

Update the App Runner Terraform Code

Open the main.tf file in terraform/app-runner/ directory and enter the Terraform Organization name you created earlier then save the file.

terraform {
  backend "remote" {
    organization = "<Enter the Terraform Cloud Organization Name here>" # Enter the Terraform Cloud Organization Name here

    workspaces {
      name = "app-aws-circleci"
    }
  }
}

Note: Specifying your Terraform Cloud Organization Name in these files is critical and will produce and error if not completed.

Congratulations! You’ve just learned how to create a CI/CD pipeline job that leverages Terraform to create a new AWS Public ECR from within your pipeline.

That concludes this section of the workshop. Congratulations! You’ve just learned how to create a CI/CD pipeline job that leverages Terraform to create a new AWS Public ECR from within your pipeline.

In the next section, you’ll learn how to create a Docker image and upload it to your newly created ECR.