Doug.Instance

Pipelines for AWS Lambda – Part 3: The Pipeline

Nov 5, 2021

TL/DR;

You can create a GitHub Action pipeline with sam pipeline init, but it will be configured for python and feature branches that start with "feature".

GitHub Action Pipeline

The next step of the tutorial is to run sam pipeline init. Unlike sam pipeline bootstrap, this command does not deploy resources directly to AWS. In the example below, I entered placeholders for the ARNs for the resources created in Part 1, but you can enter these ARNs when configuring your pipeline.

$ sam pipeline init

sam pipeline init generates a pipeline configuration file that your CI/CD system
can use to deploy serverless applications using AWS SAM.
We will guide you through the process to bootstrap resources for each stage,
then walk through the details necessary for creating the pipeline config file.

Please ensure you are in the root folder of your SAM application before you begin.

Select a pipeline structure template to get started:
Select template
    1 - AWS Quick Start Pipeline Templates
    2 - Custom Pipeline Template Location
Choice: 1

Cloning from https://github.com/aws/aws-sam-cli-pipeline-init-templates.git
CI/CD system
    1 - Jenkins
    2 - GitLab CI/CD
    3 - GitHub Actions
    4 - AWS CodePipeline
Choice: 3
You are using the 2-stage pipeline template.
 _________    _________ 
|         |  |         |
| Stage 1 |->| Stage 2 |
|_________|  |_________|

Checking for existing stages...

[!] None detected in this account.

To set up stage(s), please quit the process using Ctrl+C and use one of the following commands:
sam pipeline init --bootstrap       To be guided through the stage and config file creation process.
sam pipeline bootstrap              To specify details for an individual stage.

To reference stage resources bootstrapped in a different account, press enter to proceed []: 

This template configures a pipeline that deploys a serverless application to a testing and a production stage.

What is the GitHub secret name for pipeline user account access key ID? [AWS_ACCESS_KEY_ID]: 
What is the GitHub Secret name for pipeline user account access key secret? [AWS_SECRET_ACCESS_KEY]: 
What is the git branch used for production deployments? [main]: 
What is the template file path? [template.yaml]: 
We use the stage name to automatically retrieve the bootstrapped resources created when you ran `sam pipeline bootstrap`.


What is the name of stage 1 (as provided during the bootstrapping)?
Select an index or enter the stage name: Build
What is the sam application stack name for stage 1? [sam-app]: build-stack
What is the pipeline execution role ARN for stage 1?: pipeline-execution-arn
What is the CloudFormation execution role ARN for stage 1?: clouformation-execution-arn
What is the S3 bucket name for artifacts for stage 1?: build-bucket
What is the ECR repository URI for stage 1? []: 
What is the AWS region for stage 1?: us-east-1
Stage 1 configured successfully, configuring stage 2.


What is the name of stage 2 (as provided during the bootstrapping)?
Select an index or enter the stage name: deploy
What is the sam application stack name for stage 2? [sam-app]: deploy-stack
What is the pipeline execution role ARN for stage 2?: pipeline-execution-arn
What is the CloudFormation execution role ARN for stage 2?: clouformation-execution-arn
What is the S3 bucket name for artifacts for stage 2?: deploy-bucket
What is the ECR repository URI for stage 2? []: 
What is the AWS region for stage 2?: us-east-1
Stage 2 configured successfully.

SUMMARY
We will generate a pipeline config file based on the following information:
    What is the GitHub secret name for pipeline user account access key ID?: AWS_ACCESS_KEY_ID
    What is the GitHub Secret name for pipeline user account access key secret?: AWS_SECRET_ACCESS_KEY
    What is the git branch used for production deployments?: main
    What is the template file path?: template.yaml
    What is the name of stage 1 (as provided during the bootstrapping)?
Select an index or enter the stage name: Build
    What is the sam application stack name for stage 1?: build-stack
    What is the pipeline execution role ARN for stage 1?: pipeline-execution-arn
    What is the CloudFormation execution role ARN for stage 1?: clouformation-execution-arn
    What is the S3 bucket name for artifacts for stage 1?: build-bucket
    What is the ECR repository URI for stage 1?: 
    What is the AWS region for stage 1?: us-east-1
    What is the name of stage 2 (as provided during the bootstrapping)?
Select an index or enter the stage name: deploy
    What is the sam application stack name for stage 2?: deploy-stack
    What is the pipeline execution role ARN for stage 2?: pipeline-execution-arn
    What is the CloudFormation execution role ARN for stage 2?: clouformation-execution-arn
    What is the S3 bucket name for artifacts for stage 2?: deploy-bucket
    What is the ECR repository URI for stage 2?: 
    What is the AWS region for stage 2?: us-east-1
Successfully created the pipeline configuration file(s):
    - .github/workflows/pipeline.yaml

This will create the GitHub pipeline configuration which I have captured in this gist.

Configuring Runtime Platform

Using the default template creates a pipeline to build a python app. So the first change I made was to replace these lines to configure the python actions:

- uses: actions/setup-python@v2

with the node configuration as follows (note the version specification):

- uses: actions/setup-node@v2
  with:
    node-version: '14'

Configuring Branch Naming

Another minor issue with the generated pipeline is that it assumes a certain convention for naming branches where all feature branches start with "feature". Typically for my open source projects, I just use the GitHub issue number and title as my branch name (so something like 123-my-issue-title). Therefore I modified the branch filters at the top of the pipeline configuration as follows:

on:
  push:
    branches:
      - 'main'
      - '[0-9]+**'

Then modified the build-and-deploy-feature stage as follows so it runs on any branch other than main:

build-and-deploy-feature:
  # this stage is triggered only for feature branches (feature*),
  # which will build the stack and deploy to a stack named with branch name.
  if: github.ref != 'refs/heads/main'

A similar change was required for delete-feature since this runs only in feature branches. Notice that the condition looks at github.event.ref and not github.ref as shown in the previous change.

delete-feature:
  if: github.event.ref != 'refs/heads/main' && github.event_name == 'delete'

Finally, this naming convention breaks sam delpoy since this uses a CloudFormation stack name that matches the branch name. Since the stack cannot start with a number, I added a "feature-" prefix to the stack name in the build-and-deploy-feature stage as shown:

- name: Deploy to feature stack in the testing account
  shell: bash
  run: |
    sam deploy --stack-name feature-$(echo ${GITHUB_REF##*/} | tr -cd '[a-zA-Z0-9-]') \
      --capabilities CAPABILITY_IAM \
      --region ${TESTING_REGION} \
      --s3-bucket ${TESTING_ARTIFACTS_BUCKET} \
      --no-fail-on-empty-changeset \
      --role-arn ${TESTING_CLOUDFORMATION_EXECUTION_ROLE}

Summary

The pipeline configuration created by sam pipeline init is fairly comprehensive. It handles creating a unique deployment stack for feature branches, deleting those stacks when the branch is deleted, and a multi-phase deployment for production which includes integration tests. Unfortunately this pipeline defaults to python so we have to update to node.js or whatever platform you prefer. Also, it assumes all feature branches are prefixed with feature so we need to modify the template if we are not following this convention.