Pipelines for AWS Lambda – Part 3: The Pipeline

TL/DR;

You can create a GitHub Action pipeline with sam pipeline init, but it will be configured for python and feature branches that start with “feature”.

GitHub Action Pipeline

The next step of the tutorial is to run sam pipeline init. Unlike sam pipeline bootstrap, this command does not deploy resources directly to AWS. In the example below, I entered placeholders for the ARNs for the resources created in Part 1, but you can enter these ARNs when configuring your pipeline.

$ sam pipeline init

sam pipeline init generates a pipeline configuration file that your CI/CD system
can use to deploy serverless applications using AWS SAM.
We will guide you through the process to bootstrap resources for each stage,
then walk through the details necessary for creating the pipeline config file.

Please ensure you are in the root folder of your SAM application before you begin.

Select a pipeline structure template to get started:
Select template
	1 - AWS Quick Start Pipeline Templates
	2 - Custom Pipeline Template Location
Choice: 1

Cloning from https://github.com/aws/aws-sam-cli-pipeline-init-templates.git
CI/CD system
	1 - Jenkins
	2 - GitLab CI/CD
	3 - GitHub Actions
	4 - AWS CodePipeline
Choice: 3
You are using the 2-stage pipeline template.
 _________    _________ 
|         |  |         |
| Stage 1 |->| Stage 2 |
|_________|  |_________|

Checking for existing stages...

[!] None detected in this account.

To set up stage(s), please quit the process using Ctrl+C and use one of the following commands:
sam pipeline init --bootstrap       To be guided through the stage and config file creation process.
sam pipeline bootstrap              To specify details for an individual stage.

To reference stage resources bootstrapped in a different account, press enter to proceed []: 

This template configures a pipeline that deploys a serverless application to a testing and a production stage.

What is the GitHub secret name for pipeline user account access key ID? [AWS_ACCESS_KEY_ID]: 
What is the GitHub Secret name for pipeline user account access key secret? [AWS_SECRET_ACCESS_KEY]: 
What is the git branch used for production deployments? [main]: 
What is the template file path? [template.yaml]: 
We use the stage name to automatically retrieve the bootstrapped resources created when you ran `sam pipeline bootstrap`.


What is the name of stage 1 (as provided during the bootstrapping)?
Select an index or enter the stage name: Build
What is the sam application stack name for stage 1? [sam-app]: build-stack
What is the pipeline execution role ARN for stage 1?: pipeline-execution-arn
What is the CloudFormation execution role ARN for stage 1?: clouformation-execution-arn
What is the S3 bucket name for artifacts for stage 1?: build-bucket
What is the ECR repository URI for stage 1? []: 
What is the AWS region for stage 1?: us-east-1
Stage 1 configured successfully, configuring stage 2.


What is the name of stage 2 (as provided during the bootstrapping)?
Select an index or enter the stage name: deploy
What is the sam application stack name for stage 2? [sam-app]: deploy-stack
What is the pipeline execution role ARN for stage 2?: pipeline-execution-arn
What is the CloudFormation execution role ARN for stage 2?: clouformation-execution-arn
What is the S3 bucket name for artifacts for stage 2?: deploy-bucket
What is the ECR repository URI for stage 2? []: 
What is the AWS region for stage 2?: us-east-1
Stage 2 configured successfully.

SUMMARY
We will generate a pipeline config file based on the following information:
	What is the GitHub secret name for pipeline user account access key ID?: AWS_ACCESS_KEY_ID
	What is the GitHub Secret name for pipeline user account access key secret?: AWS_SECRET_ACCESS_KEY
	What is the git branch used for production deployments?: main
	What is the template file path?: template.yaml
	What is the name of stage 1 (as provided during the bootstrapping)?
Select an index or enter the stage name: Build
	What is the sam application stack name for stage 1?: build-stack
	What is the pipeline execution role ARN for stage 1?: pipeline-execution-arn
	What is the CloudFormation execution role ARN for stage 1?: clouformation-execution-arn
	What is the S3 bucket name for artifacts for stage 1?: build-bucket
	What is the ECR repository URI for stage 1?: 
	What is the AWS region for stage 1?: us-east-1
	What is the name of stage 2 (as provided during the bootstrapping)?
Select an index or enter the stage name: deploy
	What is the sam application stack name for stage 2?: deploy-stack
	What is the pipeline execution role ARN for stage 2?: pipeline-execution-arn
	What is the CloudFormation execution role ARN for stage 2?: clouformation-execution-arn
	What is the S3 bucket name for artifacts for stage 2?: deploy-bucket
	What is the ECR repository URI for stage 2?: 
	What is the AWS region for stage 2?: us-east-1
Successfully created the pipeline configuration file(s):
	- .github/workflows/pipeline.yaml

This will create the GitHub pipeline configuration which I have captured in this gist.

Configuring Runtime Platform

Using the default template creates a pipeline to build a python app. So the first change I made was to replace these lines to configure the python actions:

      - uses: actions/setup-python@v2

with the node configuration as follows (note the version specification):

      - uses: actions/setup-node@v2
        with:
          node-version: '14'

Configuring Branch Naming

Another minor issue with the generated pipeline is that it assumes a certain convention for naming branches where all feature branches start with “feature”. Typically for my open source projects, I just use the GitHub issue number and title as my branch name (so something like 123-my-issue-title). Therefore I modified the branch filters at the top of the pipeline configuration as follows:

on:
  push:
    branches:
      - 'main'
      - '[0-9]+**'

Then modified the build-and-deploy-feature stage as follows so it runs on any branch other than main:

  build-and-deploy-feature:
    # this stage is triggered only for feature branches (feature*),
    # which will build the stack and deploy to a stack named with branch name.
    if: github.ref != 'refs/heads/main'

A similar change was required for delete-feature since this runs only in feature branches. Notice that the condition looks at github.event.ref and not github.ref as shown in the previous change.

  delete-feature:
    if: github.event.ref != 'refs/heads/main' && github.event_name == 'delete'

Finally, this naming convention breaks sam delpoy since this uses a CloudFormation stack name that matches the branch name. Since the stack cannot start with a number, I added a “feature-” prefix to the stack name in the build-and-deploy-feature stage as shown:

      - name: Deploy to feature stack in the testing account
        shell: bash
        run: |
          sam deploy --stack-name feature-$(echo ${GITHUB_REF##*/} | tr -cd '[a-zA-Z0-9-]') \
            --capabilities CAPABILITY_IAM \
            --region ${TESTING_REGION} \
            --s3-bucket ${TESTING_ARTIFACTS_BUCKET} \
            --no-fail-on-empty-changeset \
            --role-arn ${TESTING_CLOUDFORMATION_EXECUTION_ROLE}

Summary

The pipeline configuration created by sam pipeline init is fairly comprehensive. It handles creating a unique deployment stack for feature branches, deleting those stacks when the branch is deleted, and a multi-phase deployment for production which includes integration tests. Unfortunately this pipeline defaults to python so we have to update to node.js or whatever platform you prefer. Also, it assumes all feature branches are prefixed with feature so we need to modify the template if we are not following this convention.

Pipelines for AWS Lambda – Part 2: The Code

TL/DR;

One of the great things about AWS Lambda is that you can write your code and deploy without worrying about the hosting environment (kind of). So let’s talk about what that code should look like so you really don’t have to worry.

Background

As I mentioned in my previous post, the AWS Serverless Application Model (SAM), has made (some) things better about developing serverless functions in AWS Lambda. We are going to create a fairly basic Hello World API. The code itself is relatively simple but Lambda only works when deployed with all of the correct resources and permissions linked correctly. Using SAM, we will deploy the Lambda function and an API gateway. The resources and permissions for this initial implementation are pretty simple, but there are still mistakes that can be made so I’ll walk through the troubleshooting steps.

Creating the Lambda Code

Before we talk about deployment, we need to have some code to deploy. To make sure we capture all of the things we need for our function to work, we are just going to scaffold a new project using sam init. There is a large collection of starter templates maintained by AWS and SAM uses this repository to scaffold new projects. Below shows the selections I used to generate a “hello world” project in Node.js:

$ sam init
Which template source would you like to use?
	1 - AWS Quick Start Templates
	2 - Custom Template Location
Choice: 1
What package type would you like to use?
	1 - Zip (artifact is a zip uploaded to S3)	
	2 - Image (artifact is an image uploaded to an ECR image repository)
Package type: 1

Which runtime would you like to use?
	1 - nodejs14.x
	2 - python3.9
	3 - ruby2.7
	4 - go1.x
	5 - java11
	6 - dotnetcore3.1
	7 - nodejs12.x
	8 - nodejs10.x
	9 - python3.8
	10 - python3.7
	11 - python3.6
	12 - python2.7
	13 - ruby2.5
	14 - java8.al2
	15 - java8
	16 - dotnetcore2.1
Runtime: 1

Project name [sam-app]: sam-test-node

Cloning from https://github.com/aws/aws-sam-cli-app-templates

AWS quick start application templates:
	1 - Hello World Example
	2 - Step Functions Sample App (Stock Trader)
	3 - Quick Start: From Scratch
	4 - Quick Start: Scheduled Events
	5 - Quick Start: S3
	6 - Quick Start: SNS
	7 - Quick Start: SQS
	8 - Quick Start: Web Backend
Template selection: 1

    -----------------------
    Generating application:
    -----------------------
    Name: sam-test-node
    Runtime: nodejs14.x
    Dependency Manager: npm
    Application Template: hello-world
    Output Directory: .
    
    Next steps can be found in the README file at ./sam-test-node/README.md

You can view the template code in GitHub to se what is created. Let’s walk through each file.

Function Code

In the hello-world folder, you will find app.js. This file contains all of the code required for the function. There is some commented out code that requires axios for making a simple HTTP call, but the active code does not have any dependencies so if you simply upload this code into a new Lambda function and test it via the AWS console, you will get a simple output message looking like this:

{ "message": "hello world" }

The full code for the function is below:

// const axios = require('axios')
// const url = 'http://checkip.amazonaws.com/';
let response;

/**
 *
 * Event doc: https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html#api-gateway-simple-proxy-for-lambda-input-format
 * @param {Object} event - API Gateway Lambda Proxy Input Format
 *
 * Context doc: https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html 
 * @param {Object} context
 *
 * Return doc: https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html
 * @returns {Object} object - API Gateway Lambda Proxy Output Format
 * 
 */
exports.lambdaHandler = async (event, context) => {
    try {
        // const ret = await axios(url);
        response = {
            'statusCode': 200,
            'body': JSON.stringify({
                message: 'hello world',
                // location: ret.data.trim()
            })
        }
    } catch (err) {
        console.log(err);
        return err;
    }

    return response
};

There isn’t much code here but what is here is very important. First off, the lambdaHandler function is exposed as a static function meaning you do not need to create an instance of a class to invoke the function. This is important because this is how Lambda expects to invoke the handler so when you specify the handler in the Lambda configuration, it must point to a static function.

Also notice that the handler function is marked async. If you do not specify an async function, a third parameter named callback will be passed to your handler and you will need to invoke this callback as shown in the AWS documentation.

Note: The event parameter varies based on the type of invocation and documentation is not as thorough as you would think. If you write the parameter out with console.log(event), you can see the contents in the CloudWatch log for the Lambda.

Note that the error handler in this code returns the any error caught by the Lambda handler. This allows Lambda to log the invocation as an error. If your Lambda returns a valid response with an error statusCode value (ex: 500), it will still be logged as a successful invocation since the Lambda itself did not fail.

SAM Template

The next file generated by sam init is the template.yaml file which is also placed in the root folder. This template is similar to CloudFormation and in fact can contain most CloudFormation syntax. However, SAM provides simplified syntax and linkage for creating serverless applications. Let’s take a look at the file generated when I ran sam init.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  sam-test-node

  Sample SAM Template for sam-test-node
  
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 3

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello-world/
      Handler: app.lambdaHandler
      Runtime: nodejs14.x
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get

Outputs:
  # ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
  # Find out more about other implicit resources you can reference within SAM
  # https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

Note: We don’t specify a name for our Lambda function or API Gateway. When we deploy using SAM, we provide a stack name that is used for the CloudFormation stack but also carried to other resources for consistent naming. This allows us to identify resources created for testing purposes based off of the branch they were created from.

The first meaningful section of the template is the Globals configuration. This allows you to specify – you guessed it – global information that applies to all resources. In this example, the timeout is set to 3 seconds for all Lambda functions. This just happens to be the default, but you can set any default values here you want to apply for all functions. Since we only have one function in this template, we could have just as easily placed this Timeout key in the Properties section of the Lambda function configuration, but it is placed in Globals as an example.

The second important section is the Resources section. Even though there is only one resource specified, SAM will actually create 2 resources: the Lambda function and the API Gateway. The deployment process will also create a third resource: a Lambda Application which will provide information on all of the resources, deployments, and Lambda invocations all in one place.

The first key under Resources is HelloWorldFunction. This is a logical ID that can be used to reference the function in other parts of the template. The Type key specifies that this is a Lambda function, and the Properties key contains all of the configuration for the function (see the AWS documentation for more options for configuring a function). The CodeUri key is optional and defines the base path for your code and, as I mentioned before, the Handler key in the points to the static function in your Lambda code. If you define multiple Lambda Functions in one template and all of your code is in a folder such as src or bin, you can define the CodeUri in the Globals section and have it apply to all of your functions. Otherwise, you can simply include the path in the Handler key like hello-world/app.lambdaHandler and remove the CodeUri key. The Runtime key allows you to specify a specific version of your runtime. I’m using Node.js version 14 in this series of posts, but you can find the list of supported runtimes in the AWS documentation.

The Events section under the Lambda function is where the most significant SAM magic happens. Once again we provide a logical key (HellowWorld) and then give it a Type value (Api) and then we can configure the resource with Properties. In this example we set the Path of the API to /hello and the Method to get. Under the hood, there is a lot more going on here. SAM does all of this based on just these 2 entries:

  • Creates an API Gateway
  • Creates a Route with path /hello and HTTP method GET
  • Creates an integration between the /hello route and the Lambda function
  • Creates a $default deployment stage for the API Gateway

Summary

In this post, we created code using sam init. The two most important files created by this code are the lambda function code and the SAM template (template.yaml). The code generated by SAM is obviously just a placeholder and will require significant editing. The SAM template is very important to how your function is deployed, but we are sticking with the basic Lambda “Hello World” example with a REST API.

CodeTender: A Brief History

I was lucky to be involved in building a micro-services platform with Vasil Kovatchev. Vasil is a great software architect and overall great guy. One concept he introduced was the “bartender” script. The idea was to build a working micro-service with a placeholder name that won’t conflict with any code. The first such template was called “Pangalactic Gargleblaster”. The bartender script replaces placeholders in the code (“Pangalactic” and “Gargleblaster”) and does some other stuff to make the service work with the platform. The bartender script is a bash script and the “other stuff” is…well…more bash script. We were quickly serving up not just Pangalactic Gargleblasters, but also Flaming Volcanos and Tequila Sunrises and Whisky Sours. I fell in love with this concept, but I don’t, however, love the bash script since it is very tightly coupled to our ecosystem. Since node.js is a de facto standard for dev utilities (no offense, Python), I set out to build a CLI with these basic requirements:

  • Clone a git repo or copy a folder to a specified folder
  • Display prompts to collect values to replace placeholders with
  • Run scripts before or after replacing placeholders

Since “bartender” is a bit too common of a name, I went with “CodeTender”. That was March of 2018 (according to my commit history). Fast forward a few years and version 0.0.4-alpha was still full of bugs and my somewhat ambitious issue list was collecting dust along with the code. A somewhat unrelated thread with Vasil and another colleague reminded me of CodeTender so I set out to get it working. A few minutes (hours maybe) later, and the basics were working. So of course, time to use it.

I have been playing with Home Assistant lately and wanted to start playing with custom dashboard cards. I found a boilerplate project and decided why just clone the repo when I could get CodeTender working even better and make a Home Assistant custom card template? So after about another week, I have burned down a lot of my issue list and added some cool features. It is still very alpha, but it’s close to ready for prime time.

I will formally launch CodeTender in a future post, but for anyone interested, you can check it out on my GitHub.