Building an Automated Serverless Workflow Manager with AWS Step Functions

Introduction

In the world of serverless computing, managing complex workflows that involve multiple AWS Lambda functions and other serverless services can be challenging. In this article, I will guide you through the process of building an “Automated Serverless Workflow Manager” using AWS Step Functions. This project will enable you to define, automate, and monitor intricate serverless workflows with ease.

Step 1: Project Setup and Configuration

Create a new project directory and initialize it with a package.json file using npm init. Install the AWS SDK for JavaScript and the AWS Step Functions module using the following commands:

npm install aws-sdk
npm install @aws-sdk/client-stepfunctions

Step 2: Create AWS Lambda Functions

For this example project, let’s create two simple AWS Lambda functions. One function will generate a random number, and the other will multiply it by 2.

// randomGenerator.js > Lambba 1
exports.handler = async (event) => {
  const randomNumber = Math.floor(Math.random() * 100);
  return {
    number: randomNumber,
  };
};

// multiplyByTwo.js > Lambba 2
exports.handler = async (event) => {
  const inputNumber = event.number;
  const result = inputNumber * 2;
  return {
    result,
  };
};

Step 3: Define the Serverless Workflow

Now, let’s define the serverless workflow using AWS Step Functions. The workflow will call the two Lambda functions in sequence, passing the output of the first function as input to the second function.

// workflowDefinition.json
{
  "Comment": "A Simple Serverless Workflow Example",
  "StartAt": "GenerateRandomNumber",
  "States": {
    "GenerateRandomNumber": {
      "Type": "Task",
       // Replace this with Lambda 1 ARN
      "Resource": "ARN_OF_GENERATE_RANDOM_NUMBER_LAMBDA_FUNCTION",  
      "ResultPath": "$.input",
      "Next": "MultiplyByTwo"
    },
    "MultiplyByTwo": {
      "Type": "Task",
       // Replace this with Lambda 2 ARN
      "Resource": "ARN_OF_MULTIPLY_BY_TWO_LAMBDA_FUNCTION",
      "InputPath": "$.input",
      "ResultPath": "$.output",
      "End": true
    }
  }
}

Step 4: Deploy the Step Function

In this code snippet, I am using the AWS SDK for JavaScript to deploy the AWS Step Function. The script reads the workflow definition from the workflowDefinition.json file (created in Step 3). It then calls the createStateMachine method from the stepFunctions object to create the Step Function state machine.

The params object contains the configuration for creating the state machine. The name of the state machine is “CBS” and provide the workflow definition JSON as the definition. The roleArn parameter should be replaced with the ARN of an IAM role that grants permissions for Step Functions to invoke your Lambda functions. This role should have the necessary permissions to execute the Lambda functions used in the workflow.

const AWS = require("aws-sdk");
const stepFunctions = new AWS.StepFunctions();

const workflowDefinition = require("./workflowDefinition.json");

const params = {
  name: "CBS",
  definition: JSON.stringify(workflowDefinition),
  roleArn: "ARN_OF_EXECUTION_ROLE",
};

stepFunctions.createStateMachine(params, (err, data) => {
  if (err) console.error("Error creating state machine:", err);
  else console.log("State machine created:", data.stateMachineArn);
});

After running the code, State Machine has been created 🙂

Step 5: Start the Serverless Workflow

In this code snippet, we use the AWS SDK for JavaScript to start the execution of the Step Function state machine. The script calls the startExecution method from the stepFunctions object to begin the workflow execution.

The params object contains two properties:

• stateMachineArn: Replace "ARN_OF_STATE_MACHINE" with the ARN of the state machine you created in Step 4 (CBS). This ARN uniquely identifies the Step Function state machine.
• input: This is the input data passed to the Step Function when starting the execution. In this example, we use an empty JSON object ({}) as the input, but you can customize it based on your workflow requirements.

const AWS = require("aws-sdk");
const stepFunctions = new AWS.StepFunctions();

const params = {
  stateMachineArn: "ARN_OF_STATE_MACHINE",
  input: "{}",
};

stepFunctions.startExecution(params, (err, data) => {
  if (err) console.error("Error starting execution:", err);
  else console.log("Execution started:", data.executionArn);
});

Conclusion:

Congratulations! You have successfully built an “Automated Serverless Workflow Manager” using AWS Step Functions. This project allows you to create, automate, and monitor complex serverless workflows seamlessly. You can extend this project to handle more intricate workflows and integrate additional AWS services as needed, Such as:

• Order Processing Workflow
• User Onboarding
• Serverless Microservices Orchestrator

You now have a powerful tool for building sophisticated, scalable, and automated serverless applications on AWS. Happy DevOps-ing 🙂