Serverless computing is an exciting technology and has many practical applications for building scalable applications in a cloud environment. One extremely practical use case is receiving messages in a messaging queue. In this article, I will be building an example to demonstrate this by connecting an SQS messaging queue to a Lambda function which can be accomplished quite simply by utilizing the Serverless Framework.

Large web applications have many moving pieces and some have functionality that is better performed behind the scenes rather than making users wait. One example of this is file processing. If you are accepting files from users and performing some type of processing on those files this can be time-consuming and not mention may have many steps to perform for a single transaction. For situations like this depending on how long the processing takes you may not want to perform the processing while your users are waiting.

This is a good opportunity to harness the power of messaging queues. With a queue in place, you can drop messages in with the info needed to perform the file processing or any other type of work to kick off. The other part of this process is you will need code running that will kick off every time there is a new message in the queue.

SQS

Simple Queue Service is a service offered by AWS that is a managed messaging queue. This means all you need to do is create it and AWS will handle everything else as far as deployment and scaling. We will be using the SQS service in this example to set up a queue to receive messages.

Lambda

Lambda is AWS’s serverless computing service. With Lambda you can set up code to run and pick from several different options for what triggers that code to execute. Lambda functions are very flexible and have many use cases.

Serverless Example

To build our example you will need to either have or create an AWS account. The only other prerequisite is that you will need to have NodeJS and npm already installed on your machine. If you need to install NodeJS check it out here.

We will also be using the Serverless Framework to build out our example. Runt the following command in a terminal to install the Serverless CLI locally on your machine.

$ npm install -g serverless

Now that the CLI is installed, run the following command in a terminal to kick off the CLI and bootstrap a new project.

$ serverless

The CLI will ask you a couple of questions to get create the new project. We will select ‘AWS Node.js’ when it asks what type of project do we want to create.

The last question will be the name of your project. I will call it sqs-serverless-example.

To set up your AWS IAM user credentials you can run the following command in a terminal.

$ serverless config credentials --provider aws --key YOUR_KEY_HERE --secret YOUR_KEY_SECRET_HERE

Note that this will set up those credentials in this file: ~/.aws/credentials and will look like the following example. So if you need to update these credentials later this is the file to do that.

[default]
aws_access_key_id=YOUR_KEY_HERE
aws_secret_access_key=YOUR_SECRET_KEY_HERE

We will now write the code to handle receiving new messages. This will execute every time there is a new message dropped in the queue. This is really just a stub with no real functionality for this example, but based on what is in the message body you could call an API, update a record in a database, or practically anything else you could think of.

Open up the handler.js file and update the code with the following contents.

Next, we need to update the serverless.yml to wire up this handler to a Lambda function and also create a resource with our SQS queue. Update this file with the following contents.

This file tells Serverless how to set up all of our resources on deployments. With our code in place, we can deploy this to our AWS environment with the following command.

$ serverless deploy

This will go through and set everything up so now we should be able to view everything we just set up in our AWS management console. We will start by opening up the SQS service in the management console so we can send a test message to our messaging queue.

We select fileProcessingQueue from the SQS list and then select the “Send and receive messages” button in the upper right corner.

In the test message body, we will put a simple JSON stringified object with a file ID.

"{"file_id":"0f28f8f8-edcd-4824-8f3c-d1b888dda9ce"}"

Hit “Send message” next and that should send our test message off into the queue. We can check to make sure this went through by next jumping into the Lambda management console.

From here we select our file receiver function from the list. We switch to the Monitoring tab and from that tab select View logs in CloudWatch.

In the CloudWatch logs, we should have a single entry to view. If you click on that log link you should be able to view the result of the test message we just sent from the SQS console which means our Lambda is doing exactly what we expect it to do. As new messages come into the queue our Lambda handler is processing messages off of the queue.

And with that test, we can call our demo a success. Once you’re done with the cloud resources, don’t forget to run the following command to tear down all of the resources we created in this project.

$ serverless remove

This is the simplest possible example of an SQS handler, but if you’re thinking of incorporating SQS and Lambda into your architecture I hope this article gives you a better idea of how that can be easily accomplished. Thanks for reading!