AWS Lambda Polling

What’s wrong with polling?

A lot of customers may require to implement the delivery of some long-running activities, like a query to a data warehouse or lake, or for a retail order fulfillment. They might get a polling solution developed just like the one shown below:

1. POST will first send a request.
2. GET will return an empty response.
3. A different GET will also return an empty response.
4. Yet a different other GET will return an empty response.
5. In the end, GET will return the data you had long been looking.

Hardships of traditional polling methods

• Unneeded chattiness and additional cost because of polling for result sets. Each time that your frontend tends to poll an API, it will add extra costs by leveraging infrastructure for computing the result. Empty polls are unnecessary and a total waste!
• Delay in your mobile battery life. A major contributor of apps associated with draining battery life is that of excessive polling.
• Hindered data arrival because of polling schedule. Part of the approaches entitled to polling contain an incremental back off for limiting the amount of empty polls, which may at times cause a delay between data readiness and its arrival.

Problems of Long polling:

• User request deadlocks are capable of delaying your application’s performance. Long synchronous user responses will lead to unintended user waiting times or UI deadlocks, that may impact mobile devices particularly.
• Leaking in memory and consumption might cause your app to shut down. Maintaining open long-running tasks queries can actually exhaust your backend and cause failure scenarios, bringing your application down.
• HTTP default timeouts over browsers: cause inconsistent experience for clients.

 

Creating a Poll for Job Status

This example project is going to create an AWS Batch job poller. It shall implement an AWS Step Functions state machine which utilizes AWS Lambda for creating a Wait state loop capable of checking on an AWS Batch job.

Sample project will do the following:

– Create and configure every resource

– Allows your Step Functions workflow to submit an AWS Batch job

– Keep waiting for that job to finish before finally ending successfully

Important Note

You are capable of implementing this pattern while not utilizing a Lambda function as well.

This example project is going to create the state machine + 2 Lambda functions + an AWS Batch queue, and configure all the related IAM permissions.

 

How to Create a State Machine and Provision Resources?

 

1. Go to the Step Functions consoleand select Create a state machine.
2. Select Sample Projects, then click on Job Status Poller.

State machine Code + Visual Workflow are shown.

Important Note

The Code section located under this state machine identifies the AWS resources that are going to get created for the project at hand.

3. Select Create Resources.

Create Project Resources page gets displayed, showing the resources that are going to be created. For the project at hand, the resources are going to include the following:

– SubmitJob Lambda function

– CheckJob Lambda function

– SampleJobQueue Batch Job Queue

Important Note

It is possible to take 10 minutes for those resources and related IAM permissions to get created. As the Create Project Resources page is displaying Creating resources, you may go ahead and open the Stack ID link for viewing the resources under provision.

Upon completion, you will get to the New execution page, having an example input that looks close to this.

{

“jobName”: “my-job”,

“jobDefinition”: “arn:aws:batch:us-east-2:123456789012:job-definition/SampleJobDefinition-343f54b445d5312:1”,

“jobQueue”: “arn:aws:batch:us-east-2:123456789012:job-queue/SampleJobQueue-4d9d696031e1449”,

“wait_time”: 60

}

How to start an Execution?

After finishing with the creation of your state machine, you may now start an execution.

For starting a new execution

1. From the New executionpage, fill in an optional execution name, then click on Start Execution.
2. For getting optional support in finding and identifying your execution, you are capable of specifying an ID for it inside the Enter an execution name In case you choose not to type an ID, Step Functions will automatically generate a unique ID.

Note

Step Functions provide you with the ability to create the following with non-ASCII characters:

– State machine names

– Execution names

– Activity names

The non-ASCII names are not capable of working with CloudWatch. For assuring that you have the possibility to start tracking CloudWatch metrics, select a name which merely utilizes ASCII characters.

3. It is possible to optionally head to the newly created state machine. From the Step Functions Dashboard, select New execution, then fill in the input code by utilizing the name or ARN of every one of the newly created resources.

As an example, below is shown the input for the previously mentioned execution by utilizing simply the resource names:

{

“jobName”: “my-job”,

“jobDefinition”: “SampleJobDefinition-343f54b445d5312”,

“jobQueue”: “SampleJobQueue-4d9d696031e1449”,

“wait_time”: 60

}

Important Note

wait_time orders the Wait state to begin looping each 60 seconds.

4. Click on Start Execution.

A newly created execution of your state machine will now begin, and a newly created page displaying your running execution is also shown.

5. -Optional- From the Execution Detailssection, select Info to check out the Execution Status + the Started and Closed
6. For viewing the changing status of AWS Batch job along with the looping results of this execution, select Output.

 

AWS Lambda EventSourceMapping

AWS lambda cost calculator