Push and pull invocation models of Lambda functions

In the last post I mentioned that Lambda has two basic invocation paths: request-response and event modes.

When a Lambda function is invoked in request-response mode, the function is invoked immediately, and the response is returned (i.e. response.json from the referred post will have some entries). This invocation type is called synchronous invocation.

For the event mode, the invocation is placed in a queue (usually an internal SQS queue, which is not visible to us), and the function is invoked later. This makes it an asynchronous invocation, where there’s no immediate response (response.json is empty), and if the invocation is unsuccessful, Lambda retries it two more times.

But depending on the event source, the way Lambda functions are called can be different.

1. Two invocation models

Lambda function invocations are initiated by various event sources by other AWS services. Depending on the service triggering the Lambda function invocations, we can talk about two models of invocations, the push and the pull models.

Note that push and pull models can both result in event and request-response type invocations.

2. Push model

The push model describes the scenario when Lambda functions are invoked every time when events occur in another AWS service. In this case, the data source gives a signal to Lambda to start invoking the Lambda function with the new data record. That is, the event source pushes the new data record to Lambda, which will trigger the function invocation.

2.1. S3 events

One frequently used example for the push model is when an S3 event occurs.

If we have a bucket where a file is uploaded to or deleted from (these are the events), S3 can trigger the invocation of a Lambda function by sending the parameters of the event (bucket name, file name, event name etc.) to Lambda. The Lambda function can then apply some logic on the input, for example, it can create a thumbnail from an uploaded image.

The invocation type is event (asynchronous) for S3 events.

2.2. API Gateway

API Gateway is the other service, which is very frequently used with Lambda, such as a Lambda function is configured as the backend for API Gateway.

In this case, when an API Gateway event occurs, for example a configured endpoint is called with a GET method, API Gateway passes on the request to Lambda. The Lambda function returns a response, and this will be the response from the API which the client receives.

For example, an embarrassingly simple backend Lambda function can look like this:

exports.handler = async function (event) {
  const responseBody: {
    message: `Hello from ${event.path}!`
  }

  return {
    statusCode: 200,
    headers: {
      'Content-Type': 'text/plain',
    },
    body: JSON.stringify(responseBody.message)
  }
}

In this case, the return object will be passed on to API Gateway, and when the configured endpoint is hit, the responseBody object will be seen in the browser.

This invocation type is request-response.

2.3. CloudWatch

CloudWatch Events can be configured to send a trigger to Lambda when something interesting (from CloudWatch perspective) occurs.

For example, CloudWatch can be configured to trigger Lambda function invocation based on a predefined schedule. A good example is when a job needs to run once a day to do some fancy calculations. It might not worth paying for a server 24/7 to run a few-second job once a day, so this scenario is a good use case for using CloudWatch Events.

3. Pull model

The other invocation model is called the pull model.

With this, the events don’t come to Lambda and invoke the function but Lambda’s internal poll mechanism queries the data source, and the function is called with the new event records.

3.1. DynamoDB

If it comes to serverless database, then the solution will most probably be DynamoDB.

Lambda polls DynamoDB for new records, and forms batches from the results. An example is when there is a change in a DynamoDB table (either a new record or an updated one). The next time Lambda polls DynamoDB, it will notice the new record, and a function invocation is triggered with the new entry being the input for the function.

The invocation type in the case of DynamoDB is request-response.

3.2. Kinesis Streams

Amazon Kinesis Data Streams is the perfect solution for ingesting large number of data from hundreds of thousands of sources. Good use cases for Kinesis Streams are sensor data or click events.

Lambda polls Kinesis Stream once each second for each shard (a shard is a sequence of data records), and concurrently invokes a Lambda function for each shard.

A good example can be to ingest and analyze data from Uber-like services, where you can monitor the position of the vehicles real time and can read interesting facts about them (Kinesis Streams and Lambda are only a small part of the stack, though).

Kinesis Streams works with request-response invocation type.

4. Summary

Lambda has two types of invocation models, the push and the pull models.

The push model describes the scenario when an event source (for example, S3) notifies Lambda of any changes.

With the pull model, Lambda polls the data source at a high frequency, and invokes the function with any new record.

Thanks for reading, and see you next time.