Persisting Express session in DynamoDB container

16 May 2019
10 min read
Node.js
One way to authenticate users is to establish a session, and store the session id in the browser cookie. If the session is not persisted, a server crash will invalidate it, and the user has to start over it again, which might not be the best user experience. One option to store (persist) session data is to use DynamoDB, which is very fast and provides a flexible database structure.

DynamoDB is a highly available NoSQL database, which supports key/value and document type data. I wrote more about DynamoDB in this post, so I won’t cover these concepts here. If you are not familiar with the basics of DynamoDB, please read that post first.

1. Pre-requisites

The following tools have to be installed on your machine (other than Node.js) if you want to recreate this mini-project:

You’ll also need to configure the AWS CLI for the live access, which sets up the credentials in a file called credentials, and places it in the .aws folder.

2. Add DynamoDB locally

Luckily, it’s possible to set up DynamoDB locally through a Docker image, so we don’t have to connect to AWS and use their service for testing our code. Everything we’ll do here can be reproduced on AWS by replacing the local endpoint in the configuration object with the real one.

In order to set up the database container, we can create a super simple docker-compose.yml file in the root folder:

version: '3'

services:
  db:
    image: amazon/dynamodb-local
    ports:
      - '8000:8000'

The name of the image we run the container off of is called amazon/dynamodb-local, and it will listen to port 8000.

3. Install the packages

Let’s start by creating a project folder:

mkdir session-with-dynamodb && cd session-with-dynamodb

From the project folder, type npm init -y. This command will create the package.json file, and it’ll also skip the questions npm asks when the folder is being set up.

Next, we’ll install the packages:

npm i express express-session aws-sdk connect-dynamodb

Let’s briefly talk about these dependencies.

Express is the most popular web framework for Node.js. It’s widely used, and has a rich interface for creating web servers and API endpoints.

The session data will be provided by express-session, and it’ll be incorporated into the code as middleware.

AWS SDK provides a Node -wrapper around the AWS services.

Finally, Connect DynamoDB provides the DynamoDB session store.

4. Add the code

Now that the dependencies are installed, let’s create the server.js file in the root of the project folder. The file can look like this:

const express = require('express')
const session = require('express-session')
const AWS = require('aws-sdk')
const DynamoDBStore = require('connect-dynamodb')(session)

const app = express()

AWS.config.update({
  region: 'us-west-2',
  endpoint: 'http://localhost:8000'
})
const dynamodb = new AWS.DynamoDB()

const PORT = 3000

app.use(session({
  secret: 'my-secret',
  resave: false,
  saveUninitialized: false,
  store: new DynamoDBStore({
    client: dynamodb,
    table: 'session'
  })
}))

app.use((req, res, next) => {
  if (!req.session.views) {
    req.session.views = 0
  }
  next()
})

app.get('/private', (req, res) => {
  const numberOfViews = ++req.session.views
  res.send(`The /private page has been visited ${ numberOfViews } times.`)
})

app.listen(PORT, () => {
  console.log(`Server is running on port ${ PORT }`)
})

Here’s the step-by-step explanation of the code.

4.1. Session

We start by require-ing the dependencies at the top of the file, and then we initiate the app by calling the express function.

Next, we update the AWS configuration. The compulsory properties are the region (it won’t be used, because we’ll do everything locally, but it’s necessary for the live connection) and the endpoint, which is localhost:8000 (this needs to be replaced with the real DynamoDB endpoint in production). This is the port on our machine that the DynamoDB container will open up for us.

After initiating DynamoDB and declaring the PORT, we’ll create the session middleware using app.use.

express-session has a bunch of properties, and only a fraction of them is used here.

The secret is usually stored as an environment variable, here we’ll be fine with some utterly secure secret. resave and saveUninitialized are set to false, they both help preventing race conditions from happening when the client makes parallel requests.

The store property is very important: This is where we can define the database that stores the session. We define the dynamodb instance as the client (i.e. the database which stores the session data), and give a name to the table (session) for the session. The default table name is sessions, but I thought it would be cool to change it, and demonstrate the usefulness of the table property.

With this, we have set up the session store. There’s no need to manually create the table, because connect-dynamodb will do this for us.

4.2. Endpoint and server

We can create another middleware with app.use, and this will be a simple counter, which counts the number times our endpoint is hit. This is done by creating a property called views in the session object (which will be added by express-session), and set its initial value to 0.

It’s important to call next inside the middleware. If we miss this step, the code flow will stop here, and the server will simply hang. When next is called, the control is passed on to the next line in the file.

This very next line is the only endpoint we define here (/private). It could be anything else, feel free to play around with it and add more endpoints. Each time the /private endpoint is hit, we’ll increase the number of views by one, and the current number of visits will be returned in the res.send method.

Finally, we start the server by calling the listen method.

5. Run the code and test it

We can now start the server and do some tests!

5.1. Start the database and the server

Let’s run the DynamoDB container in the terminal, and start the server in another terminal:

docker-compose up

and

node server.js

When the server is started, connect-dynamodb will automatically create the session table using the create-table command behind the scenes. We’ll receive some pieces of really useful information about the session table, for example, the time and date of creation or the status of the table (ACTIVE). A very important key is the ItemCount, which is initially equal to 0. This comes as no surprise as we haven’t created any items (i.e. no sessions have been initiated) yet.

5.2. Test the code and use AWS CLI to query the table

We can use the AWS CLI to make some queries. First, let’s list all tables (it’s not hard to count them, there should only be one). In another terminal, run the following command:

aws dynamodb list-tables --endpoint http://localhost:8000

This will return all tables for this endpoint (which points to the DynamoDB Docker container). The list is not particularly long:

{
  "TableNames": [
    "session"
  ]
}

So let’s open the browser, and type localhost:3000/private in the address bar. The message on the screen should be The /private page has been visited 1 times..

This means that the counter started to work, and the session data should already be saved to the DynamoDB session table.

We can quickly check this by running the following command from the terminal:

aws dynamodb describe-table --table-name session --endpoint http://localhost:8000

The describe-table command has another compulsory option called table-name, and we have to name the table we want to learn more about. In our case, this table is session.

The return value should be similar to this:

{
  "Table": {
    "AttributeDefinitions": [
      {
        "AttributeName": "id",
        "AttributeType": "S"
      }
    ],
    "TableName": "session",
    "KeySchema": [
      {
        "AttributeName": "id",
        "KeyType": "HASH"
      }
    ],
    "TableStatus": "ACTIVE",
    "CreationDateTime": 1557746582.467,
    "ProvisionedThroughput": {
      "LastIncreaseDateTime": 0.0,
      "LastDecreaseDateTime": 0.0,
      "NumberOfDecreasesToday": 0,
      "ReadCapacityUnits": 5,
      "WriteCapacityUnits": 5
    },
    "TableSizeBytes": 161,
    "ItemCount": 1,
    "TableArn": "arn:aws:dynamodb:ddblocal:000000000000:table/session",
    "BillingModeSummary": {
      "BillingMode": "PROVISIONED",
      "LastUpdateToPayPerRequestDateTime": 0.0
    }
  }
}

These details are very useful, and many of them are covered in the DynamoDB post I referred to earlier. But the most relevant part of the data is the ItemCount, which shows 1 instead of 0, that is the session data have been successfully stored.

If we refresh the browser, we’ll see that the counter increments the number of visits.

Let’s quickly open the developer tools in the browser, and go to Application, and Cookies. express-session placed a cookie called connect.sid in the browser, so the session will be kept if the browser is refreshed.

On the server side the cookie is stored in the req.session object.

5.3. Test if the session is persisted - it should be

Let’s stop the server now, and restart it, then refresh the browser again. With this, we can simulate a server crash.

If the session wasn’t persisted, the counter would be reset to 1, and a new session id would be placed in the cookie. But we don’t see this; instead, the number of visits has been kept, and has incremented again.

Persisting the session is useful in case of a server crash. The user won’t have to restart everything by logging in again, which is great!

It’s also possible to set an expiration in express-session, which we are not doing now. If you are interested, please read the documentation about the available options.

Let’s run the AWS CLI command again:

aws dynamodb describe-table --table-name session --endpoint http://localhost:8000

The value of ItemCount is still 1, because the same user with the same browser is connected to the server (although the server has been crashed/restarted).

5.4. Add a second session

Now from a different browser (or using curl), let’s connect to locahost:3000/private again, and simulate a new user connecting to the server.

The return message should be The /private page has been visited 1 times. again, because it’s a new connection, and that user hasn’t visited the page yet.

Run the AWS CLI command again. Guess what will be the value of ItemCount:

aws dynamodb describe-table --table-name session --endpoint http://localhost:8000

That’s right, it will be 2, because a new session has been initiated.

And that’s it, this is how we can store and persist session data in DynamoDB. Now we can stop the container (docker-compose down) and the server (Ctrl + C).

6. Conclusion

One of the use cases of AWS DynamoDB is to store web session data as DynamoDB is fast and highly available. It’s possible to test DynamoDB locally by using a Docker image and running a container off of it.

The express-session library is responsible for establishing the session, and placing the session id in the cookie. But, by default, it’s an in-memory store. If we want to persist it, we need to use one of the connect modules - in this case, it’s connect-dynamodb.

Thanks for reading, and see you next time.

Previous post
Next post