Oops, Momento ate 60% of my Lambda latencies!

Conventional wisdom says DynamoDB is so fast that a Lambda application does not need a cache. I wanted to test that assumption. In this synthetic API, adding Momento took less than an hour and reduced client-side p999 latency by 60% for one lookup and 86% for a fan-out follower lookup.

Serverless has radically enhanced developer productivity. Its pay-per-use model offloads capacity management while saving money. It enables developers to focus on their core business instead of nitty-gritty operational details.

Caching creates a conundrum: as soon as you want to accelerate a serverless stack with a cache, you have to go serverful. Setting up a caching fleet can be painful, as CBS Sports experienced. Serverless caching changes that equation by improving availability, elasticity, and scale.

Local caches are not as effective on Lambda as they are on traditional servers that process hundreds of requests concurrently. Intuitively, I knew that a cache should be faster than DynamoDB, but I wanted to prove it. Momento’s serverless cache made the experiment simple: after I built the application, adding the cache and deploying the new Lambdas took less than an hour.

A small, synthetic test

The open-source demo includes a serverless application that you can build and deploy with the AWS Serverless Application Model (AWS SAM) CLI. Its benchmark script uses Locust to drive a small amount of synthetic traffic from your laptop. The application is a basic TypeScript REST API backed by DynamoDB, with Momento added as a cache. It emits metrics to CloudWatch for review on a dashboard.

Imagine a social network where each user has followers. The front-end application downloads each follower’s name to render on the device. The user model looks like this:

interface User {
    id: string,
    name: string,
    followers: Array<string>,
}

The Lambda application produces these CloudWatch metrics for comparison:

• Momento: momento-get and momento-getfollowers
• DynamoDB: ddb-get and ddb-getfollowers

I used the /bootstrap-users endpoint to generate 100 test users with 5 random followers each. I then exposed two pairs of endpoints.

1 lookup per request

GET /users makes 1 call to DynamoDB, while GET /cached-users makes 1 call to Momento. Both return a response like this:

{
  "id": "1",
  "followers": [
    "26",
    "65",
    "49",
    "25",
    "6"
  ],
  "name": "Lazy Lion"
}

I love using jq with JSON responses like this.

6 lookups per request

GET /followers and GET /cached-followers retrieve the passed user ID from DynamoDB or Momento. They then make N additional calls to the same service to look up each follower’s name. In this test, N is 5.

[
  "Angry Fish",
  "Lazy Otter",
  "Angry Sloth",
  "Clingy Sloth",
  "Dumb Lion"
]

The look-aside cache

The look-aside code first checks Momento. On a cache miss, it fetches the value from DynamoDB and stores it for the next request.

const getCachedUser  = async(userId: string): Promise => {
    let user = await getUserMomento(userId)
    if (!user) {
        console.log("no user found in momento fetching from DDB")
        user = await getUserDDB(userId)
        // Set item in cache so next time can get faster
        await momento.set("momento-demo-users", userId, JSON.stringify(user))
    }
    return user
}

const getUserDDB = async (id: string) => {
    const dbRsp = await ddbClient.send(new GetCommand({Key: {id}, TableName: "momento-demo-users"}));
    return dbRsp.Item as User
};

const getUserMomento = async (id: string) => {
    const rsp = await momento.get("momento-demo-users", id)
    const user = rsp.text()
    if (user == null) {
        return null
    }
    return JSON.parse(user)
};

What changed

Single-user lookups

Looking strictly at client-side latency, the single-user endpoint’s average dropped by about 43%, from 6.75 ms to 3.8 ms. Its p99 dropped by about 57%, from 29.7 ms to 12.6 ms. Its p999 dropped by 60%, from 70.7 ms to 27.9 ms.

Follower lookups

The difference grew for the Lambdas that required multiple calls. Average client-side latency dropped by about 53%, from 16.5 ms to 7.61 ms. The p99 dropped by 72%, from 72 ms to 19.5 ms. At p999, response time fell by 86%, from 536 ms to 72.3 ms.

What I learned

• The largest change was in the tail. In less than an hour of work, Momento dropped my p999 latency by more than 86%. That means lower Lambda costs, happier users, and a more scalable system without worrying about hot keys or hot partitions in DynamoDB.
• The cache was quick to add. Once the DynamoDB-backed Lambdas were set up, each Momento endpoint took less than 5 minutes. That included creating the cache, adding the look-aside pattern, and deploying the function.
• Fan-out makes tail latency matter even more. The results offer an early indication that multiple DynamoDB calls can pull even average Lambda latency toward the tail of those calls.

Want to test the same path? Follow the hands-on tutorial in the demo repository to build the API and run the synthetic benchmark.