A rapid traffic spike once meant user downtime and a long night. Even early cloud adopters ran most applications and infrastructure on self-managed EC2 instances.
Today’s tools are miles ahead of where they were a decade ago. AWS Lambda offers serverless, pay-per-use compute that can scale to infinity and run only when needed. Serverless databases such as Amazon DynamoDB and fully managed systems such as Google Spanner and MongoDB Atlas can dynamically handle any workload we throw at them. CDNs such as Cloudflare can speed up response times and block DDoS attacks of more than 17 million requests per second.
Any developer with a credit card and an idea can use these services. They provision in seconds, scale dynamically with an application’s needs, and take advantage of the modern cloud’s elasticity.
Yet somehow our caches are stuck in the 20th century. They’re slow to provision and require careful cluster planning or extreme overprovisioning to avoid incidents. Their unpredictable tail latencies can hurt users or take applications down. They also charge full price for instances even when utilization is low.
Momento built its serverless cache service to close that gap. It’s a cache for the serverless, cloud-enabled era: one that doesn’t slow you down while you develop a new idea and holds up during peak demand.
Instance planning creates a false choice
With instance-based provisioning, you must select and manage a specific virtual instance. That means choosing its CPU, RAM, disk, and network configuration. You also need to monitor low-level system metrics and plan for failovers, redundancy, and backups.
This is a datacenter with an API. It improves on running your own datacenter, but you remain responsible for incidents that affect your instances and cause application downtime.
Inelastic infrastructure also forces an unwelcome tradeoff. You can overprovision the cache for peak traffic and pay for idle capacity. Or you can underprovision to save money and leave users with the consequences when the cache can’t keep up.
The modern infrastructure we love does not depend on individual instances. It is resource-based infrastructure.
Shared resources make dynamic scaling possible
With resource-based infrastructure, a provider shares a giant pool of resources across many customers in a datacenter. For AWS Lambda, that means a giant pool of worker instances hosting customer functions. For Amazon DynamoDB, it means region-wide request-routing and metadata services, along with a pool of storage instances that manage customer data.
Sharing infrastructure at that scale provides a level of dynamism and resiliency that instance-based infrastructure cannot match. A single customer can’t send enough requests to take down a region’s DynamoDB request router. Nor can one customer overload a region’s AWS Lambda worker pool.
Designing around shared resources also changes how providers handle failure. At this scale, instance failure is a fact of life, and individual failures can affect multiple customers. Providers must design for failover and enforce API limits that prevent one customer from affecting another.
Momento is the first cache to use resource-based infrastructure. This cloud-first design lets it scale dynamically without preprovisioning or coarse-grained “autoscaling” policies.
For users, that means less time in Excel spreadsheets, estimating capacity from clumsy load tests. It means fewer outages from unexpected traffic spikes. The cache should just work instead of getting in the way.
Usage-based pricing follows the same model
Resource-based infrastructure does more than help Momento handle traffic fluctuations. It also supports pricing that aligns more closely with the value you receive.
With instance-based infrastructure, you pay for provisioned instances regardless of use. If your users are sleeping and utilization is in the single digits, you still pay the instance’s full rate. Because instances are harder to scale up and down, you are also likely to remain overprovisioned in anticipation of traffic peaks.
Momento charges for the resources you use. You pay per GB of data written to or read from Momento. That’s it: no always-on monthly cost, data-tiering calculations, or memory limits.
This model connects the price you pay more closely to the value you receive. It also makes Momento Caches practical in staging and developer environments because ephemeral tests do not incur an always-on infrastructure charge.
Measure cache performance from the client
Billing, scaling, and management matter, but caches are ultimately about performance. An application needs ultra-low latency when traffic is highest. It also needs enough predictability and consistency for effective planning.
Momento’s performance testing focuses on two metrics:
- Throughput, measured in requests per second (RPS)
- Client-side latency, measured in milliseconds (ms) to receive a response
Client-side latency measures how long the application waits for a response. Server-side latency misses network variability and the effect of cross-AZ traffic. Network and connection behavior are unavoidable parts of data access, so the client measurement shows when cached data becomes available to the application.
Latency is a distribution, usually reported at a specific percentile. Momento believes tail latencies matter, so it focuses on p999 latency: the 99.9th percentile. Only 0.1% of measured cache requests are slower than that value. This measurement applies to requests to the cache. If one web request makes several cache requests, more than 0.1% of users may experience a slower request.
Momento customers regularly load test the service and consistently report client-side p999 latency below 5 ms. The team also completed a routine 500K RPS soak test with client-side p99 latency of 2.5 ms.
Momento is ready to handle caching needs at a scale that other caches have a hard time matching.
Bring the cache into the cloud era
Cloud infrastructure has progressed by replacing instance management with resource-based services. Momento applies that model to caching with dynamic scaling, pay-per-use pricing, and performance measured from the client.
Try Momento Cache to provision a cache in seconds and integrate it with your application in five lines of code.
