Home / Blog /

The Misconception of Auto-Scaling

27 April 2020 • 3 min read

Throughout my course of work, I met several people who have a misconception of auto-scaling; they believed that with auto-scaling, their websites would be safe from all traffic surges. When their websites experience high traffic surges and subsequently, became inaccessible, they learnt it the hard way. Auto-scaling had limitations, and believing that it can handle all traffic surges is a misconception.

Traffic Surges

Let us understand whether auto-scaling will work for each type of traffic surge. For simplicity, I have classified traffic surges into two types: gradual and steep. Gradual Traffic

The graph above is an illustration of a gradual increase in traffic. In this scenario, auto-scaling will help because there is time to scale-out new instances to serve the traffic, assuming the existing instances are able to handle the load during the time when the new instances are initializing. Steep Traffic

The graph above is an illustration of a steep increase (or sudden surge) in traffic. In this scenario, auto-scaling will not help because there is no time to scale-out, as new instances typically take at least 5 minutes to complete initialization. Note: Similar graphs (number of requests against time) can be retrieved from CDN (e.g. CloudFront) and monitoring / logging tool (e.g. CloudWatch).

Recommendations to handle steep traffic surges

1. Anticipate the traffic and scale-out in advance

If you expect a surge of traffic at a certain date and time, you may scale-out manually in advance by increasing the minimum instances of the auto-scaling configuration. This makes your infrastructure more resilient to sudden traffic surges.

2. Use Content Delivery Network (CDN) to offload traffic

A CDN is very useful in offloading most, if not all, of the traffic by serving content from cache. There will be lesser traffic hitting the origin (virtual machine instances). In addition, it also helps to defend against DDoS attacks. That being said, you need to configure your CDN properly so that all content are cached properly.

And one more thing...

If your website is undergoing a tech refresh, you may make use of the opporunity to leverage on Platform-as-a-Service (PaaS, e.g. Azure App Service, Google App Engine, Netlify) as part of your website architecture. You may use Serverless services, such as Lambda and API Gateway, to run your backend services as well. By leveraging on PaaS and Serverless, you do not need to worry about scaling-out to handle traffic surges; the platform providers will handle it for you.

TL;DR

Auto-scaling is useful for gradual traffic surge, but has its limitations when there is a steep traffic surge. Implementing a CDN to offload traffic and scale-out in advance are better ways to deal with steep traffic surges. If your website is undergoing a tech refresh, you may make use of the opportunity to leverage on Platform-as-a-Service and Serverless services as part of your website architecture.

CloudInfrastructure