Since I was a tad over-excited while drafting out the previous post, I had to split it: it would’ve been just too much for one. Instead I tried to separate the two posts so the first one starts explaining what this is all about while the second (this) one goes a bit more into describing things you probably already familiar with:
- what I mean when I’m talking about “cloud”
- comparsion of the basic building blocks for computing
and one that’s not that obvious: why did I moved from one cloud to another.
This item is the second one of a series called Pajthy to AWS where I try to capture the process of migrating one of my open-source pet projects into a serverless setup in AWS.
Different “levels” of the cloud
When people talk about the cloud they could mean a lot of different aspects of the same thing. What I want to show you here is my categorization of the standard computing units you’ll find at your respective cloud provider; I’ll bring examples from AWS.
Before I go into details for the three categories there is one thing I think it’s important to say beforehand: none of the products of these categories are better or worse than the other one generally, however there is probably a best choice for your use case that will reveal itself after taking all the pros and cons into consideration. You know, at the end of the day, engineering is all about tradeoffs 🔧.
In the following examples I presume that all services are scaling automatically based on the load, eg. when the system detects that your computing units cannot take much more it will scale up new instances; also when there is no need for running so many it will automatically scale down some. Of course you could have a person to do this manually. It sounds like a fun job.
Virtual machines
Virtual machines (VMs) are just like your own computer at home, but in the cloud. You cannot press the physical power button but you have to click on the start button on the cloud provider’s website to start it. On AWS, these are called EC2 instances.
These kind of building blocks are easy to start with because people already have countless use cases for them: you can use them for anything (just install a Windows with an Excel then use it that anything). It’s easy for finding people to operate them since it needs the same skillset as any computer software did in the past 20 (or more) years.
There is one big drawback though: it does not scale well. Below there is a chart I spent almost an hour on (you couldn’t tell, could you?) that shows how many VMs would you need for our imaginary service to be able to deal all the requests it gets. For this example I simplified the issue: you can serve up to 200 requests per second with a single VM.
Do you see all that green area above the blue lines? We have to run the VMs and pay for all those idle resources or we’ll have to deal with service degradation; our imaginary service will start throttling requests if it’s out of resources. Users do not like throttling. Even worse, you cannot go under one machine, so even if you have a single (or less) rps you’d still need to run a whole VM.
Containers
Containers are like little virtual machines on your big virtual machines (at least from the aspect of this post, they are). You could have a single VM where you run containers, multiple ones, for different services - that way you might be able to use the idle resources with service B while there is no load on service A. This setup is called Elastic Container Service (ECS) at AWS.
Providers usually have an option if you don’t have a service B to take up all the idle processing time you have: they let you to provision the required number of containers and just have at it. At Amazon, this is called Fargate.
On the chart below a single container can deal with 25 requests per second.
As you see, the area of the idle green resource is much smaller in this graph; you could get even closer to the actual requirements by adjusting the resources of your containers. However keep in mind that while scaling up more containers is way faster than provisioning new VMs, it could still take several minutes for them to be able to take requests; it’s a good thing to be above that estimated load.
Functions
Cloud functions are just what you’d think they are: actual code chunks written in your favorite programming language that can take up parameters and give back a result. Behind the scenes these functions are wrapped in containers that can start up really fast (less than a second) and they will only run when there are ongoing requests for them. At AWS these are called Lambda functions and they get billed by milliseconds.
On the chart there is no overreaching green area since you only pay for the time when your function is actually running.
By “serverless” we mean this setup - there is not an always running server that you have to operate. You just need to respond to the incoming requests, the provider will take care of the rest.
Why change then if I was already “in the cloud”
Of course as we get smaller and more responsive, it’s causing more and more work for the cloud provider - having containers instead of VMs means a higher operational costs for them, having functions instead of containers is even higher; and yes, at the end of the day, you’ll be the one paying for that work too. There are other means of cost-optimization - eg. if you always need 3 VMs worth of containers then it might be cheaper to just have that three VMs, and serve the varying load from cloud-operated containers.
With Digital Ocean I have a single VM with containers on it - and I’ll probably stick to that setup in the future as well for developing software; the flexibility I can achieve with containers is worth the extra costs. Also we’re talking about machine that costs $5 a month…
But with pajthy I wanted learn new things and even if right now it’s a service with close-to-zero usage I want to be prepared for the overwhelming interest of people in the near future 🤞. And since all dependencies the app needs are available for any of the above listed computing units, my only angle left was cost-effectiveness.
In the following chart I tried to illustrate how the two kind of costs are relate to each other for the different computing solutions.
So in my case, I don’t mind the CapEx - hours spent on writing code - while I want to minimize OpEx - money spent for running the code. So lambda was the obvious choice.
With the next post I’ll start going into the migration details for the frontend service.