Be ready to scale

It makes sense to only use the resources you need to save on costs, and in my last post I talked about using only a single node for your entire stack, so for example your web server, web application and and database all running on one node.

Hopefully, sooner rather than later, your system will require to scale horizontally as you have exceeded all the resources available on a single node. To make the process as painless as possible you should use a configuration management tool that allows modularity. I personally prefer Puppet and it allows you to create modules with classes that can take parameters.

When setting up your single node you can create a module per piece of your system, so you could have a database module, and a web server module and a app server module etc.  You will probably also have modules for the software your system is build on, Nginx, PostgreSql for example. In order to distinguish the app stack modules I prefix them with the name of my system. So I will have

mysystem-db
mysystem-app
mysystem-web.

Each of these modules is essentially a class that can take parameters  such as database connection string's host names etc that then can be used in the module directly or via its template when generating configuration files. When you have one node most of the pertinent parameters will be pointing to the same host or localhost.  However once you want to scale out you can easily pass in the new database host for example.

Once you have these building block modules you can have different server roles that you puppet main site script can check and then apply the particular modules.  On my initial node I would have say a primary role that would just apply all the modules with parameters pointing to localhost.  Then one I want a separate database server for example I would have two roles, once webapp and one database. The webapp would apply the mysystem-app and mysystem-web and the database role would apply the mysystem-db.

The other advantage of having puppet being able to setup your single node from a vanilla OS is that you can recover from disaster much quicker than if you have set everything up manually.

You can see from my previous posts I prefer to use a puppetmasterless setup and have me push updates with git rather than the puppetmaster, handling it.  But this begs the question how does puppet know which node role to apply?  This is easily achieved using a few more advanced bits of puppet and a special module we build that discovers the role.  We seed the role with out bootstrap shell script which is passed as a parameter to it. I will reveal all in another post.

Feel free to leave comments or ask questions if anything it unclear.

Scale when you need to....

In my post one node I talked about a kind of scaling within a single node (a server), where you run multiple instances of you service and let a load balancer round robin the requests.  This is a sort of horizontal intra-node scaling that helps keep your service handling requests and if one of your instances dies then you can still rely on the other ones to keep serving.

If you found that the load was becoming to high you could launch more instances, but you will come to a point where this doesn't really help due to resource problems, such as CPU/memory or IO saturation.  It is tempting at this point to start to architect an elaborate multi-node setup ripping your application stack into layers. For example having 3 separate database nodes with replication and 4 app server nodes with auto scaling and a few load balancing nodes, maybe a few memcache nodes. However you will soon disappear down a rabbit hole of complexity and stress,  unless you have setup a similar infrastructure before.

Keeping everything on one node means all the parts of your application are as close as possible and keeps configuration to a minimum.  Scaling vertically by adding memory/cpu cores and better IO will keep you going and most cloud providers offer big fat nodes for you to grow into.  Of course this can only go so far and eventually you will have to spin up another node, but you shouldn't do this until you need to. 

There are usually a few arguments against running on a single node (assuming your service can run well on a single node).

One is having only one node puts you at risk of outages as you have no fallback if your server goes bye bye. Another is when it does come time to scale horizontally by adding more nodes you will have to pick apart you application stack, causing a delay in getting the load under control.

These arguments are valid however they can be easily overcome by making you configuration modular and being able to recreate your node from scratch in a few minutes as well as using a decent VPS service, that has good uptime guarantees. So you can keep things simple and scale as and when you really need to.

I will go into these in more detail in my next post.