We are happy to announce that we have released updated cloud pool APIs and implementations, making cloud pools more easy to work with and adding support for features related to application health checking that should benefit all our users. Head over to scale.cloudpool github repository and download the current cloud pool implementations for OpenStack, Amazon Web Services (EC2, Spot Instances, and Auto Scaling Groups) and enjoy the updates!
Cloud pools in a nutshell
Cloud pools offer a very handy layer of abstraction while working with different cloud providers. They request new VMs and shut down ones you no longer need in in an intelligent way, allowing you to merely state how many VMs of the configured type are desired at this point in time. The cloud pool will handle starting and stopping VMs for you, including the nice feature that you can ask it to keep a VM around until just before it starts its next billable hour. Why shut down if you have already paid for it?
If a VM is terminated by the cloud infrastructure, and the desired size was not decreased by the operator, the cloud pool will immediately replace the wrongfully terminated VM. This gives peace of mind: if we have asked for X VMs running, the cloud pool will do its very best to deliver X VMs at all times.
New in version 3 of the cloud pool API
The API is now fully asynchronous. Set the desired size of your deployment to a given value, and the call returns immediately. The cloud pool will then resize your deployment accordingly, as made possible by the cloud infrastructure.
We separate machine states from membership status and service states. Machine states are what cloud infrastructure providers offer: they tell if you if the VM is running on their infrastructure or not. But merely having it running on the infrastructure does not mean that it is operational! For that reason, we offer additional states, to keep track of a higher level of states, related to how the VM is doing its job and what its role is in the deployment:
- Membership status is used to indicate to the cloud pool that a certain machine needs to be given special treatment. The membership status can, for example, be set to protect a machine from being terminated (by setting its evictability) or to mark a machine as being in need of replacement (by setting its activity flag). This allows one to, for example, isolate a failed machine for further inspection and to provision a replacement to sustain sufficient capacity. It also allows us to have “blessed”/”seed” pool members that may not be terminated by the cloud pool.
- Service state lets us handle cases where we need to be able to reason about the operational state of the service running on the machine. For example, we may not want to register a running machine to a load balancer until it is fully initialized and ready to accept requests, and we may want to unregister unhealthy machines. To this end, a cloud pool may include a service state for a machine.
Whereas the machine state should be viewed as the execution state of the machine, the service state should be viewed as the operational health of the service running on the machine. Service states have no semantic implications to the cloud pool. They should be regarded as informational “marker states” that may be used by third party services (such as a load balancer). Membership status controls how the cloud pool handles a given instance. If a stateless simple frontend VM experiences issues, perhaps it is easiest and most reasonable to just replace it without further investigation, but VMs running more complicated logic might warrant further inspection, so they should be kept around, but errors should cause exclusion from the set of operational machines and replacements should be started to keep performance acceptable.
Running VMs can now be attached or detached from a cloud pool, which enables a currently deployed application to with with a newly started cloud pool.
All Java, all the way
We have retired our Python cloud pool implementations due to maintenance costs. A single Java codebase, where most of the functionality is shared between each specific cloud infrastructure implementation reduces the number of possible bugs, and makes it faster to develop new and exciting features.
Obtaining and running cloud pools
The code for our cloud pools API and implementations is open source (Apache 2), and available right now from our scale.cloudpool github repository. Instructions for how to download, build, and run the code is given there. We have added support for building Docker images for each cloud pool implementation, to make it easier to run them. We are finishing details on our Docker transition, and when that work finishes, you will find them on Docker hub as well.